Universal Web Structured Extraction & Change Intelligence

Crawl public URLs with HTTP-first extraction, emit clean structured page records, fingerprints and field-level change intelligence without requiring custom JavaScript.

Why use this Actor

General web scrapers are flexible but often require code or prompts, while recurring workflows need stable structured outputs and explicit change decisions. This Actor converts structured source data into a decision-ready result so recurring workflows do not need a separate manual analysis step. It supports repeatable monitoring, scheduled execution, dashboards, and agent workflows. Scores are deterministic and traceable to observable input fields.

Key features

• No-code URL input with HTTP-first structured metadata and text extraction.
• Stable content fingerprints and previous-snapshot field diffs.
• JSON-LD, headings, links, email and social-link extraction in one schema.
• Automatic MONITOR/REVIEW_CHANGE actions for recurring workflows.
• Dataset input mode for normalization of upstream crawls and 256 MB operation.
• Structured Dataset output for API, Task, schedule, and agent use.
• Current-versus-previous snapshot analysis when prior records are supplied.
• Limited-permission, data-first execution design.

Input

The JSON input schema includes explicit fields and prefilled examples. Important fields include currentItems, currentDatasetId, previousItems, previousDatasetId, queries, urls, directFetch, language, countryCode, maxResults, maxItems. Inline records are convenient for small tests. Dataset identifiers are better for recurring pipelines because one Actor can consume the structured output of another without copying rows manually. Stable entity identifiers improve snapshot comparison.

Output

Results are written to the default Apify Dataset as structured JSON. Output fields include ok, sourceType, analysisKind, entityId, signalScore, changeScore, agentAction, agentReason, metrics, changes, observedAt, source. These fields are designed for dashboards, databases, automation tools, and downstream Actors. Decision fields summarize observable evidence instead of returning only a large unranked export.

Example

Run the prefilled example once in the Apify Console. For a recurring workflow, save the current Dataset or upstream Dataset ID and provide a later snapshot on the next run. Compare the emitted scores, changes, and action fields. Through API or public Tasks, keep the same field mapping between runs so time-series comparisons remain meaningful.

Use cases

• recurring competitive monitoring.
• AI-agent and automation workflows.
• market research and prospect prioritization.
• period-over-period change detection.
• Scheduled daily or weekly monitoring with repeatable decision logic.
• Agent workflows that need compact actions rather than raw rows.
• Research pipelines that need stable JSON and traceable signals.

Pricing

The Actor uses pay-per-event pricing with one primary measurable result. The base design price is USD 0.0025 per primary result before Apify tier discounts. There is no mandatory external AI subscription in the deterministic scoring path. Platform usage varies with run duration, memory, and data volume. The release gate checks margin from real validation costs before publication.

Limitations

• Only public or appropriately licensed data should be supplied or fetched.
• Instagram and TikTok direct public-URL mode is best-effort; Dataset input is the stable mode when those sites block anonymous requests.
• Signals are deterministic decision support, not guarantees.
• Public-source availability and upstream schemas can change over time.
• Missing source fields can reduce the strength of a score or comparison.
• Decision scores are prioritization signals, not promises of commercial or operational outcomes.
• Users remain responsible for source access rules and platform terms.

Operational notes

For production use, prefer scheduled runs with stable input definitions. Review the Dataset after the first production run before connecting downstream systems. Preserve raw snapshots when a source changes schema so transformations remain auditable. Market evidence and competitor capabilities are rechecked before publication because Store conditions can change.