Actor picture

Crawler Results Deduplicate

petr_cermak/crawler-results-deduplicate

This acts takes crawler execution results and deduplicates them.

No credit card required

Author's avatarPetr Cermak
  • Modified
  • Users6
  • Runs19
Actor picture
Crawler Results Deduplicate

This acts takes crawler execution results and deduplicates them.

Example input:

{
    "_id": "EXECUTION_ID",
    "data": "{
        \"compareKey\": \"YOUR_COMPARE_KEY\"
    }"
}

The optional "data" attribute must be a stringified JSON and can contain a "compareKey" attribute. This key is used to select an object attribute for comparison. If no key is provided, whole object will be used for comparison. It is expected to be called from a crawler webhook.

Industries

See how Crawler Results Deduplicate is used in industries around the world