Crawler Results Deduplicate avatar
Crawler Results Deduplicate

Under maintenance

Pricing

Pay per usage

Go to Store
Crawler Results Deduplicate

Crawler Results Deduplicate

Under maintenance

Developed by

Petr Cermak

Petr Cermak

Maintained by Community

This acts takes crawler execution results and deduplicates them.

0.0 (0)

Pricing

Pay per usage

1

Total users

10

Monthly users

1

Last modified

4 years ago

act-crawler-results-deduplicate

This acts takes crawler execution results and deduplicates them.

Example input:

{
"_id": "EXECUTION_ID",
"data": "{
\"compareKey\": \"YOUR_COMPARE_KEY\"
}"
}

The optional "data" attribute must be a stringified JSON and can contain a "compareKey" attribute. This key is used to select an object attribute for comparison. If no key is provided, whole object will be used for comparison. It is expected to be called from a crawler webhook.