Under maintenance

Pricing

Pay per usage

Try for free

Go to Store

Crawler Results Deduplicate

Under maintenance

Try for free

Developed by

Petr Cermak

This acts takes crawler execution results and deduplicates them.

0.0 (0)

Pricing

Pay per usage

Total users

Monthly users

Last modified

4 years ago

Developer tools

Open source

You can access the Crawler Results Deduplicate programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

$echo '{}' |
<apify call petr_cermak/crawler-results-deduplicate --silent --output-dataset

Crawler Results Deduplicate API through CLI

The Apify CLI is the official tool that allows you to use Crawler Results Deduplicate locally, providing convenience functions and automatic retries on errors.

Install the Apify CLI

$npm i -g apify-cli
$apify login

Other API clients include:

Crawler Results Deduplicate API in Python

Crawler Results Deduplicate API in JavaScript

Crawler Results Deduplicate OpenAPI definition

Crawler Results Deduplicate API

Execution To Xlsx

petr_cermak/execution-to-xlsx

The act takes a JSON input containing a crawler executionID, downloads the results and converts them to XLSX.

Petr Cermak

Microsoft SQL Server Insert

petr_cermak/MSSQL-insert

This actor takes a crawler execution and inserts its results into a remote MSSQL database.

Petr Cermak

Example Process Crawl Results

apify/example-process-crawl-results

Iterates through all results from a crawler run and count them. Needs to be called from the crawler's finish webhook by adding an URL to finish the webhook of your crawler. Use this actor as a starting point to develop custom post-processing of data from the crawler.

Apify

Execution To Knack

petr_cermak/execution-to-knack

Act for inserting crawler execution results into Knack database.

Petr Cermak

MySQL Insert

petr_cermak/MySQL-insert

This act takes a crawler execution and inserts it's results into a remote MySQL database.

Petr Cermak

PostgreSQL Insert

petr_cermak/PostgreSQL-insert

This act takes a crawler execution and inserts it's results into a remote PostgreSQL database.

Petr Cermak

Icon Burglar

yuri/Icon-burglar

This act takes the URLs prepared by a crawler and downloads the images to finally zip them

Yuri Manders

Send Legacy PhantomJS Crawler Results

drobnikj/send-crawler-results

This actor downloads results from Legacy PhantomJS Crawler task and sends them to email as attachments. It is designed to run from finish webhook.

Jakub Drobník

Kijiji Crawler

service-paradis/kijiji-crawler

The Kijiji crawler is a web scraping tool that allows you to extract data from the Kijiji selling platform. With this crawler, you can get information search results and local advertisement details.