This is looks what I need but I am not sure how to call this task on a schedule run dynamically so I can keep passing references to the datasets IDs as I run them and merge all this calls into a single dataset

thanks!

Míša Fialová (misa)

Hello! You may want to check out this blog post for guidance on using the scraper: https://blog.apify.com/how-to-scraped-data-deduplication/ It should provide valuable insights. If you encounter any difficulties during setup, don't hesitate to reopen the issue for further assistance.

Add comment

Save To S3

drinksight/save-to-s3

Designed to be run from an ACTOR.RUN.SUCCEEDED webhook, this actor downloads a task run's default dataset and saves it to an S3 bucket.

Richard Weaver

Forward dataset as POST data

anchor/forward-dataset-webhook

This actor forwards the results of an Actor to an endpoint, instead of having to fetch the results manually. It will download the dataset and attach it to the body of a POST request you will specify. It acts as a new webhook. Simplify your Actor process !!!

Anchor

Dataset Query Engine

jiri.spilka/dataset-query-engine

Use natural language queries to retrieve results from an Apify dataset. This Actor provides a query engine that loads a dataset, executes SQL queries, and synthesizes results.

Jiří Spilka

4.6

API / JSON scraper

pocesar/json-downloader

Scrape any API / JSON URLs directly to the dataset, and return them in CSV, XML, HTML, or Excel formats. Transform and filter the output. Enables you to follow pagination recursively from the payload without the need to visit the HTML page.

Paulo Cesar

540

Power Webhook Integration

pocesar/run-webhook-digest

Allows you to provide multiple HTTP endpoints, that receive a more complete JSON from the run, and allow you to hit those endpoints using a proxy, and enable you to do conditional webhook calls with some lines of Javascript code and you can link/chain one actor to another

Paulo Cesar

Example Process Crawl Results

apify/example-process-crawl-results

Iterates through all results from a crawler run and count them. Needs to be called from the crawler's finish webhook by adding an URL to finish the webhook of your crawler. Use this actor as a starting point to develop custom post-processing of data from the crawler.

Apify

4.5

Actors MCP Server

apify/actors-mcp-server

⚠️ Legacy: This Actor is outdated. For the latest features and full documentation, visit https://mcp.apify.com. Easily connect any Apify Actor to AI agents using Anthropic’s Model Context Protocol (MCP) with our actively maintained MCP server.

Apify

4.9

Torrent Scraper

ondrejklinovsky/torrent-scraper

Scrape information about torrents from popular torrent sites. Download the data in JSON, CSV, and Excel.

Ondrej Klinovský

148

Google Sheets Import & Export

lukaskrivka/google-sheets

Import data from datasets or JSON files to Google Sheets. Programmatically process data in Sheets. Easier and faster than the official Google Sheets API and perfect for importing data from scraping.