No credit card required

Dataset Image Downloader & Uploader

lukaskrivka/images-download-upload

No credit card required

Download image files from image URLs in your datasets and save them to a Zip file, Key-Value store, or directly your AWS S3 bucket.

2023-03-13

Add two new option items and outputDatasetId to input.

2023-02-07

Rewrite to crawlee, and overall updates.
Added a new option zip-file for uploadTo dropdown to upload a zip file of all the images to the dataset.
Fixed issue with downloading images.

Version-2 - 2019-12-02

Input has been completely redesigned. Please read it again and adjust.
Removed integration for cancelled Apify Crawler product.
Added automatic integration with actor webhooks.
Removed continueRunId from input. Use actor resurrect feature instead.
batchSize can have now arbitrary size. Previously it was limited to max dataset load (250,000).
Removed maxItems. You can use predownloadFunction to manipulate input data.
Added stateFields for a possibility to clean log and decrease memory usage of big batches.
Added duplicatesIndexes to state object for abillity to track duplicates.

Developer

Lukáš Křivka

Actor metrics

12 monthly users
76.0% runs succeeded
4.3 days response time
Created in Nov 2018
Modified about 1 month ago

Categories

Developer tools

Automation

Integrations

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

62.2k

Website Content Crawler

apify/website-content-crawler

Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.

Apify

12.6k

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

4.1k

Content Checker

jakubbalada/content-checker

Monitor a website or web page for content changes. Automatically saves before and after screenshots and sends an email notification when content changes are detected.

Jakub Balada

AI Web Agent

apify/ai-web-agent

Use natural language prompts to browse the web, click on elements, fill and submit forms, extract data, and take screenshots using the OpenAI API.

Apify

385

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

3.2k

Merge, Dedup & Transform Datasets

lukaskrivka/dedup-datasets

The ultimate dataset processor. Extremely fast merging, deduplications & transformations all in a single run.

Lukáš Křivka

1.6k

Actor fail manager

lukaskrivka/actor-fail-manager

Automatically triggered on a failed run to analyze if the run should be resurrected and to create an error report for the author.

Lukáš Křivka

2.5k

Send Email

apify/send-mail

The actor automatically sends an email to a specific address. This actor is useful for notifications and reporting. With only 3 lines of javascript code, you'll be on top of your scraping actors and never miss important results or issues.

Apify

2.4k

BeautifulSoup Scraper

apify/beautifulsoup-scraper

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Apify

592