# Bulk File Downloader — Links or a Dataset to Files and a ZIP (`steadyscrape/bulk-file-downloader`) Actor

Bulk download files from a list of URLs or from a field of another run's dataset (images, PDFs, videos, documents): each file stored with a public link, size, type and SHA-256, duplicates skipped, plus one ZIP of everything. A bulk image downloader you can also call via API.

- **URL**: https://apify.com/steadyscrape/bulk-file-downloader.md
- **Developed by:** [Steady Scrape](https://apify.com/steadyscrape) (community)
- **Categories:** Automation, Developer tools, Integrations
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 file downloadeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk File Downloader — links or a dataset to files and a ZIP

Give it **a list of file links**, or **pick the dataset of another run** (a scraper's results) and the field that holds
the links, and it downloads the files: images, PDFs, videos, audio, documents, archives. Every file is stored with a
**public download link**, its size, type and SHA-256, and the run ends with **one link that downloads everything as a
ZIP**.

- **Download the images of a scrape in one step.** Point it at the dataset of an Airbnb, Zillow, Pinterest, Amazon or
  Shopify run, name the field (`images.url`, `imageUrl`, `photos`), and every picture lands in one store and one ZIP —
  each row says which item and which field it came from.
- **Pay for files, not for failures.** A dead link, a web page, a refused site, a file over your limits or a duplicate
  is a free row that says why.
- **Duplicates skipped by content.** The same bytes under two links are stored and charged once.
- **Hard limits you set before anything is downloaded**: most files, most megabytes in total, most megabytes per file,
  and your spending limit — checked before each download starts and while its bytes arrive, so a run never goes past
  them.
- **Polite by default**: robots.txt is respected (you can turn that off), never more than two downloads at once from one
  site, busy servers retried with their `Retry-After`.

### Use cases

- Archive the product photos, listing pictures or attachments a scraper found.
- Collect PDFs (reports, filings, catalogues) from a list of links for an AI or search pipeline.
- Mirror a media list into Apify storage, with checksums to verify it later.
- Turn a CSV export of links into a ZIP you can hand to someone.

### Input

| field | what it does |
|---|---|
| `urls` | Direct file links, one per line. |
| `datasetId` | Pick another run's dataset; its items' links are downloaded. Only the dataset you pick can be read. |
| `datasetFields` | Dotted paths to the links in each item, walking through arrays (`images.url`). Empty: every link in a media-named field (image, photo, thumbnail, video, file, pdf…) or ending in a file extension. |
| `fileTypes` | Keep only images, videos, audio, documents, archives or other. Empty keeps every kind. |
| `maxFiles` | Most files stored (default 1,000). |
| `maxTotalMegabytes` | Most megabytes stored in total (default 5,000). |
| `maxFileMegabytes` | Most megabytes per file (default 500, up to 2,048; give the run more memory for files over 500 MB). |
| `respectRobotsTxt` | On by default. |
| `skipDuplicates` | On by default: identical bytes are stored once. |
| `maxConcurrency` | Downloads at once (default 4, never more than 2 per site). |

```json
{
    "datasetId": "<the dataset of your scraper run>",
    "datasetFields": ["images.url"],
    "fileTypes": ["images"],
    "maxFiles": 500
}
```

### Output

One row per link:

```json
{
    "url": "https://a0.muscache.com/im/pictures/1a2b3c.jpg",
    "finalUrl": "https://a0.muscache.com/im/pictures/1a2b3c.jpg",
    "status": "ok",
    "reason": null,
    "fileName": "1a2b3c.jpg",
    "key": "files-000000-1a2b3c.jpg",
    "fileUrl": "https://api.apify.com/v2/key-value-stores/…/records/files-000000-1a2b3c.jpg?signature=…",
    "contentType": "image/jpeg",
    "fileType": "images",
    "bytes": 184322,
    "sha256": "9f2c…",
    "duplicateOf": null,
    "billedTransferUnits": 0,
    "sourceItemIndex": 0,
    "sourceField": "images.url",
    "downloadedAt": "2026-10-13T12:00:00.000Z"
}
```

| status | meaning | charged |
|---|---|---|
| `ok` | stored; `fileUrl` opens it for anyone | yes |
| `duplicate` | same bytes as a file already stored; `duplicateOf` and `fileUrl` point to it | no |
| `refused` | a private or local address, or a video/music/social platform (see below) | no |
| `robots_disallowed` | the site's robots.txt does not allow it, or could not be read | no |
| `skipped` | a web page, or a kind of file you did not ask for | no |
| `too_large` | bigger than `maxFileMegabytes` | no |
| `over_limit` | it would have passed `maxTotalMegabytes` or your spending limit; the run stopped there | no |
| `unreachable` | the server answered an error, nothing, or not at all (after three tries for busy servers) | no |
| `invalid` | not an http(s) link | no |
| `failed` | downloaded but could not be stored | no |

The run's `SUMMARY` record has the files and megabytes stored, the rows per status, how many links were left when a
limit stopped the run, and `zipUrl`: **one signed link that downloads every stored file as a ZIP**, without an Apify
login.

### Pricing

Pay per event, no start fee:

- **`file-downloaded`: US$0.001** per file stored (US$1 per 1,000), which includes its first 10 MB.
- **`transfer-10mb`: US$0.001** per started 10 MB beyond the first 10 MB of a file.

A 200 KB photo costs US$0.001; a 25 MB video US$0.003. Everything that is not stored is free.

### What it refuses, and your responsibility

- **Private and local networks**, always: loopback, private ranges, link-local and the cloud metadata service, and every
  other reserved address, checked on every redirect and on the address each connection really lands on.
- **Video, music and social platforms**: YouTube, TikTok, Instagram, Facebook, Netflix, Spotify, Twitch, Vimeo,
  Dailymotion, SoundCloud, Disney+, Hulu, Prime Video, Max, Apple TV and Apple Music, Deezer. Their media is licensed
  content behind terms that forbid downloading it.
- **robots.txt**: on by default. Turning it off, and downloading anything at all, means **you are responsible for having
  the right to download and store those files** — copyright, licences and the sites' terms are yours to respect.
  Most image CDNs allow it (Airbnb's, Amazon's, Shopify's, Zillow's, eBay's and Wikimedia's did in October 2026), but
  some allow only the robots on their own list: Pinterest's `i.pinimg.com` disallows everyone else, so its images come
  back `robots_disallowed` unless you turn robots.txt off.

### Limits

- Links are followed through up to five redirects. A page is not crawled for its files: give the file links, or a
  dataset that has them.
- A file waits on the run's disk until it is stored; a run has twice its memory in disk. The default 1 GB run handles
  files up to 500 MB comfortably; give the run more memory for bigger ones.
- Up to 100,000 links per run.

# Actor input Schema

## `urls` (type: `array`):

Direct links to files: images, PDFs, videos, audio, documents, archives. One per line. A link that opens a web page is not a file and comes back as a free row that says so.

## `datasetId` (type: `string`):

Pick the dataset of another run (a scraper's results, an import) and this Actor downloads the files its items link to: product photos, listing pictures, attachments, PDFs. Only the dataset you pick here can be read.

## `datasetFields` (type: `array`):

The fields of each dataset item that hold file links, as dotted paths; arrays are walked through, so images.url takes the url of every element of images. Leave empty to take every link in a field named like media (image, photo, thumbnail, video, file, pdf…) or ending in a file extension.

## `fileTypes` (type: `array`):

Keep only these kinds; the rest come back as free skipped rows, read no further than their headers. Empty keeps every kind of file. Web pages are never stored.

## `maxFiles` (type: `integer`):

The run stops starting downloads once this many files are stored.

## `maxTotalMegabytes` (type: `integer`):

The run never stores more than this. A file that would pass it is not stored and not charged, and the run stops there.

## `maxFileMegabytes` (type: `integer`):

A bigger file is a free too_large row. A file waits on the run's disk until it is stored, and a run has twice its memory in disk: for files over 500 MB, give the run more memory.

## `respectRobotsTxt` (type: `boolean`):

On: a link a site's robots.txt disallows for this Actor (or for every robot) is a free row, and so is every link of a site whose robots.txt cannot be read. Off: you take responsibility for having the right to download those files.

## `skipDuplicates` (type: `boolean`):

A file with the same bytes (SHA-256) as one already stored in this run is not stored again and not charged; its row points to the first.

## `maxConcurrency` (type: `integer`):

Files downloaded at the same time, never more than two from one host.

## Actor input object example

```json
{
  "urls": [
    "https://media.w3.org/2010/05/sintel/poster.png",
    "https://www.rfc-editor.org/rfc/rfc9309.pdf",
    "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png"
  ],
  "datasetFields": [],
  "fileTypes": [],
  "maxFiles": 1000,
  "maxTotalMegabytes": 5000,
  "maxFileMegabytes": 500,
  "respectRobotsTxt": true,
  "skipDuplicates": true,
  "maxConcurrency": 4
}
```

# Actor output Schema

## `files` (type: `string`):

No description

## `zip` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://media.w3.org/2010/05/sintel/poster.png",
        "https://www.rfc-editor.org/rfc/rfc9309.pdf",
        "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png"
    ],
    "datasetFields": []
};

// Run the Actor and wait for it to finish
const run = await client.actor("steadyscrape/bulk-file-downloader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": [
        "https://media.w3.org/2010/05/sintel/poster.png",
        "https://www.rfc-editor.org/rfc/rfc9309.pdf",
        "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png",
    ],
    "datasetFields": [],
}

# Run the Actor and wait for it to finish
run = client.actor("steadyscrape/bulk-file-downloader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://media.w3.org/2010/05/sintel/poster.png",
    "https://www.rfc-editor.org/rfc/rfc9309.pdf",
    "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png"
  ],
  "datasetFields": []
}' |
apify call steadyscrape/bulk-file-downloader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,steadyscrape/bulk-file-downloader"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/LcrFK5zIUokPHGkY7/builds/r4iOiEleBbB8oaiK1/openapi.json