# Bulk File & Image Downloader: Links, Pages, ZIP (`subtleasianmerch/bulk-file-downloader`) Actor

Download files from a list of links, or the images and linked PDFs, CSVs or ZIPs on web pages, into Apify storage. One row per file with a download link, type, size, SHA-256 and image size. Optional ZIP. Failed files are free.

- **URL**: https://apify.com/subtleasianmerch/bulk-file-downloader.md
- **Developed by:** [Public Data Tools](https://apify.com/subtleasianmerch) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 file downloadeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk File & Image Downloader: Links, Pages, ZIP

Download **files in bulk** from a list of links, or **every image on a web page** and the PDFs, CSVs, spreadsheets or ZIPs it links to. Each file is saved in your Apify storage with a download link that works, and you get one row per file with its type, size, SHA-256 and, for images, width and height. Turn on one ZIP of everything if you want a single download.

- 📥 **Two ways in, one run:** paste direct file links (images, PDFs, CSVs, Excel, ZIPs, audio, video, anything), or pages to collect files from.
- 🖼️ **Images from pages:** `<img>` with srcset and lazy-load attributes, `<picture>` sources and the og:image; the largest version by default.
- 📎 **Linked files by extension:** `pdf`, `csv`, `xlsx`, `zip`, `docx`, `mp3`… from links, embeds and media sources on the page.
- 🧾 **A row per file:** download link, file name, content type, size in bytes, SHA-256, image width and height, the page it was on, its alt text.
- 🧹 **Filters and limits:** file types and extensions, minimum and maximum size, minimum image size, duplicates by link and by content, a file count and a total size the run never passes.
- 🤖 **Polite:** robots.txt for pages and the files found on them, 2 connections and 25 Mbit/s per site by default, Retry-After honoured, an honest user agent.
- 💸 **$1 per 1,000 files** up to 2 MB each; larger files add $0.0002 per MB. Skipped, duplicate and failed files are free.

**Try it:** the input comes pre-filled with the Wikipedia article on *The Starry Night*, at most 10 files and 5 MB, and images of at least 200 × 200 pixels. Click **Start**: in a few seconds you get ten images of the painting and related works, with their sizes and hashes, for $0.01.

### What you get

**One row per file**, in the **Files** table (with a preview of each image) and as JSON, CSV or Excel:

| Data | What's in it |
|---|---|
| **Status** | `downloaded` (stored and charged) or, free: `duplicate`, `skipped`, `not downloaded`, `disallowed`, `blocked`, `refused`, `failed`, with the reason in plain words |
| **The file** | `key` in the run's key-value store, `downloadUrl` (opens the file, also from outside Apify), `fileName` from the server or the link, `contentType`, `fileType` (image, pdf, document, spreadsheet, archive, audio, video, other), `extension`, `sizeBytes`, `sha256` |
| **Images** | `width` and `height` in pixels, read from the file's header (JPEG, PNG, GIF, WebP, AVIF, HEIC, BMP, TIFF, ICO, SVG) |
| **Where it came from** | `sourceUrl`, `finalUrl` after redirects, `httpStatus`, `pageUrl` and how it was found (`img`, `srcset`, `picture`, `og:image`, `link`, `embed`, `media`, or `input` for your own links), the image's `altText` or the link's text |
| **Duplicates** | `duplicateOf`: the key of the stored file with the same SHA-256 |
| **Billing** | `extraMegabytes`: started MB above the first 2 MB |
| **When** | `fetchedAt` |

**The files themselves** are in the run's key-value store (the **Storage** tab, or the "Downloaded files" link in the output), named by your pattern: `00001-Mona_Lisa.jpg` by default. With the ZIP on, `files.zip` holds them all.

**The `RUN_SUMMARY` record** has counts per status, every page with what was found on it, each host's requests and limits, the robots.txt answers, and the files found on pages that weren't tried because a limit was reached.

### Who it's for

- **E-commerce and marketplace teams:** pull product photos from your own or your suppliers' pages, or from a scraper's image links, into one place with sizes and hashes.
- **Researchers and data teams:** collect image or document sets from listed pages, de-duplicated by content, with the page, alt text and fetch time for each.
- **Site migrations and backups:** save every image and PDF your old pages show before you move them.
- **Developers and automations:** turn a column of file links from another Actor, a sheet or an API into stored files with stable links, then pass them on.
- **Archivists and compliance teams:** keep copies of published PDFs and spreadsheets with a SHA-256 for each.

### How much does it cost?

You pay per file stored, plus Apify's standard run start fee (a fraction of a cent):

| Event | Price |
|---|---|
| File stored, first 2 MB included | $0.001 |
| Each started MB of a file above its first 2 MB | $0.0002 |
| ZIP of all files (only when on), per started MB of the ZIP | $0.0001 |

| Example | Cost |
|---|---|
| The pre-filled example: 10 images | $0.01 |
| 1,000 product photos under 2 MB | $1 |
| One 20 MB file | $0.0046 ($0.001 + 18 × $0.0002) |
| 200 PDFs of 5 MB | $0.32 |
| The same 200 PDFs with a ZIP (1 GB) | $0.42 |

**Free:** skipped files (filters, size limits), duplicates, files a robots.txt disallows, blocked and failed downloads, files not tried because a limit was reached, pages, and runs that fail.

Why a price per MB? Apify bills this Actor for every megabyte it writes to your storage and for the time each download takes, and the ZIP is a second copy, so big files cost more to deliver. Files up to 2 MB, most images and PDFs, pay only the file price. Set a maximum cost per run in the run options; the run stops cleanly at that amount.

### How to use it

1. **Add file links,** one per line, under **File links**, or pages under **Pages to collect files from**, or both.
2. **For pages, choose what to collect:** images (on by default; the largest version, or as shown), the og:image, and linked files by extension (for example `pdf`, `csv`, `xlsx`).
3. **Filter (optional):** file types, extensions, minimum and maximum file size, minimum image width and height.
4. **Set the limits:** the most files and the most megabytes per run, and whether to skip duplicates by link and by content.
5. **Name the files (optional):** a pattern such as `{index}-{name}`, `{host}-{name}` or `{hash}`; turn on the ZIP if you want one archive.
6. **Click Start.** The **Files** table shows each file with a preview, size and download link. Export it as JSON, CSV or Excel.

Only the pages you list are read; links from them to other pages aren't followed. Images a page adds with JavaScript, or sets in CSS, aren't in its HTML, so they aren't found.

### Example input

**Product photos and a price list from your own links:**

```json
{
  "fileUrls": [
    "https://example.com/images/sku-1001.jpg",
    "https://example.com/images/sku-1002.png",
    "https://example.com/downloads/price-list.xlsx"
  ],
  "fileNamePattern": "{name}"
}
```

**Every image and PDF on two pages, at least 300 px wide, in one ZIP:**

```json
{
  "pageUrls": ["https://example.com/products/chairs", "https://example.com/catalogue"],
  "linkExtensions": ["pdf"],
  "minImageWidth": 300,
  "zip": true
}
```

**Only the PDFs and spreadsheets a page links to, no images:**

```json
{
  "pageUrls": ["https://example.org/reports"],
  "collectImages": false,
  "linkExtensions": ["pdf", "csv", "xlsx"],
  "maxFiles": 200,
  "maxTotalMb": 500
}
```

### Example output

A row from the pre-filled example (a thumbnail of a public-domain painting on Wikimedia Commons), shortened:

```json
{
  "rowType": "file",
  "status": "downloaded",
  "index": 9,
  "sourceUrl": "https://thumb.wikimedia.org/wikipedia/commons/thumb/e/e7/Vincent_van_Gogh_-_Green_Field_-_Google_Art_Project.jpg/330px-Vincent_van_Gogh_-_Green_Field_-_Google_Art_Project.jpg?utm_source=en.wikipedia.org&utm_campaign=parser&utm_content=thumbnail",
  "pageUrl": "https://en.wikipedia.org/wiki/The_Starry_Night",
  "foundIn": "srcset",
  "fileName": "Vincent_van_Gogh_-_Green_Field_-_Google_Art_Project.jpg",
  "key": "00009-Vincent_van_Gogh_-_Green_Field_-_Google_Art_Project.jpg",
  "downloadUrl": "https://api.apify.com/v2/key-value-stores/…/records/00009-Vincent_van_Gogh_-_Green_Field_-_Google_Art_Project.jpg?signature=…",
  "contentType": "image/jpeg",
  "fileType": "image",
  "extension": "jpg",
  "sizeBytes": 43977,
  "sha256": "…",
  "width": 330,
  "height": 271,
  "duplicateOf": null,
  "extraMegabytes": 0,
  "fetchedAt": "2026-11-04T…Z",
  "error": null
}
```

And a free one:

```json
{
  "status": "disallowed",
  "sourceUrl": "https://en.wikipedia.org/w/resources/assets/poweredby_mediawiki.svg",
  "pageUrl": "https://en.wikipedia.org/wiki/The_Starry_Night",
  "error": "Not downloaded: robots.txt of en.wikipedia.org disallows it. Not charged."
}
```

#### Good to know

- **Download links:** `downloadUrl` is the stored record's address. When your Apify storage is private (the default for new accounts), the link carries a signature so it opens without an Apify token; anyone with the link can open that file. The files stay for your plan's data retention period, like any run's storage.
- **Names:** keys may only use letters, digits and `! - _ . ' ( )`, so other characters become `-` (accents are dropped: `Café.jpg` → `Cafe.jpg`). A repeated key gets `-2`, `-3`. The original name is in `fileName`.
- **File types** come from the file's first bytes where they say (images, PDF, ZIP and other archives, audio, video), else from the server's file name, the link or the Content-Type. A Word, Excel or PowerPoint file keeps its own extension.
- **Limits:** a file over the per-file size isn't downloaded when the server says its size, and is stopped as soon as it passes the limit when it doesn't. The total size is checked before each file is stored, so the run never stores more than you set. Each file passes through the run's disk on its way to storage (about 8 GB free at the default 1 GB of memory in tests); large files are never held whole in memory.
- **Duplicates:** with **Skip duplicate files** on, a file with the same content as one already stored is listed with `duplicateOf` and not stored or charged again.
- **Web pages given as file links** are skipped with a hint to put them under **Pages**, unless you add `html` to **Only these extensions**.
- **Bigger runs:** up to 10,000 files and 20 GB per run, and 1,000 pages; 10 downloads at a time by default (up to 20).

### Integrations

- **Apify API:** start a run and get the rows back as JSON in one call:

  ```bash
  curl -X POST "https://api.apify.com/v2/acts/subtleasianmerch~bulk-file-downloader/run-sync-get-dataset-items?token=YOUR_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{"fileUrls": ["https://example.com/a.pdf", "https://example.com/b.jpg"]}'
  ```

  Replace `YOUR_TOKEN` with your Apify API token. Each row's `downloadUrl` gives you the file.
- **After another Actor:** feed it the image or file links another scraper found, with a webhook or an Apify integration that starts this Actor when the first one finishes.
- **Make, Zapier and n8n:** send the stored files' links to Google Drive, Dropbox, S3, Slack or a database with Apify's apps for these tools.
- **Schedules:** run it daily or weekly in Apify Console to keep copies of files that change.

### FAQ

#### Is it legal to use?

The files belong to their owners. You are responsible for having the right to download the files you choose and to use them the way you plan to: your own files, files under an open licence (and following its terms, such as credit), or files you have permission for. Each row keeps the source address, the page and the fetch time, so you can credit and check sources; this Actor doesn't decide or record licences. It respects robots.txt for the pages you give and for every file it finds on them, sends an honest user agent, keeps to a few connections per site, and doesn't get past blocks: a site that refuses, asks it to wait for long, or shows a bot check gets a free row and is left alone. Direct links you give are fetched as given. This isn't legal advice.

#### Does it follow robots.txt?

Yes, for crawling: before reading a page, and before downloading each file found on a page, it reads that host's robots.txt (rules for `bulk-file-downloader`, else for `*`) and skips what it disallows, listed as `disallowed` and free. A robots.txt that can't be read (a server error) counts as "disallow everything". A Crawl-delay of up to 60 seconds is kept, with one connection to that host. Links you list yourself under **File links** aren't crawled, so robots.txt isn't consulted for them, as with any download you start yourself.

#### Why was an image skipped or missing?

The row says why: too small (bytes or pixels), the wrong type or extension, a duplicate, disallowed by robots.txt, or a server that refused. If an image you see in your browser isn't listed at all, the page adds it with JavaScript or shows it as a CSS background; this Actor reads the HTML the server sends, without a browser. Give the image's own link under **File links** instead.

#### Can it download from sites that need a login?

No. It fetches public links only, without cookies or passwords. Links that already carry their own access, such as signed S3 or Google Cloud links, work; credentials in links are hidden in the rows and logs.

#### Can it reach my private network?

No. Addresses on private, local and cloud metadata networks are refused, including when a link redirects to one or when a name resolves to one.

#### How do I get all files at once?

Turn on **Also save one ZIP of all files**: `files.zip` appears in the key-value store and in the output's "ZIP of all files" link. Or download the records from the **Storage** tab, or use each row's `downloadUrl`.

#### What happens with HTTP 429 (too many requests)?

The Actor pauses that host for as long as the server asks (Retry-After, up to 120 seconds) and tries again, up to three times. A server that asks for a longer pause, or keeps answering 429, gets its remaining files listed as `blocked`, free; the Actor doesn't switch addresses or proxies to get around it.

### Related Actors

More Actors from the same developer that work well with this one:

| Actor | Use it to |
|---|---|
| [Website Screenshot API](https://apify.com/subtleasianmerch/website-screenshot-api) | capture the pages themselves as images or PDFs |
| [Bluesky Scraper](https://apify.com/subtleasianmerch/bluesky-scraper) | collect posts with their image links, then download the images here |

Questions, bugs or ideas? Open an issue on this Actor's **Issues** tab.

# Actor input Schema

## `fileUrls` (type: `array`):

Direct links to files: images, PDFs, CSVs, spreadsheets, ZIPs, audio, video, anything. Each is downloaded as it is and stored in the run's key-value store. Up to 10,000 links. robots.txt isn't consulted for links you give yourself (they aren't crawled).

## `pageUrls` (type: `array`):

Web pages whose images (and, if you choose below, linked files) are downloaded. Only the pages you list are read; links to other pages aren't followed. Each page's robots.txt is respected, and so is the robots.txt of every host the files come from. Up to 1,000 pages.

## `collectImages` (type: `boolean`):

Download the pages' images: <img> (with srcset and the usual lazy-load attributes such as data-src), <picture> sources and the og:image. Images that a page adds with JavaScript, or sets in CSS, aren't seen.

## `imageSize` (type: `string`):

Which version of a responsive image to download. Largest: the biggest candidate in srcset and <picture> (by width, or by pixel density). As shown: the plain src the page shows by default.

## `includeOgImage` (type: `boolean`):

Also download the page's sharing image (og:image), often a large version of the main picture.

## `linkExtensions` (type: `array`):

Also download files the pages link to, by extension: for example pdf, csv, xlsx, zip, docx or mp3. Links (<a href>), embedded files (<iframe>, <embed>, <object>) and <video>/<audio> sources are checked. Add jpg or png to get full-size images that thumbnails link to. Empty: no linked files.

## `sameSiteOnly` (type: `boolean`):

Skip files hosted on other sites than the page (for example on a CDN with another domain). The site is the registered domain, so www.example.com and cdn.example.com count as one site.

## `fileTypes` (type: `array`):

Keep only these kinds of file. The kind is read from the file's first bytes where they say (images, PDF, archives, audio, video), else from its name or Content-Type. Others are listed as skipped, free. Empty: every kind.

## `extensions` (type: `array`):

Keep only files with these extensions, e.g. jpg, png, pdf (jpg and jpeg count as one). The extension comes from the file's bytes where they say, else from its name. Add html to save web pages as files too. Empty: every extension.

## `minFileSizeKb` (type: `integer`):

Skip files smaller than this, e.g. tracking pixels and icons. 0: no minimum.

## `maxFileSizeMb` (type: `integer`):

Skip files larger than this. When the server gives the size, a larger file isn't downloaded at all. Files are written to the run's disk first, so very large files need more memory (more memory gives more disk).

## `minImageWidth` (type: `integer`):

Skip images narrower than this (read from the image's header). 0: no minimum.

## `minImageHeight` (type: `integer`):

Skip images lower than this. 0: no minimum.

## `maxFiles` (type: `integer`):

Stop after storing this many files. Skipped, duplicate and failed files don't count.

## `maxTotalMb` (type: `integer`):

The run never stores more than this many megabytes of files in all. A file that would pass it isn't stored. With the ZIP on, the ZIP is a second copy of the same files.

## `dedupeByUrl` (type: `boolean`):

Download each address once per run, even if several pages show it (utm\_ tracking parameters are ignored when comparing).

## `dedupeByHash` (type: `boolean`):

Store each distinct file once: a file with the same content as one already stored is listed as a duplicate, with the stored copy's key, and not charged.

## `fileNamePattern` (type: `string`):

The key of each file in the key-value store. Placeholders: {index} (the file's number, 00001…), {name} (its name from the server or the link, without extension), {ext}, {host} (where it was downloaded from), {page} (the page's host and path, or 'links'), {hash} (the first 12 characters of its SHA-256) and {date} (the run's date). The extension is added when the pattern has no {ext}. Keys may use only letters, digits and ! - \_ . ' ( ); other characters become '-', and a repeated key gets -2, -3.

## `zip` (type: `boolean`):

At the end, also save files.zip with every stored file (named by its key, not compressed again). Charged per started MB of the ZIP. At most 4 GB and 65,534 files.

## `maxConcurrency` (type: `integer`):

How many files are downloaded side by side, across all sites.

## `maxConnectionsPerHost` (type: `integer`):

At most this many requests at a time to one host. 2 is the most Wikimedia's robot policy allows for its media servers. A robots.txt Crawl-delay (up to 60 s) sets one connection with that pause, for pages and the files found on them.

## `maxSpeedMbpsPerHost` (type: `integer`):

The most this Actor downloads per second from one host, in megabits (25 Mbit/s is about 3 MB a second, Wikimedia's limit for its media servers). Raise it for your own servers.

## `downloadTimeoutSecs` (type: `integer`):

A file that takes longer is given up (free). A server that sends nothing for 30 s is given up too. HTTP 429 and 503 answers are retried after the pause the server asks for (Retry-After, up to 120 s).

## Actor input object example

```json
{
  "pageUrls": [
    "https://en.wikipedia.org/wiki/The_Starry_Night"
  ],
  "collectImages": true,
  "imageSize": "largest",
  "includeOgImage": true,
  "sameSiteOnly": false,
  "minFileSizeKb": 0,
  "maxFileSizeMb": 100,
  "minImageWidth": 200,
  "minImageHeight": 200,
  "maxFiles": 10,
  "maxTotalMb": 5,
  "dedupeByUrl": true,
  "dedupeByHash": true,
  "fileNamePattern": "{index}-{name}",
  "zip": false,
  "maxConcurrency": 10,
  "maxConnectionsPerHost": 2,
  "maxSpeedMbpsPerHost": 25,
  "downloadTimeoutSecs": 300
}
```

# Actor output Schema

## `results` (type: `string`):

One row per file: status, download link, type, size, SHA-256 and image dimensions.

## `files` (type: `string`):

The files saved in the run's key-value store; each dataset row links to its own.

## `zip` (type: `string`):

files.zip, when "Also save one ZIP" is on.

## `runSummary` (type: `string`):

Counts per status, pages, hosts, robots.txt answers and anything skipped, saved as RUN_SUMMARY.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "pageUrls": [
        "https://en.wikipedia.org/wiki/The_Starry_Night"
    ],
    "minImageWidth": 200,
    "minImageHeight": 200,
    "maxFiles": 10,
    "maxTotalMb": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("subtleasianmerch/bulk-file-downloader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "pageUrls": ["https://en.wikipedia.org/wiki/The_Starry_Night"],
    "minImageWidth": 200,
    "minImageHeight": 200,
    "maxFiles": 10,
    "maxTotalMb": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("subtleasianmerch/bulk-file-downloader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "pageUrls": [
    "https://en.wikipedia.org/wiki/The_Starry_Night"
  ],
  "minImageWidth": 200,
  "minImageHeight": 200,
  "maxFiles": 10,
  "maxTotalMb": 5
}' |
apify call subtleasianmerch/bulk-file-downloader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,subtleasianmerch/bulk-file-downloader"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/TptgQGLzzcdKY6NzL/builds/GClGlvAz0eSyDKxv7/openapi.json