# Bulk File Downloader — URL List to Storage or ZIP (`actor-forge/bulk-file-downloader`) Actor

Download any files in bulk from URL lists or the file links on a web page: PDF, ZIP, CSV, XLSX, DOCX, MP3, MP4 and more. Public download links, real type from the bytes, size, SHA-256, type/glob filters, dedupe, size caps, optional ZIP. $2 per 1,000 files + $0.001 per 10 MB.

- **URL**: https://apify.com/actor-forge/bulk-file-downloader.md
- **Developed by:** [ActorForge](https://apify.com/actor-forge) (community)
- **Categories:** Developer tools, Automation, Integrations
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 file downloadeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk File Downloader — URL List to Storage or ZIP

Download any files in bulk from URL lists or the file links on a web page: PDF, ZIP, CSV, XLSX, DOCX, MP3, MP4 and more. Public download links, real type from the bytes, size, SHA-256, type/glob filters, dedupe, size caps, optional ZIP. $2 per 1,000 files + $0.001 per 10 MB.

**Download any files in bulk, from a list of URLs or from the links on a web page.** Paste direct links to PDFs, ZIP archives, CSV/XLSX exports, Word documents, MP3s, videos or any other file, or give it pages (a reports archive, a dataset index, a podcast page) and it follows every file link it finds: `<a href>` links, audio/video sources, embeds and objects. Each file is stored in the run's key-value store with a **public download link**, and its **real type is detected from the file bytes** (magic numbers: PDF, ZIP, DOCX/XLSX/PPTX, ODF, EPUB, legacy Office, GZIP/7z/RAR/TAR, MP3/WAV/FLAC/OGG, MP4/MOV/WebM, images, SQLite, Parquet, JSON/CSV/XML…), with the server's content type as a fallback. Every row carries the **MIME type, size, SHA-256, source URL, page and link text**. **Filter by file type or URL glob** (`*.pdf`, `/reports/**`), skip duplicates by content, cap the size **per file (default 100 MB) and in total**, and optionally get everything as **ZIP files**. Use it for archiving public documents, collecting datasets for analysis or machine learning, backing up website downloads, mirroring podcast episodes, or as a file-fetching tool for AI agents and automations. Pricing follows the real cost of moving data: **$0.002 per file plus $0.001 per started 10 MB**, so 1,000 small files cost about $3. Broken links, web pages, filtered types, oversized files and duplicates are never charged.

### What data does it return?

Every run outputs clean, schema-validated rows like these:

| fileName | extension | mimeType | bytes | detectedBy | url |
| --- | --- | --- | --- | --- | --- |
| 00001-minimal-document.pdf | pdf | application/pdf | 16978 | magic | https://raw.githubusercontent.com/py-pdf/sample-files/main/001-trivial/minimal-document.pdf |
| 00002-horse.mp3 | mp3 | audio/mpeg | 28915 | magic | https://www.w3schools.com/html/horse.mp3 |
| 00003-addresses.csv | csv | text/csv | 328 | magic | https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv |

### How much does it cost?

Transparent pay-per-event pricing — you only pay for what you get, and the
actor **always respects your run budget**: set a maximum total charge and it
stops cleanly at the cap instead of overrunning.

| Event | Price | Charged | Worked example |
| --- | --- | --- | --- |
| `file-downloaded` | $0.002 | per file saved (type detection, SHA-256, download link and ZIP included) | 1,000 small files = $2 + $1 transfer = $3; 100 PDFs of 2 MB = $0.30 |
| `transfer-10mb` | $0.001 | per started 10 MB of each file saved | a 3 MB file = 1 event ($0.001); a 95 MB video = 10 events ($0.01); 1 GB of large files ≈ $0.10 |
| `image-result` | $0.003 | per image saved, only in image mode (page image scanning) | a page with 40 images = $0.12 |
| `apify-actor-start` | $0.00005 | per run per GB of memory (platform start fee) | One run at 1 GB = $0.00005 |

### Input options

| Field | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `mode` | string | no | `"files"` | Any files (default): every file type from a URL list or page links. Images only: scans pages for every image (img, srcset, lazy, CSS backgrounds) and is billed per image — see Pricing. |
| `urls` | array | yes | — | Direct file URLs (any type) and/or web pages whose file links should be downloaded — mix freely. A page's static HTML is scanned for links, media sources and embeds. Links added by JavaScript aren't found. |
| `fileTypes` | array | no | `[]` | Only keep files of these types, e.g. pdf, zip, csv, docx, mp3. Matched against the type detected from the file bytes and against the URL extension. On pages, links with these extensions are followed. Empty = every type (page links: common document, archive, data and media extensions). |
| `urlGlobs` | array | no | `[]` | On pages, also follow links matching these globs, e.g. *.pdf, /reports/\*\*, https://cdn.example.com/files/*. Use for download links without an extension (/download?id=…). \* stays inside one folder, \*\* crosses folders. |
| `maxFileSizeMb` | number | no | `100` | Skip files larger than this (up to 500 MB). Oversized downloads are aborted early and never charged. |
| `maxTotalSizeMb` | number | no | `2048` | Stop downloading once the files saved add up to this many MB (0 = no cap). Keeps transfer charges predictable. |
| `minFileSizeKb` | number | no | `0` | Skip files smaller than this. Skipped files are not charged. |
| `maxFiles` | integer | no | `1000` | Stop after saving this many files in total. |
| `skipDuplicates` | boolean | no | `true` | Skip files whose content (SHA-256) is identical to one already downloaded, even from different URLs. Duplicates are not charged. |
| `createZip` | boolean | no | `false` | Bundle everything into ZIP file(s) of up to 200 MB each. Links are in the run's OUTPUT record. No extra charge. |
| `respectRobotsTxt` | boolean | no | `true` | Skip pages that the site's robots.txt disallows for crawlers (direct file URLs you list are always fetched). |

### Limits & honest notes

- Pages are scanned as static HTML without a browser, which is fast and cheap. Links added only by JavaScript are not found; pass those file URLs directly instead.
- Pages disallowed by the site's robots.txt are skipped by default (you can turn this off). If a site's robots.txt can't be fetched (HTTP 5xx or a network error, after one retry), all of that site's pages are skipped (RFC 9309), each with a free error row. Direct file URLs you list are always fetched. Login-protected files are not supported.
- Files over the per-file cap (default 100 MB, up to 500 MB) are aborted early and not charged. The total size cap (default 2 GB) stops the run cleanly; give the run 2-4 GB of memory for many large files. ZIP files are split into parts of up to 200 MB.
- Without a type or glob filter, page links are followed only when they end in a common document, archive, data or media extension (images are left out; add jpg or png to File types to include them).
- Download links work for the lifetime of the run's storage (Apify's default retention for your plan).
- Only download files you have the right to use.
- Internal and private network addresses are always refused, including through redirects or DNS tricks.
- Respects your run budget: set a maximum total charge and the actor stops cleanly at the cap.

### Related ActorForge actors

- [Bulk Image Downloader](https://apify.com/actor-forge/bulk-image-downloader) — every image from a page, srcset and lazy-loaded included
- [Word, Excel & PowerPoint to Markdown](https://apify.com/actor-forge/office-to-markdown) — turn downloaded DOCX/XLSX/PPTX into Markdown or JSON
- [Image Converter & Compressor](https://apify.com/actor-forge/image-converter)

### Changelog

- **2026-10-19** — Initial release: any file type from URL lists or page links, magic-byte type detection, type/glob filters, SHA-256 dedupe, per-file and total size caps, ZIP export.

***

*Built and maintained by **ActorForge**. Reliability first: every actor in the
portfolio is monitored by an automated nightly self-test, and breakages are
typically fixed the same day.*

# Actor input Schema

## `mode` (type: `string`):

Any files (default): every file type from a URL list or page links. Images only: scans pages for every image (img, srcset, lazy, CSS backgrounds) and is billed per image — see Pricing.

## `urls` (type: `array`):

Direct file URLs (any type) and/or web pages whose file links should be downloaded — mix freely. A page's static HTML is scanned for links, media sources and embeds. Links added by JavaScript aren't found.

## `fileTypes` (type: `array`):

Only keep files of these types, e.g. pdf, zip, csv, docx, mp3. Matched against the type detected from the file bytes and against the URL extension. On pages, links with these extensions are followed. Empty = every type (page links: common document, archive, data and media extensions).

## `urlGlobs` (type: `array`):

On pages, also follow links matching these globs, e.g. *.pdf, /reports/\*\*, https://cdn.example.com/files/*. Use for download links without an extension (/download?id=…). \* stays inside one folder, \*\* crosses folders.

## `maxFileSizeMb` (type: `number`):

Skip files larger than this (up to 500 MB). Oversized downloads are aborted early and never charged.

## `maxTotalSizeMb` (type: `number`):

Stop downloading once the files saved add up to this many MB (0 = no cap). Keeps transfer charges predictable.

## `minFileSizeKb` (type: `number`):

Skip files smaller than this. Skipped files are not charged.

## `maxFiles` (type: `integer`):

Stop after saving this many files in total.

## `skipDuplicates` (type: `boolean`):

Skip files whose content (SHA-256) is identical to one already downloaded, even from different URLs. Duplicates are not charged.

## `createZip` (type: `boolean`):

Bundle everything into ZIP file(s) of up to 200 MB each. Links are in the run's OUTPUT record. No extra charge.

## `respectRobotsTxt` (type: `boolean`):

Skip pages that the site's robots.txt disallows for crawlers (direct file URLs you list are always fetched).

## Actor input object example

```json
{
  "mode": "files",
  "urls": [
    "https://raw.githubusercontent.com/py-pdf/sample-files/main/001-trivial/minimal-document.pdf",
    "https://www.w3schools.com/html/horse.mp3",
    "https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv"
  ],
  "fileTypes": [],
  "urlGlobs": [],
  "maxFileSizeMb": 100,
  "maxTotalSizeMb": 2048,
  "minFileSizeKb": 0,
  "maxFiles": 1000,
  "skipDuplicates": true,
  "createZip": false,
  "respectRobotsTxt": true
}
```

# Actor output Schema

## `results` (type: `string`):

One row per downloaded file with a direct download link, detected type, size, SHA-256 and source.

## `summary` (type: `string`):

Counts of saved and skipped files, total bytes, whether the size cap was reached and, when createZip is on, download links for the ZIP file(s).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://raw.githubusercontent.com/py-pdf/sample-files/main/001-trivial/minimal-document.pdf",
        "https://www.w3schools.com/html/horse.mp3",
        "https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("actor-forge/bulk-file-downloader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://raw.githubusercontent.com/py-pdf/sample-files/main/001-trivial/minimal-document.pdf",
        "https://www.w3schools.com/html/horse.mp3",
        "https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("actor-forge/bulk-file-downloader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://raw.githubusercontent.com/py-pdf/sample-files/main/001-trivial/minimal-document.pdf",
    "https://www.w3schools.com/html/horse.mp3",
    "https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv"
  ]
}' |
apify call actor-forge/bulk-file-downloader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,actor-forge/bulk-file-downloader"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/goQ2foiMFblsbPxvX/builds/23vv4nCHqYUEKgliN/openapi.json