# Bulk File Downloader — Download Files from URLs (`nexgensignal/bulk-file-downloader`) Actor

Download files in bulk from a list of links, or every PDF, ZIP, CSV or Excel file a page links to. Each file is saved with a download link, file name, size, type, SHA-256, Last-Modified and ETag. Duplicates are skipped. Part of NexGen Signal — data tools and official-source data products

- **URL**: https://apify.com/nexgensignal/bulk-file-downloader.md
- **Developed by:** [NexGen Signal](https://apify.com/nexgensignal) (community)
- **Categories:** Integrations, Automation, Developer tools
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $33.50 / 1,000 file downloadeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 📦 Bulk File Downloader — Download Files from URLs

Give a list of file links, or web pages plus the file types you want from them, and get **each file downloaded, checksummed and saved** with a download link: file name, extension, content type, size, SHA-256, Last-Modified and ETag.

Any type works: PDF, ZIP, CSV, Excel, Word, JSON, XML, audio, video. Set `linkExtensions` (for example `pdf`, `zip`, `xlsx`) and every link on your pages with those extensions is collected and downloaded. Files are streamed to disk and to storage, so large files (up to 2 GB) do not need large memory. The same file reached by two links is delivered once.

The unit of output is one file delivered. Files that are skipped (too big, wrong type, duplicate, blocked by robots.txt, unreachable) are not billed.

### Who buys this and for what job

- **Analysts** pulling every report, form or data file a government or company page links to.
- **Compliance and legal teams** archiving published documents with hashes and dates as evidence.
- **Data engineers** fetching CSV, ZIP and Excel files into pipelines on a schedule.
- **Researchers** collecting papers and attachments from a list of pages.

Buyers usually search for this as **"bulk file downloader"**, **"download files from URL list"** or **"download all PDFs from a website"**.

### 📊 Sample Output

![Table of downloaded files: file name, extension, bytes, last modified and file link.](https://api.apify.com/v2/key-value-stores/IXCaMKjxSmUTLHhmq/records/bulk-file-downloader.png)

| file_name | extension | content_type | bytes | last_modified |
|---|---|---|---|---|
| 4.5_week.csv | csv | text/csv | 26004 | 2026-10-17T04:10:55Z |
| fw4.pdf | pdf | application/pdf | 208845 | 2025-12-11T14:10:39Z |

Rows from live run `h1ZDF1gkKSmBbqW2O` on Apify (the Store example: an IRS form and a USGS earthquake CSV). Notice the Last-Modified dates: the IRS form is from December 2025, the USGS file was refreshed minutes before the run.

### ✅ What you get

Every field of each record:

- `record_id`, `sha256` — file hash (record_id is its first 24 characters).
- `url`, `final_url`, `found_on_page` — the link, where the file was served from after redirects, and the page it was found on (for page links).
- `file_name`, `extension`, `content_type`, `bytes` — the server's file name (Content-Disposition, else the address), extension, type and size.
- `last_modified`, `etag` — the server's Last-Modified date (UTC) and ETag, for change tracking.
- `file_key`, `file_url` — key and link of the saved file in the run's key-value store (empty when saveFiles is off).
- `downloaded_at` — when it was downloaded (UTC).

### ⚙️ Sample inputs

**Two direct files (the Store example)**

```json
{
  "fileUrls": ["https://www.irs.gov/pub/irs-pdf/fw4.pdf", "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv"],
  "maxFiles": 10
}
```

Each link is downloaded as it is.

**Every PDF a page links to**

```json
{
  "fileUrls": ["https://www.irs.gov/forms-instructions-and-publications"],
  "linkExtensions": ["pdf"],
  "maxFiles": 100
}
```

Collects the page's .pdf links, in page order.

**Check files for changes without storing them**

```json
{
  "fileUrls": ["https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv"],
  "saveFiles": false,
  "maxFiles": 10
}
```

Size, hash, Last-Modified and ETag only.

Input fields:

| Field | Meaning |
|---|---|
| `fileUrls` | file links, or pages when linkExtensions is set (up to 10,000). |
| `linkExtensions` | file types to collect from pages, e.g. pdf, zip, csv. Empty = download every link as it is. |
| `maxLinksPerPage` | file links taken per page (default 100). |
| `allowedExtensions` | keep only these types (checked after download). Empty = all. |
| `maxFileMb` | skip files bigger than this (default 500, up to 2,000). |
| `saveFiles` | store files with links (default true). |
| `maxFiles` | stop after this many delivered files (default 1,000). Each delivered file is billed. |

### 🧾 JSON sample record

```json
{
  "record_id": "9862b456ff4fb444090d3045",
  "url": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv",
  "final_url": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv",
  "found_on_page": null,
  "file_name": "4.5_week.csv",
  "extension": "csv",
  "content_type": "text/csv",
  "bytes": 26004,
  "sha256": "9862b456ff4fb444090d3045d9a1f656e3f317f1708cd8c0fc64d2b5ea9a51d9",
  "last_modified": "2026-10-17T04:10:55Z",
  "etag": null,
  "file_key": "file-9862b456ff4fb444090d3045.csv",
  "file_url": "https://api.apify.com/v2/key-value-stores/j8yFMcW2LaFlA6FtL/records/file-9862b456ff4fb444090d3045.csv",
  "downloaded_at": "2026-10-17T04:10:55Z"
}
```

### 🔧 How it works

Files are downloaded 4 at a time and streamed to a temporary file while the SHA-256 is computed, then streamed into the run's key-value store under `file-<hash>.<ext>`. The temporary file is deleted straight away.

Every run reads each host's robots.txt first: a path it disallows is skipped, listed in the run receipt and never charged. Only public internet addresses are fetched (no private, loopback or cloud-metadata addresses, checked again on every redirect). No logins, no CAPTCHA solving, no proxies.

Saved files go to the run's own key-value store. If your Apify account keeps storages private, add your API token to the link (`?token=…`) or open the store from the run in Apify Console.

Delivery is pushed first and charged after. The run receipt (`RUN_RECEIPT`) lists every link with its outcome (and, for pages, how many file links were found and what happened to each), the robots.txt result per host, and the delivered and charged counts.

### 💰 Pricing example

Per file delivered, no start fee, no monthly charge:

| Apify plan | Price per file |
|---|---|
| Free | $0.05 |
| Bronze | $0.045 |
| Silver | $0.04 |
| Gold, Platinum, Diamond | $0.0335 |

- 10 files: $0.50 on the Free plan, $0.335 on Gold.
- 1,000 files: $50.00 on Free, $33.50 on Gold.
- 10,000 files: $500.00 on Free, $335.00 on Gold.

**Coverage and size.** Proof runs on 17 Oct 2026: 40 IRS PDFs collected from two IRS pages in about 7 seconds; a 3.1 MB file was skipped with maxFileMb 1; a second link to the same CSV was recognised as the same file and not billed again.

### ⚖️ Legal & ToS

Files belong to their owners. You choose the links. Make sure you have the right to download and use the content you point it at; the actor fetches only what you give it (and, where you turn it on, what those pages link to).

Every run reads each host's robots.txt first: a path it disallows is skipped, listed in the run receipt and never charged. Only public internet addresses are fetched (no private, loopback or cloud-metadata addresses, checked again on every redirect). No logins, no CAPTCHA solving, no proxies.

### ❓ FAQ

**Does it unzip archives?**
No. Files are stored exactly as downloaded.

**How big can a file be?**
Up to 2,000 MB (maxFileMb). Files are streamed, not held in memory.

**Does it follow links more than one page deep?**
No. Pages you give are read once for file links; it does not crawl.

### 🆘 Troubleshooting

- **file_url answers 403** — Your account keeps storages private: add `?token=YOUR_TOKEN` or open the store from the run.
- **A page link was saved as an .html file** — Set linkExtensions to collect the files the page links to.
- **SOURCE UNAVAILABLE** — None of your links could be reached after retries. Nothing was charged.

### 🔗 Related Actors

- [Website Tech Stack Detector](https://apify.com/nexgensignal/website-tech-stack-api) — what a site is built with
- [Website Contact Scraper](https://apify.com/nexgensignal/bulk-website-contact-extractor) — contact details from website lists
- [PR Newswire Scraper](https://apify.com/nexgensignal/press-release-monitor-api) — press releases by company or topic

🏢 **About NexGen Signal** — data tools and official-source data products on Apify. Browse the catalog: [apify.com/nexgensignal](https://apify.com/nexgensignal)

### ⭐ Found this useful?

If this actor saved you time, a rating on its Apify Store page helps other buyers find it.

Part of NexGen Signal — data tools and official-source data products.

# Actor input Schema

## `fileUrls` (type: `array`):

Direct file links, or web pages when linkExtensions is set. Up to 10,000 per run.

## `linkExtensions` (type: `array`):

When a link is a web page, download the files it links to with these extensions, e.g. pdf, zip, csv, xlsx. Empty = every link is downloaded as it is.

## `maxLinksPerPage` (type: `integer`):

Most file links to take from one page, in page order.

## `allowedExtensions` (type: `array`):

Keep only files with these extensions (checked after download, using the server's file name or content type). Empty = all types.

## `maxFileMb` (type: `integer`):

Skip files bigger than this.

## `saveFiles` (type: `boolean`):

Store each file in the run's key-value store with a link. Off = details only (size, type, hash, dates).

## `maxFiles` (type: `integer`):

Stop after this many files are delivered. Each delivered file is billed; skipped ones are not.

## Actor input object example

```json
{
  "fileUrls": [
    "https://www.irs.gov/pub/irs-pdf/fw4.pdf",
    "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv"
  ],
  "linkExtensions": [],
  "maxLinksPerPage": 100,
  "allowedExtensions": [],
  "maxFileMb": 500,
  "saveFiles": true,
  "maxFiles": 10
}
```

# Actor output Schema

## `results` (type: `string`):

The delivered (and billed) records.

## `receipt` (type: `string`):

Per-link outcome, robots result, delivered and charged counts.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "fileUrls": [
        "https://www.irs.gov/pub/irs-pdf/fw4.pdf",
        "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv"
    ],
    "maxFiles": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("nexgensignal/bulk-file-downloader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "fileUrls": [
        "https://www.irs.gov/pub/irs-pdf/fw4.pdf",
        "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv",
    ],
    "maxFiles": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("nexgensignal/bulk-file-downloader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "fileUrls": [
    "https://www.irs.gov/pub/irs-pdf/fw4.pdf",
    "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.csv"
  ],
  "maxFiles": 10
}' |
apify call nexgensignal/bulk-file-downloader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nexgensignal/bulk-file-downloader"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/iLgu4FnMp9FugZ6Na/builds/7fXtWTDtgPnxziU01/openapi.json