Bulk File Downloader: Download Files from URLs to ZIP

This bulk file downloader takes a list of direct file links (or web pages that link to files) and saves every file to a key-value store with a public link, its real type, its size and a SHA-256 hash. It removes duplicates, lets you allow or block file extensions, and packs everything into one ZIP. It is a bulk file downloader API for PDFs, Word and Excel files, CSVs, audio, video, archives, images and any other file type. No browser, no login and no API key.

Rights and responsibility. You are responsible for having the right to download and reuse the files you fetch, and for following the terms of the sites you point it at. This Actor does not log in, sends no cookies and no forged Referer, does not work around hotlink protection, and does not download from video or file-sharing pages (those return web pages, which are never saved). If a host answers 401, 403 or 429, you get a failed row with that reason. For pageUrls it reads and obeys the site's robots.txt.

What can this bulk file downloader do

• Download files from URLs. One dataset row per URL with status: downloaded, duplicate, skipped or failed, and a reason when it is not downloaded.
• Download all the files linked on a page. Add page links in pageUrls: the Actor reads <a href>, <source>, <embed>, <object>, <audio> and <video> links, resolves relative links (including <base href>) and downloads the ones that end in a wanted extension (pdf, docx, xlsx, zip, csv, mp3, mp4...).
• Know what you really got. The type is detected from the file's bytes, not the file name or the server header. A .docx link is stored as docx, a download link without an extension that sends a PDF is stored as pdf, and a .pdf link that sends an error page is reported as failed instead of being saved as a "PDF".
• Allow or block extensions. allowedExtensions keeps only the types you want. blockedExtensions refuses programs, installers, scripts and web pages by default (exe, msi, bat, cmd, sh, dll, jar, js, html and more). Web pages (HTML) are never saved, and a program or script you unblock is stored as .bin, never as .exe.
• Remove duplicates. Identical files (same SHA-256) are delivered once; later copies get status duplicate with a link to the first copy, and are free.
• URL to ZIP. Turn on zip to get all delivered files in one files.zip with a manifest.csv (file, source URL, source page, SHA-256, size), with a public link. File names inside the ZIP are sanitised (no folders, no ..), and the ZIP has a size cap.
• Polite and robust. Per-host limits and delay, retries with backoff, a size cap checked on the header and while streaming, redirects checked hop by hop, and blocked private addresses.

How to download files from URLs

1. Paste direct file links in File URLs (urls), one per line, and/or page links in pageUrls.
2. Optionally set allowedExtensions, maxFileSizeMb, dedupe, zip and keepOriginalNames.
3. Run it. Each file is saved in the run's key-value store under <sha256>.<ext>, and the dataset lists its public link (fileUrl). With zip on, the SUMMARY record has zipUrl.

Input example

{
  "urls": [
    "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
    "https://people.sc.fsu.edu/~jburkardt/data/csv/addresses.csv",
    "https://www.w3.org/Icons/w3c_main.png",
    "https://jsonplaceholder.typicode.com/todos/1"
  ],
  "zip": true
}

More examples: "pageUrls": ["https://example.com/reports"], "allowedExtensions": ["pdf", "xlsx"] (every PDF and Excel file linked on a page), "keepOriginalNames": true (ZIP entries named like the originals), "maxFileSizeMb": 20, "maxFiles": 500 (tighter limits for a big list).

Output

One dataset row per URL:

{
  "url": "https://www.w3.org/Icons/w3c_main.png",
  "finalUrl": "https://www.w3.org/assets/logos/w3c-2025-transitional/w3c-full-315x48.png",
  "status": "downloaded",
  "reason": null,
  "contentType": "image/png",
  "extension": "png",
  "sizeBytes": 7816,
  "sha256": "03ab7291bbb7f7e8bb31f13144e107d48293a87e4bcfb2f25e77cdb2f4e183ab",
  "fileUrl": "https://api.apify.com/v2/key-value-stores/<storeId>/records/03ab7291...183ab.png",
  "sourcePage": null,
  "fileName": "w3c-full-315x48.png",
  "key": "03ab7291bbb7f7e8bb31f13144e107d48293a87e4bcfb2f25e77cdb2f4e183ab.png",
  "index": 3
}

• status: downloaded (charged), duplicate (same content as an earlier file, or the same URL listed twice), skipped (your rules: extension, size, spending limit, robots.txt) or failed (network error, HTTP error, a web page instead of a file, a file that is not what its URL says, blocked address).
• reason: why a URL was not downloaded, or a note on a downloaded file (for example URL ends in .jpg but the file is png; saved as .png).
• extension is what the file is stored with: from its bytes, then the URL, the Content-Disposition name and the server's content type. bin means unknown, or a program or web extension that was refused.
• The key is sha256 plus the extension, so it is always safe and never contains parts of the URL. fileName is the sanitised original name.
• sourcePage is the page a file link was found on (pageUrls), empty for URLs you listed yourself.
• The SUMMARY record has the counts, the counts per extension, the charged units, zipUrl and any notes.

Bulk file downloader API

Call this Actor from the Apify API, Python, JavaScript, Make, Zapier or n8n. Example with the Apify Python client:

from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("data-done-right/bulk-file-downloader").call(run_input={
    "urls": ["https://example.com/report.pdf", "https://example.com/data.csv"],
    "allowedExtensions": ["pdf", "csv"], "zip": True})
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row["status"], row["url"], row["fileUrl"])
summary = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("SUMMARY")["value"]
print(summary["zipUrl"])

Files are read back from the run's key-value store (client.key_value_store(run["defaultKeyValueStoreId"])) or by their fileUrl.

Use it in Claude, ChatGPT and Cursor

Connect Apify's MCP server (mcp.apify.com) and your AI assistant can fetch files for you. For example: "download these 40 PDF links, skip duplicates, and give me the ZIP link". The only input you need is a list of file URLs (urls) or a page URL (pageUrls).

Bulk download files: use cases

• Archive documents: turn a spreadsheet of PDF, DOCX or XLSX links into files you own, with hashes to verify them.
• Collect open data: download the CSV, JSON and ZIP files a statistics office or data portal links on its pages.
• Mirror your own assets: back up the images, audio and video files of a site you run.
• Feed other tools: hand the stored files to a PDF-to-text, file conversion or dataset pipeline.
• Check links: see which file URLs are dead, are really web pages, or are too large.

URL to ZIP

Turn on zip and every delivered file is also put into files.zip in the run's key-value store, with manifest.csv listing each entry's source URL, source page, SHA-256 and size. By default entries are named 0001-name.ext; with keepOriginalNames they are name.ext (a counter is added if two files share a name). Names are sanitised: only letters, digits, dot, dash and underscore, never a folder or ... zipMaxMb (default 100, up to 300) caps the ZIP: files that would not fit are still saved and linked in the dataset, but left out of the ZIP, and the SUMMARY says how many. Already compressed formats (images, archives, Office files, audio, video) are stored without recompressing. The ZIP is free.

How much does it cost

Pay per event, only for files that are delivered:

• $0.0016 per downloaded file (file). 100 small files cost about $0.16.
• $0.0006 per started 10 MB block beyond the first 10 MB of each file (megabyte-block), so large files cost fairly. A 25 MB file starts 3 blocks and costs the file price plus 2 blocks, $0.0028. A file of 10 MB or less costs just the file price. 10 MB is 10,000,000 bytes, measured on the delivered file.
• Free: duplicates, skipped and failed URLs, the ZIP, the dataset and the SUMMARY record.

Paid Apify plans get 10% (Silver) to 20% (Gold and up) off. maxFiles caps the number of URLs, maxFileSizeMb caps each file, and if you set a maximum spend per run the Actor stops cleanly when it is reached, keeping the files already delivered (the remaining URLs get a skipped row).

FAQ

How do I download files from a list of URLs?

Put the links in urls and run. Every link gets a row; the files that were delivered have a fileUrl you can open or download. Turn on zip to also get everything in one ZIP.

How do I download all the PDFs (or other files) linked on a web page?

Put the page link in pageUrls and set allowedExtensions to the types you want, for example ["pdf"]. The Actor fetches the page (respecting its robots.txt), finds the links and downloads those files. Without allowedExtensions it follows links to documents, archives, audio, video and images. Links are read from the HTML as served, so links that a page only creates with JavaScript are not found. Links are not followed into other pages: it reads each page you list, one level, and it does not crawl.

Does it respect robots.txt?

For pageUrls, yes: a page that robots.txt disallows for the BulkFileDownloader agent (or *) is not fetched and gets a skipped row. If robots.txt cannot be read because of a server error, the page is not fetched either. Direct file links you list yourself in urls are not checked against robots.txt, because you chose them; you are responsible for the right to download them. Files linked from a page are fetched like any other file link: robots.txt is applied to the page, not to the files it points to.

Which files are refused?

Web pages (HTML) are never saved: a link that returns one gets a failed row that points you to pageUrls. Programs, installers and scripts (exe, msi, bat, cmd, com, scr, ps1, sh, dll, jar, vbs, hta, js, apk, dmg, pkg, deb, rpm) are blocked by default, both by the URL ending and by the file's first bytes. If you remove them from blockedExtensions they are stored as .bin with the type application/octet-stream, never with a program extension. Other types are stored with their real type only when it is a harmless download type; XML is stored as a plain download, and SVG files are saved (with their image type) only after a check that they contain no scripts or active content; otherwise they are skipped.

How is the extension decided?

From the bytes first (PDF, PNG, JPG, ZIP, MP3, MP4, GIF, WebP, 7z, RAR and more have fixed signatures). ZIP-based formats (docx, xlsx, pptx, odt, epub) take the extension from the URL, the Content-Disposition file name or the content type. Text formats without a signature (CSV, JSON, TXT) take it from the URL, then the file name, then the content type. A name ending in .php or .asp names the script, not the file, so it is ignored. If nothing says, the extension is bin.

Can it get files behind a login or bypass hotlink protection?

No. It sends no cookies, credentials or Referer, and it does not pretend to be a browser: its User-Agent says it is a bot. Links with a user name or password in the URL are refused. Pages of login-walled social sites (Instagram, Facebook, LinkedIn, X, TikTok) are not supported. A host that refuses the download gives a failed row with the HTTP status. Share links of cloud drives (Google Drive, Dropbox, OneDrive) usually return a web page, so they fail with a clear message; use the direct download link if the owner provides one.

How are duplicates found?

By the SHA-256 of the downloaded bytes. The same URL listed twice is fetched only once. Turn dedupe off to deliver (and be charged for) every copy.

How big can a file be?

100 MB by default and at most. The limit is checked on the Content-Length header and again while streaming, so an oversized or endless response is cut off early. Files that are over the limit are skipped and free. The ZIP can hold up to 300 MB.

How long are the file links kept?

Files are saved in the run's default key-value store, which expires after your plan's data retention period like other unnamed storages. Set outputStoreName to save them in a named store that stays until you delete it. Anyone who has a fileUrl can open it.

Is it safe to give it any URL?

Links to localhost, private networks, link-local and cloud-metadata addresses are blocked, including names that resolve to them and redirects into them (every redirect hop is checked again; up to 5 are followed). When you use a proxy, the proxy opens the connection, but the same address checks still run before every request.

Is it polite to the servers?

Requests to the same host start at least perHostDelayMs apart (250 ms by default) and at most perHostConcurrency (3 by default) run at once per host. So a higher concurrency only speeds up lists that are spread across several hosts. The timeoutSecs clock for a file starts when its turn at the host comes, not while it waits in line. Network errors, timeouts and 429/5xx answers are retried up to maxRetries times (2 by default) with backoff, and Retry-After is honored. 401, 403 and 404 are not retried.

Do I need an API key or login?

No.

Limits

• Direct links only: it does not click, log in, run JavaScript or crawl. Pages are read one level deep.
• 100 MB per file, 300 MB per ZIP, up to 10,000 URLs per run.
• Streaming sites, social networks and file-sharing pages are not supported.
• File type detection works on the first bytes. Plain text formats are identified by the URL or the server, so a CSV that is served as text/plain from a link without an extension is stored as txt.

Related tools

• Download Images from URLs: Bulk Image & File Downloader is the image-focused sibling: resize, convert to JPG, PNG or WebP and filter by dimensions.
• All-in-One File Converter: PDF, Office, Images, Audio & Video converts the files you download to other formats.
• PDF to Text & Markdown Converter with Automatic OCR reads the text in the PDFs you download.
• Merge Datasets & Dedupe: Combine, Filter, Export CSV & Excel combines the result rows of several runs.