Download Files from URLs - Images, PDFs, Videos & ZIP
Pricing
from $3.50 / 1,000 file downloadeds
Download Files from URLs - Images, PDFs, Videos & ZIP
Bulk download files from a list of URLs or a dataset column: images, PDFs, videos, audio and documents, streamed to a key-value store and optionally packed into ZIP files. Each row gives the file URL, name, type, size and SHA-256. Failed URLs come back free with the reason.
Pricing
from $3.50 / 1,000 file downloadeds
Rating
0.0
(0)
Developer
NeverEmpty
Maintained by CommunityActor stats
0
Bookmarked
1
Total users
1
Monthly active users
3 days ago
Last modified
Categories
Share
Give it a list of file URLs, or a dataset from another Actor plus the name of the field that holds the URLs. It downloads every file and saves it to an Apify key-value store. Each file gets one row with a direct link to the saved copy, the file name, type, size in bytes and SHA-256. You can also ask for everything in a ZIP.
- Any file type. Images, PDFs, videos, audio, Office documents, archives. The type is read from the file's first bytes, not only from the URL, so a link without an extension still gets the right name and type.
- Straight from a scraper's results. Point it at a dataset (for example the output of a product or listing scraper) and the field with the image or file URL. Fields that hold a list of URLs, or objects with
url,srcorhref, are read in full. Nested fields work with dots (media.0.url). - One file or one ZIP.
filessaves each file as its own record with its own link.zippacks everything intofiles.zipwithout compression, so it is fast and every file stays byte-for-byte the same.bothdoes both. A large ZIP is split into parts (files-part2.zip, ...) so it always fits the run's disk. - Large files are streamed. Files are written to disk and uploaded from there, not held in memory. Measured on Apify at 256 MB memory: a 113 MB TIFF and a 134 MB TIFF in one run, saved as files and as a 2-part ZIP.
- Check the copy. Every row has the SHA-256 and the size of the saved file, so you can check it against the source or spot duplicate files.
- Clear file names. The file name comes from the URL or the server's
Content-Disposition. Two different files with the same name get-2,-3, ... Unsafe characters are replaced. - Not charged for files it could not save. 404, a web page instead of a file, too large, wrong type, blocked by robots.txt, needs a sign-in, no connection: each comes back as a free row that says why. A run where no file could be saved is not charged at all, not even the start fee.
- Only the URLs you give it. It does not crawl sites or follow links inside pages.
Measured on Apify at 256 MB memory: 100 Wikimedia Commons photos (23 MB) in 43 s; 3 files of 250 MB in total with both in 90 s.
Input
| Field | Type | Default | What it does |
|---|---|---|---|
urls | array of URLs | (example photo) | Direct links to the files, one per line. If this and the dataset are both left out, one example photo from Wikimedia Commons is downloaded and the log says so. |
datasetId | dataset | none | An Apify dataset whose rows hold file URLs, for example a scraper's results. Its URLs are added to urls. |
datasetField | text | url | The field of each dataset row that holds the URL (imageUrl, images, media.0.url, ...). |
saveAs | files / zip / both | files | Separate records, one ZIP (split into parts when large), or both. |
fileTypes | array | all types | Keep only these kinds: image, video, audio, pdf, document, archive, other, or extensions such as jpg, mp4, docx. Other files come back as free type-not-allowed rows. |
skipWebPages | boolean | true | An address that returns an HTML page instead of a file is not saved (free web-page row). Off: the HTML is saved like any other file. |
outputKeyValueStoreId | key-value store | this run's store | Where to save the files. Choose your own store to keep the files and links after your plan's data retention period. |
maxFileMegabytes | 1-4000 | 500 | A file larger than this is not saved (free too-large row). |
requestTimeoutSecs | 5-600 | 60 | A download that receives no data for this long is stopped and tried again. A slow but moving download is not stopped. |
maxConcurrency | 1-16 | 4 | Files downloaded at the same time. Lower it if a site answers 429. |
Example:
{"datasetId": "<your scraper's dataset id>","datasetField": "imageUrl","saveAs": "both","fileTypes": ["image"]}
Output
One dataset row per URL. outputUrl opens the saved file directly (a signed link, no API token needed). zipUrl and zipEntry say which ZIP part holds it.
{"status": "ok","position": 2,"url": "https://upload.wikimedia.org/wikipedia/commons/f/f0/Paris_view_from_Eiffel.jpg","finalUrl": "https://upload.wikimedia.org/wikipedia/commons/f/f0/Paris_view_from_Eiffel.jpg","outputUrl": "https://api.apify.com/v2/key-value-stores/<store>/records/Paris_view_from_Eiffel.jpg?signature=<signature>","key": "Paris_view_from_Eiffel.jpg","keyValueStoreId": "<store>","fileName": "Paris_view_from_Eiffel.jpg","contentType": "image/jpeg","fileType": "image","extension": "jpg","bytes": 2762548,"sha256": "4b4a6566a60033ac051e2b5a6cc8a228e165630738e73fe046b3f43c5e6e83e8","zipUrl": "https://api.apify.com/v2/key-value-stores/<store>/records/files.zip?signature=<signature>","zipEntry": "files.zip/Paris_view_from_Eiffel.jpg","httpStatus": 200,"redirects": 0,"note": null}
All columns: status, position (order in your input), url, finalUrl (after redirects), outputUrl, key, keyValueStoreId, fileName, contentType, fileType, extension, bytes, sha256, zipUrl, zipEntry, httpStatus, redirects, durationMs, note, downloadedAt.
The same URL given twice is downloaded and charged once (the log says how many were skipped).
Free rows (not charged)
status | When |
|---|---|
not-found | The site says the file does not exist (404 or 410). |
web-page | The address returned an HTML page, not a file (with skipWebPages on). |
type-not-allowed | The file is not one of the fileTypes you chose. |
too-large | Larger than maxFileMegabytes, or larger than the run's temporary disk (the row says to run with more memory). |
empty-file | The server sent a file with no content. |
robots-disallowed / robots-unreachable | The site's robots.txt does not allow automated reading of that address, or could not be read. |
login-required / blocked | The file needs a sign-in, or the site refused the reader or showed a check page. It does not sign in, solve check pages or switch to proxies. |
http-error / unreachable / unreadable / download-failed | Another HTTP error, no connection, no usable answer after retries (429 and 5xx are retried), or a download that broke off. |
storage-failed | The file was downloaded but could not be saved to the key-value store. |
bad-input | Not an http(s) URL, a private or internal network address, or a dataset field that holds no URL (the row says why). |
time-limit / budget-reached | The run was close to its timeout or to the maximum total charge you set; the remaining files were not started and not charged. |
Pricing
Pay per event:
- File downloaded: $0.0049 per file on the Free plan, down to $0.0035 on higher plans. Charged only when the file is saved and its row is returned. Includes the first 10 MB of the file.
- Each further 10 MB: $0.00295 per 10 MB above the first 10 MB on the Free plan, down to $0.0023. A 25 MB video counts 2 extra blocks; a 4 MB photo counts none. This is counted per stored copy: with Save as = both (separate files and the ZIP) each file is stored twice, so a 25 MB video counts 4 extra blocks. Files under 10 MB cost the same in every mode.
- Run start: $0.006 per run on the Free plan, down to $0.0044. Charged once, only in a run that saves at least one file.
Platform usage (compute, storage writes and data transfer) is included; you do not pay it on top. Before each file is downloaded, the Actor checks that the maximum total charge you set has room for it; a file that would go over is not downloaded and comes back as a free budget-reached row.
Examples on the Free plan: 1,000 product photos under 10 MB each = $4.906. One 100 MB video = $0.006 + $0.0049 + 9 x $0.00295 = $0.038 as files or as a ZIP, or $0.006 + $0.0049 + 18 x $0.00295 = $0.064 with both.
Memory and large files
The default memory is 256 MB. Memory is not what limits file size, because files are streamed. Disk is: the run's temporary disk grows with its memory (a run at 256 MB memory had 512 MB of disk, and the log prints the free disk at start). Measured at 256 MB: files up to 134 MB each, 250 MB in one run, saved as files and ZIP. For single files larger than about 200 MB, run with more memory. If a file does not fit, it comes back as a free too-large row that says so; the run does not crash and the other files are kept.
Use it from code
curl -X POST "https://api.apify.com/v2/acts/neverempty~download-files-from-urls/run-sync-get-dataset-items?token=<YOUR_TOKEN>" \-H "Content-Type: application/json" \-d '{"urls":["https://example.com/report.pdf","https://example.com/photo.jpg"]}'
The response is the dataset rows; outputUrl is the saved file.
Notes
- Files are fetched with a plain HTTP client from Apify's own network, following each site's robots.txt. No proxies are used to get around a refusal.
- Addresses that point to private or internal networks are not requested.
- Only download files you have the right to copy.
Support
Found a problem or need another option? Open an issue on the Issues tab of this Actor.