Dataset Image Downloader & Uploader
Pricing
Pay per usage
Dataset Image Downloader & Uploader
Download image files from image URLs in your datasets and save them to a Zip file, Key-Value store, or directly your AWS S3 bucket.
0.0 (0)
Pricing
Pay per usage
18
Monthly users
85
Runs succeeded
>99%
Response time
12 days
Last modified
2 months ago
Actor failing
Hi all, it seems that this actor is failing, by creating bad URL hash. See the attached run. It says some are successful, but nothing is uploading. It is meant to be taking URLs from the instagram-hashtagger-scrapper and downloading the corresponding URL for me.
Hi, thanks for reaching out, we'll look into it and let you know what can be done.
Hi! Instagram image URLs valid for 1-2 days, so I made new sample run with mixed videos and images output from IG hashtags: https://console.apify.com/view/runs/3Mi1KcKQJNbtAKFLt
Data flow:
- we need to use either images array or displayUrl from dataset i.e. https://api.apify.com/v2/datasets/kya3nQTKUtFYNVhrk/items?clean=true&fields=images,displayUrl&format=json
- file names should be as in Instagram to track back image source i.e. 432560543_381465707993269_5410683957186223868_n.jpg
Sample Downloader run for the above: https://console.apify.com/view/runs/rLHnUrX0CnfgAXcx7
Please reuse input below (copy and paste as json input, then modify as necessary):
1{ 2 "convertWebpToPng": false, 3 "datasetId": "kya3nQTKUtFYNVhrk", 4 "fileNameFunction": "({ url }) => new URL(url).pathname.split(\"/\").pop()", 5 "imageCheckType": "content-type", 6 "noDownloadRun": false, 7 "pathToImageUrls": "images", 8 "preDownloadFunction": "({ data }) => data.map((item) => {\r\n if (!item.images?.length) {\r\n item.images = [item.displayUrl];\r\n }\r\n return item;\r\n})", 9 "proxyConfiguration": { 10 "useApifyProxy": true 11 }, 12 "s3CheckIfAlreadyThere": false, 13 "uploadTo": "zip-file", 14 "imageCheckMaxRetries": 6, 15 "maxConcurrency": 40, 16 "downloadTimeout": 15000, 17 "batchSize": 10000 18}
Pricing
Pricing model
Pay per usageThis Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.
