Dataset Image Downloader & Uploader
No credit card required
Dataset Image Downloader & Uploader
No credit card required
Download image files from image URLs in your datasets and save them to a Zip file, Key-Value store, or directly your AWS S3 bucket.
Do you want to learn more about this Actor?
Get a demoActor failing
Closed
Hi all, it seems that this actor is failing, by creating bad URL hash. See the attached run. It says some are successful, but nothing is uploading. It is meant to be taking URLs from the instagram-hashtagger-scrapper and downloading the corresponding URL for me.
Hi, thanks for reaching out, we'll look into it and let you know what can be done.
Hi! Instagram image URLs valid for 1-2 days, so I made new sample run with mixed videos and images output from IG hashtags: https://console.apify.com/view/runs/3Mi1KcKQJNbtAKFLt
Data flow:
- we need to use either images array or displayUrl from dataset i.e. https://api.apify.com/v2/datasets/kya3nQTKUtFYNVhrk/items?clean=true&fields=images,displayUrl&format=json
- file names should be as in Instagram to track back image source i.e. 432560543_381465707993269_5410683957186223868_n.jpg
Sample Downloader run for the above: https://console.apify.com/view/runs/rLHnUrX0CnfgAXcx7
Please reuse input below (copy and paste as json input, then modify as necessary):
1{ 2 "convertWebpToPng": false, 3 "datasetId": "kya3nQTKUtFYNVhrk", 4 "fileNameFunction": "({ url }) => new URL(url).pathname.split(\"/\").pop()", 5 "imageCheckType": "content-type", 6 "noDownloadRun": false, 7 "pathToImageUrls": "images", 8 "preDownloadFunction": "({ data }) => data.map((item) => {\r\n if (!item.images?.length) {\r\n item.images = [item.displayUrl];\r\n }\r\n return item;\r\n})", 9 "proxyConfiguration": { 10 "useApifyProxy": true 11 }, 12 "s3CheckIfAlreadyThere": false, 13 "uploadTo": "zip-file", 14 "imageCheckMaxRetries": 6, 15 "maxConcurrency": 40, 16 "downloadTimeout": 15000, 17 "batchSize": 10000 18}
Lukáš KřivkaActor Metrics
36 monthly users
-
13 stars
>99% runs succeeded
2 days response time
Created in Nov 2018
Modified 23 days ago
