Dataset Image Downloader & Uploader avatar
Dataset Image Downloader & Uploader

Pricing

Pay per usage

Go to Store
Dataset Image Downloader & Uploader

Dataset Image Downloader & Uploader

Developed by

Lukáš Křivka

Lukáš Křivka

Maintained by Community

Download image files from image URLs in your datasets and save them to a Zip file, Key-Value store, or directly your AWS S3 bucket.

0.0 (0)

Pricing

Pay per usage

20

Total users

880

Monthly users

88

Runs succeeded

>99%

Issues response

8.3 days

Last modified

5 months ago

SW

Actor failing

Closed

seamless_webbing opened this issue
a year ago

Hi all, it seems that this actor is failing, by creating bad URL hash. See the attached run. It says some are successful, but nothing is uploading. It is meant to be taking URLs from the instagram-hashtagger-scrapper and downloading the corresponding URL for me.

paja avatar

Hi, thanks for reaching out, we'll look into it and let you know what can be done.

alexey avatar

Hi! Instagram image URLs valid for 1-2 days, so I made new sample run with mixed videos and images output from IG hashtags: https://console.apify.com/view/runs/3Mi1KcKQJNbtAKFLt

Data flow:

Sample Downloader run for the above: https://console.apify.com/view/runs/rLHnUrX0CnfgAXcx7

Please reuse input below (copy and paste as json input, then modify as necessary):

{
"convertWebpToPng": false,
"datasetId": "kya3nQTKUtFYNVhrk",
"fileNameFunction": "({ url }) => new URL(url).pathname.split(\"/\").pop()",
"imageCheckType": "content-type",
"noDownloadRun": false,
"pathToImageUrls": "images",
"preDownloadFunction": "({ data }) => data.map((item) => {\r\n if (!item.images?.length) {\r\n item.images = [item.displayUrl];\r\n }\r\n return item;\r\n})",
"proxyConfiguration": {
"useApifyProxy": true
},
"s3CheckIfAlreadyThere": false,
"uploadTo": "zip-file",
"imageCheckMaxRetries": 6,
"maxConcurrency": 40,
"downloadTimeout": 15000,
"batchSize": 10000
}