Dataset Image Downloader & Uploader avatar

Dataset Image Downloader & Uploader

Try for free

No credit card required

View all Actors
Dataset Image Downloader & Uploader

Dataset Image Downloader & Uploader

lukaskrivka/images-download-upload
Try for free

No credit card required

Download image files from image URLs in your datasets and save them to a Zip file, Key-Value store, or directly your AWS S3 bucket.

Do you want to learn more about this Actor?

Get a demo
SW

Actor failing

Closed

seamless_webbing opened this issue
7 months ago

Hi all, it seems that this actor is failing, by creating bad URL hash. See the attached run. It says some are successful, but nothing is uploading. It is meant to be taking URLs from the instagram-hashtagger-scrapper and downloading the corresponding URL for me.

paja avatar

Hi, thanks for reaching out, we'll look into it and let you know what can be done.

alexey avatar

Hi! Instagram image URLs valid for 1-2 days, so I made new sample run with mixed videos and images output from IG hashtags: https://console.apify.com/view/runs/3Mi1KcKQJNbtAKFLt

Data flow:

Sample Downloader run for the above: https://console.apify.com/view/runs/rLHnUrX0CnfgAXcx7

Please reuse input below (copy and paste as json input, then modify as necessary):

1{
2  "convertWebpToPng": false,
3  "datasetId": "kya3nQTKUtFYNVhrk",
4  "fileNameFunction": "({ url }) => new URL(url).pathname.split(\"/\").pop()",
5  "imageCheckType": "content-type",
6  "noDownloadRun": false,
7  "pathToImageUrls": "images",
8  "preDownloadFunction": "({ data }) => data.map((item) => {\r\n    if (!item.images?.length) {\r\n        item.images = [item.displayUrl];\r\n    }\r\n    return item;\r\n})",
9  "proxyConfiguration": {
10    "useApifyProxy": true
11  },
12  "s3CheckIfAlreadyThere": false,
13  "uploadTo": "zip-file",
14  "imageCheckMaxRetries": 6,
15  "maxConcurrency": 40,
16  "downloadTimeout": 15000,
17  "batchSize": 10000
18}
Developer
Maintained by Apify
Actor metrics
  • 24 monthly users
  • 12 stars
  • 98.6% runs succeeded
  • 2 days response time
  • Created in Nov 2018
  • Modified about 16 hours ago