Pricing
Pay per usage
Download image files from image URLs in your datasets and save them to a Zip file, Key-Value store, or directly your AWS S3 bucket.
0.0 (0)
Pricing
Pay per usage
22
963
79
Issues response
27 days
Last modified
7 months ago
datasetIdstringOptional
Id of the dataset where the data are located. Image URLs will be extracted from there.
pathToImageUrlsstringOptional
Path from item object to an array or string where the URL(s) is/are located. Provide in "javascript style", e.g. "details[0].images
fileNameFunctionstringOptional
Function that specifies how will be image filename created from its URL. If you keep this empty, it will be md5 hash of the URL.
limitintegerOptional
Max items to load from the dataset. Use with offset to paginate over the data (can reduce memory requirement of large loads).
offsetintegerOptional
How many items to skip from the dataset. Use with limit to paginate over the data (can reduce memory requirement of large loads)
outputToEnumOptional
Where to save the data from input after possibly transforming them during the download process.
"no-output": string"key-value-store": string"dataset": stringoutputDatasetIdstringOptional
Name or ID of the dataset where the data will be saved. Only relevant if you want to output to dataset!
storeInputstringOptional
If you want to input the data from key-value store instead of dataset. Notation: storeId-recordKey, e.g. - kWdGzuXuKfYkrntWw-OUTPUT
uploadToEnumOptional
Where do you want to upload the image files
"zip-file": string"key-value-store": string"s3": string"no-upload": stringuploadStoreNamestringOptional
Key-value store name where the images will be upload. Empty field means it will be uploaded to the default key-value store
s3BucketstringOptional
Only relevant if you want to upload to S3! Name of the bucket where to upload.
s3AccessKeyIdstringOptional
Only relevant if you want to upload to S3! You can create these credentials for IAM user.
s3SecretAccessKeystringOptional
Only relevant if you want to upload to S3! You can create these credentials for IAM user.
s3CheckIfAlreadyTherebooleanOptional
This option is useful if you don't want to rewrite the same image. GET requests are also cheaper than PUT requests
preDownloadFunctionstringOptional
Function that specifies how will be the data transformed before downloading the image. The input and output of the function is the whole data array. You can skip downloading images of any item if you add skipItem: true field to it.
postDownloadFunctionstringOptional
Function that specifies how will be the data transformed before downloading the image. The input and output of the function is the whole data array. By default it adds either the file URL or errors array depending if the download was successfull.
imageCheckMaxRetriesintegerOptional
How many times should actor retry if the file it tries to download fails to pass the tests. Setting this too high can lead to unecessary loops.
Default value of this property is 6
imageCheckTypeEnumOptional
Type of the image check. If the image will not pass, the download will be retied with proxy and if that doesn't pass, the image is not uploaded.
"none": string"content-type": string"image-size": stringimageCheckMinSizeintegerOptional
Minimum size of the image to pass the image check test
imageCheckMinWidthintegerOptional
Minimim width of the image in pixels to pass the image check. Works only if the image check type is 'jimp'.
imageCheckMinHeightintegerOptional
Minimim height of the image in pixels to pass the image check. Works only if the image check type is 'jimp'.
maxConcurrencyintegerOptional
You can specify how many maximum parallel downloading/uploading requests will be running. Keep in mind that the limit is here to not overload the host server.
Default value of this property is 40
downloadTimeoutintegerOptional
How long we will wait to download each image
Default value of this property is 15000
batchSizeintegerOptional
Number of items loaded from dataset in one batch.
Default value of this property is 10000
convertWebpToPngbooleanOptional
If checked, the actor will automatically convert all webp type images to standard png. This increases the size of the image.