Redbubble Image Downloader

This project is an Actor for the Apify platform that crawls and downloads images from Redbubble based on specified search terms. It uses Puppeteer for web scraping and can upload the downloaded images to any S3-compatible object storage service.

YES! Finally the feature you all have been waiting for! Images will be downloaded and uploaded to your S3-compatible storage service.

What do I need to use this?

1. An Apify account
2. An S3-compatible storage service
   • I am using scaleway in the examples, but any S3-compatible storage will work, such as AWS S3, Digital Ocean Spaces, Google Cloud Storage, Azure Blob Storage, etc.
   • If you are using scaleway, you can create an S3-compatible storage bucket here
3. A list of search terms
   • Use my other Actor called "Redbubble-keyword-scraper" (https://apify.com/lime_incline/redbubble-keyword-scraper) where you can input a list of keywords, and it will scrape the most popular images for each keyword.
   • It will output the search terms json array, which you can copy and paste into the input of this Actor.

Example Input

Search Terms Json Array

• Output from the "Redbubble-keyword-scraper" Actor
• for example:

1[
2    {
3        "searchTerm": "Japanese",
4        "page": 1,
5        "results": [
6            "https://www.redbubble.com/i/t-shirt/The-Great-Ramen-off-Kanagawa-by-ilustrata/27578492.QUQES.XYZ",
7            "https://www.redbubble.com/i/t-shirt/Great-Sushi-Dragon-by-ilustrata/31848323.QUQES.XYZ",
8            "https://www.redbubble.com/i/t-shirt/The-Great-Retro-Wave-by-vincenttrinidad/37987911.IJ6L0.XYZ",
9            "https://www.redbubble.com/i/t-shirt/Catana-by-vincenttrinidad/43638443.IJ6L0.XYZ"
10        ]
11    },
12    {
13        "searchTerm": "Attack on Titan",
14        "page": 1,
15        "results": [
16            "https://www.redbubble.com/i/t-shirt/Persona-5-queen-Makoto-by-cooltalal111/71366321.IJ6L0.XYZ",
17            "https://www.redbubble.com/i/t-shirt/Survey-Corps-by-Crossroads-Art/160073296.IJ6L0.XYZ",
18            "https://www.redbubble.com/i/t-shirt/Outside-the-Wall-Black-and-White-by-whycaraboutit/103299368.WFLAH.XYZ",
19            "https://www.redbubble.com/i/t-shirt/The-Last-Air-Bender-by-christopherML/115312187.WFLAH",
20            "https://www.redbubble.com/i/t-shirt/War-Against-Monsters-by-ewanwhitt/80444666.IJ6L0",
21        ]
22    }
23]

• Copy and paste the entire thing into the input field

S3 Bucket Name

• Name of your S3-compatible storage bucket
• for example:
• redbubble-scrape https://ibb.co/Qn4734H

Access Key ID and Secret Access Key

• Your Access Key ID for S3-compatible storage

• Your Secret Access Key for S3-compatible storage

• Scaleway Example:

• https://ibb.co/P9CNfPj

• Input here: https://ibb.co/qnytFh5

Endpoint

• Endpoint for S3-compatible storage

• Get from bucket settings

• Region is optional (use if you need to specify the region)

Proxy Configuration

For the proxy configuration, you can use any proxy that is compatible with Apify. Residential proxies are recommended. But datacenter proxies work most of the time too and are faster

Customization

• Concurrency: Adjust maxConcurrency to control how many pages are processed in parallel.
• Proxy: Modify the proxyConfiguration to use your own proxies or Apify's proxy service.

Tips for Effective Use

• Start with a small number of search terms to test the Actor's performance.
• Monitor your storage usage to ensure you have enough capacity for the downloaded images.
• Use specific search terms to get more targeted results.
• Regularly check Redbubble's robots.txt and terms of service to ensure compliance.

By following these steps and customizing the input as needed, you can easily use this Actor to download and store images from Redbubble based on your specific search terms.