Mindfactory Scraper Actor

A flexible Apify Actor that scrapes product data from Mindfactory.de. You can provide:

• a list of product URLs,
• a category name (e.g. CPU, Grafikkarten), or
• a free-text keyword search (e.g. radeon 9079, i7-14700K).

The Actor returns a structured dataset with:

• Product URL
• Product title
• Current price
• Availability / stock status
• Manufacturer / brand

⚠️ Important Note on Site Protection

Mindfactory.de uses advanced bot protection (Cloudflare Turnstile) that may present a challenge page instead of the actual content, especially when accessed from data center IP addresses (like Apify workers). This can result in empty results for keyword or category searches.

To improve success chances:

• Use direct product URLs when possible (these are less likely to trigger challenges).
• Enable Apify Proxy (especially residential proxies) in the Actor’s settings. This often reduces the chance of being blocked.
• If keyword/category search fails, try again later or switch to providing explicit URLs.

The Actor includes automatic session handling (to obtain necessary cookies) and retry logic, but cannot guarantee bypassing the challenge in all cases.

How to Use

Input

Provide a JSON object with any combination of the following fields:

Field	Type	Description
urls	Array[string]	List of direct product page URLs to scrape.
category	String	Mindfactory category name (e.g. CPU, Mainboards).
keyword	String	Free-text search term (e.g. GTX 4090, Ryzen 9 7950X).

At least one of the fields must be non-empty.

Example Inputs

Direct URLs:

{
  "urls": [
    "https://www.mindfactory.de/product_info.php/AMD-Ryzen-7-7800X3D-Box_100100706.html",
    "https://www.mindfactory.de/product_info.php/Intel-Core-i7-14700K-Box_100101234.html"
  ]
}

Category:

{
  "category": "CPU"
}

Keyword:

{
  "keyword": "radeon 9079"
}

Output

Each item in the dataset contains:

{
  "url": "https://www.mindfactory.de/product_info.php/...",
  "title": "Product Name",
  "price": "429,90 €",
  "availability": "Auf Lager",
  "manufacturer": "AMD"
}

Running the Actor

In Apify Console

1. Go to the Actor’s page.
2. Open the Input tab, paste your JSON, and click Run.
3. After the run finishes, check the Dataset tab for results (download as JSON, CSV, Excel, etc.).

Via Apify API

curl -X POST "https://api.apify.com/v2/acts/wPjW1m4GzkWxnjo9h/runs?token=YOUR_API_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"keyword":"Intel Core i7-14700K"}'

Replace YOUR_API_TOKEN with your actual Apify API token.

Scheduling

Use the Schedule tab to run the Actor automatically (e.g., every 6 hours for price monitoring).

Technical Details

• The Actor uses httpcloak to bypass common Cloudflare challenges, but may still encounter the newer Turnstile challenge.
• It first attempts to fetch the homepage to obtain necessary cookies, then proceeds with the requested URLs.
• For keyword/category searches, it tries the internal JSON API, falls back to HTML search, and (if configured) uses a Google Custom Search Engine as a last resort.
• To use the Google CSE fallback, you must set two secrets in the Actor’s configuration:
  • GOOGLE_CSE_API_KEY: Your Google API key.
  • GOOGLE_CSE_CX: Your CSE ID (limited to site:www.mindfactory.de). If these are not set, the fallback is skipped.
• The Actor respects Apify’s proxy settings if configured (via Actor.create_proxy_configuration()).

Changelog

See the CHANGELOG.md for details.

License

This Actor is provided as-is. Feel free to modify and redistribute under the MIT License.

---

Note: If you encounter persistent issues, please check the Actor’s log for error messages. The most common cause is the site’s bot protection presenting a challenge page instead of the expected content.