AI Scraper Input Generator

What does AI Scraper Input Generator do?

This Actor takes a natural language prompt (for example: "wireless noise-canceling headphones under $200") and uses OpenAI to generate the requested number of search URLs for a target e-commerce website such as eBay or Amazon. It then packages those URLs as a ready-to-run input for another Apify scraper, so you can chain it straight into an Actor such as eBay Items Scraper.

Because it runs on Apify, you get API access, scheduling, monitoring, dataset outputs, and easy integrations (Make, Zapier, webhooks, and more) without building your own infrastructure.

Why use AI Scraper Input Generator?

• Turn unstructured user intent into a ready-to-run scraper input.
• Generate multiple targeted search-URL variations from one request.
• Feed downstream Apify scrapers directly, with no manual URL building.
• Run on schedule and trigger downstream systems with predictable JSON output.
• Keep logic in one place instead of duplicating prompt-to-URL rules across apps.

How to use AI Scraper Input Generator

1. Open the Actor in Apify Console.
2. Set the OPENAI_API_KEY environment variable in the Actor settings (Settings → Environment variables, mark it as secret). For local runs, put it in a .env file in the project root.
3. Enter a plain-English prompt in the Input tab.
4. Set targetActor - the Actor that will consume the generated URLs (for example dtrungtin/ebay-items-scraper).
5. Optional: set numberOfUrls and websiteDomain (for example www.ebay.com or www.amazon.com). If you leave websiteDomain empty, it is inferred from the target Actor's example start URLs, or guessed by AI from the target Actor's details when no example URLs exist.
6. Run the Actor.
7. Open the Output tab or Dataset API URL to retrieve the generated URLs.

Input

Set input in the Actor Input tab.

{
    "prompt": "used mirrorless camera body in excellent condition",
    "targetActor": "https://apify.com/dtrungtin/ebay-items-scraper",
    "numberOfUrls": 3
}

Field notes:

• prompt (required): user intent to transform into search URLs.
• targetActor (required): Actor that will consume the generated URLs, as a full Apify Store URL (https://apify.com/dtrungtin/ebay-items-scraper) or a path (dtrungtin/ebay-items-scraper).
• numberOfUrls (optional): how many search URLs to generate, 1-50 (default 3).
• websiteDomain (optional): target website domain, e.g. www.ebay.com, www.ebay.co.uk, or www.amazon.com. If left empty, it is inferred from the target Actor's example start URLs, and if those are missing, guessed by AI from the target Actor's name, title, and description. This selects the site; if your prompt names a specific country, the generated URLs may use that country's regional domain of the same site (for example a UK-focused prompt can produce www.ebay.co.uk URLs even when websiteDomain is www.ebay.com).

The OpenAI API key is not part of the input. It is read from the OPENAI_API_KEY environment variable (Actor Settings → Environment variables in Apify Console, or a local .env file).

Output

The Actor stores one dataset item per generated URL. The OUTPUT record in the key-value store is a complete, ready-to-run input for the Actor set in targetActor (for example eBay Items Scraper) - you can pass it directly as the input of that Actor (for example in an Actor-to-Actor integration or Actor.call()). It is built from the target Actor's example input - filtered to only the fields that belong to that Actor's input schema, so leftover template placeholders are dropped - with startUrls set to the generated URLs.

Dataset items:

[
    { "url": "https://www.ebay.com/sch/i.html?_nkw=used+mirrorless+camera+body+excellent+condition" },
    { "url": "https://www.ebay.com/sch/i.html?_nkw=mirrorless+camera+body+preowned+excellent" },
    { "url": "https://www.ebay.com/sch/i.html?_nkw=used+mirrorless+camera+body+clean+condition" }
]

OUTPUT key-value store record:

{
    "startUrls": [
        { "url": "https://www.ebay.com/sch/i.html?_nkw=used+mirrorless+camera+body+excellent+condition" },
        { "url": "https://www.ebay.com/sch/i.html?_nkw=mirrorless+camera+body+preowned+excellent" },
        { "url": "https://www.ebay.com/sch/i.html?_nkw=used+mirrorless+camera+body+clean+condition" }
    ],
    "proxyConfig": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "VN"
    }
}

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data table

Field	Type	Description
url	string	Generated search URL

Supported websites

• eBay (any regional domain, e.g. www.ebay.com, www.ebay.co.uk, www.ebay.de): URLs use /sch/i.html?_nkw=... and may include filters such as buy-it-now, condition, or seller type.
• Amazon (any regional domain, e.g. www.amazon.com, www.amazon.de): URLs use /s?k=....
• Other websites: URLs are validated generically - they must be HTTPS URLs on the requested domain with search query parameters. Results depend on how well the model knows the website's search URL format.

Pricing / Cost estimation

How much does it cost to generate search URLs? Usually very little. Total cost is mainly driven by:

• OpenAI token usage (depends on prompt length)
• Apify compute units consumed by a short Actor run

For most simple prompts, this Actor completes quickly and stays within low usage ranges. If you are on Apify free tier, you can test with a small number of runs before scaling.

Tips or Advanced options

• Keep prompts specific (brand, condition, budget, category) to get better URLs.
• Add this Actor to a scheduled task to auto-generate fresh search links daily.
• Pass URLs to another Actor for listing extraction, price checks, or alerts.
• To target a regional marketplace, either set websiteDomain to that domain (e.g. www.ebay.co.uk) or mention the country in the prompt.

FAQ, disclaimers, and support

This Actor generates search URLs only. It does not guarantee listing availability, ranking order, or marketplace-specific policy outcomes. Always review and comply with the target website's terms and applicable laws before using generated URLs in automated workflows.

Known limitations:

• Requires a valid OpenAI API key in the OPENAI_API_KEY environment variable.
• Output quality depends on prompt clarity.
• Enforces the exact requested number of unique URLs and may fail if model output is invalid.

Need help or a custom version? Open an issue in the Actor repository or request a tailored implementation for your workflow.