Gluecrawl Universal Scraper

Turn a public webpage into structured data without selectors, without paying for an idle Apify container. This Actor submits to Gluecrawl first, then retrieves completed results in a separate short run.

Before you start

You need a Gluecrawl account with API access and available credits. Paste your personal Gluecrawl API key into the secret Gluecrawl API key input. Apify encrypts secret inputs; the Actor never adds your key to the dataset or logs.

Gluecrawl—not Apify—runs the extraction and charges the applicable Gluecrawl credits. This Actor does not support login-protected targets, cookies, custom proxies, schedules, or multi-URL batches in v1.

Inputs

• Mode — use start to submit a new extraction, then fetch to retrieve its result. Both runs finish promptly; neither polls or waits for Gluecrawl.
• Target URL — required in start mode: one public webpage or listing page.
• Gluecrawl API key — your personal Gluecrawl key, stored as an Apify secret.
• Goal — required in start mode; describe the data you want, for example: Extract each travel book's title and price.
• Fields — in start mode, alternatively provide reproducible field definitions such as [{"name":"title","description":"Book title"}].
• Maximum pages — start mode only; defaults to 2; choose 1–100 within your Gluecrawl plan limit.
• Gluecrawl job ID — fetch mode only; copy this from the prior start run's receipt.

Use Goal or Fields, not both.

How to run it

1. Run in start mode. It writes a small receipt such as {"status":"submitted","jobId":"..."} to the default dataset and exits.
2. After Gluecrawl has finished, run the Actor in fetch mode with the same API key and that jobId.
3. Fetch writes every extracted item to its default dataset. If Gluecrawl is still processing, fetch succeeds quickly with a pending receipt; retry it later.

Output

Completed fetches write each extracted item unchanged to the default Apify dataset. Start and pending fetches write only their status receipt. Mapping errors, scraping errors, expired/invalid keys, insufficient credits, and zero-item completions fail clearly instead of producing a misleading empty dataset.

Example

Start with https://books.toscrape.com/catalogue/category/books/travel_2/index.html and the goal Extract each travel book's title and price. Then fetch with the returned job ID. The completed fetch dataset contains one row per book.

API use

Run this Actor through the Apify Console, API, CLI, or an Apify task. Read the default dataset of the completed fetch run as you would any other Actor output.