# Web Structured Data Extractor (Claude, JSON Schema) (`gochujang/web-structured-extractor`) Actor

Pass a URL + JSON schema (or natural-language goal). Claude reads the page and returns a strict JSON object matching your schema. Product / news / hotel / real-estate / job-board extraction. BYO Anthropic API key. $0.01 per page.

- **URL**: https://apify.com/gochujang/web-structured-extractor.md
- **Developed by:** [Hojun Lee](https://apify.com/gochujang) (community)
- **Categories:** Developer tools, Automation, Other
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Web Structured Data Extractor (Claude)

> Pass a URL + JSON schema (or natural-language goal). Claude reads the page and returns a strict JSON object matching your schema. **Product / news / hotel / real-estate / job-board extraction.** BYO Anthropic API key. **$0.01 per page.**

---

### Why this exists

You want to scrape structured fields out of arbitrary web pages — price, SKU, rating, hours, contact info, reviews. Building a per-site CSS-selector scraper is brittle (sites change every week). General-purpose LLM extraction is robust but requires prompting + parsing.

This actor wraps the whole pipeline:
1. URL → clean Markdown via trafilatura
2. Markdown + your schema/goal → Claude
3. Claude returns strict JSON → we parse and validate

Same idea as DiffBot's Article API ($299/mo) or Browse AI's extraction ($99/mo) — but with your own Claude API key and no monthly subscription.

---

### What you get

```json
{
  "url": "https://...",
  "model": "claude-opus-4-7",
  "goal": "Extract product info",
  "schema_used": true,
  "extracted_data": {
    "name": "Nintendo Switch 2",
    "price_usd": 499.99,
    "in_stock": true,
    "rating": 4.8,
    "reviews_count": 1247
  },
  "raw_output": "...",
  "input_chars": 5230,
  "usage": {"input_tokens": 1450, "output_tokens": 80}
}
````

***

### Two ways to specify what to extract

#### Option 1: JSON Schema (recommended)

```json
{
  "url": "https://www.amazon.com/dp/B07VPHN6CR",
  "schema": {
    "type": "object",
    "properties": {
      "name":         {"type": "string"},
      "price_usd":    {"type": "number"},
      "in_stock":     {"type": "boolean"},
      "rating":       {"type": "number"},
      "reviews_count":{"type": "integer"}
    }
  },
  "anthropicApiKey": "sk-ant-..."
}
```

#### Option 2: Natural language goal

```json
{
  "url": "https://...",
  "goal": "Extract product name, USD price, in-stock status, average rating",
  "anthropicApiKey": "sk-ant-..."
}
```

You can combine both — schema sets the shape, goal sets emphasis.

***

### Use cases

1. **E-commerce competitor monitoring** — Track price + availability across competitors
2. **Real estate listing aggregation** — Extract beds/baths/price/sqft from Zillow, Redfin, Realtor
3. **Job board scraping** — Title, company, salary, location, remote-flag from LinkedIn, Indeed
4. **News article fact extraction** — Get the same 5 fields from any news source
5. **Hotel / travel research** — Name, rating, price/night, amenities from any booking site

***

### Pricing

**Pay-Per-Event**: `$0.01 per page` (Apify-side).

Anthropic tokens charged separately. Typical:

| Page complexity | Input tokens | Anthropic | Total |
|---|---|---|---|
| Simple product page | ~1500 | ~$0.008 | $0.018 |
| Long article | ~4000 | ~$0.020 | $0.030 |
| Big e-commerce listing | ~8000 | ~$0.040 | $0.050 |

Use Haiku for batch / cheap extraction (~10x cheaper).

***

### Setting your Anthropic API key

See [Article Summarizer README](https://apify.com/gochujang/article-summarizer) for the full BYO API key guide. Short version:

1. Get key at console.anthropic.com
2. Paste in `anthropicApiKey` input (Apify saves it encrypted)
3. Or save as Apify Account-level Secret and reference as `@MY_KEY`

***

### Tips for reliable extraction

- **Better prompts → better results.** A `goal` string like "extract product name, USD price, in-stock boolean, rating 1-5" outperforms "extract product info".
- **Constrain types in the schema.** `"type": "number"` is stricter than `"type": ["string","number","null"]`.
- **Test with Haiku first.** Haiku 4.5 is fast and 10x cheaper for prototyping. Switch to Opus 4.7 when you need accuracy.

***

### Related actors (same author)

- [Article Summarizer](https://apify.com/gochujang/article-summarizer) — TL;DR instead of structured
- [Web Page → Markdown Converter](https://apify.com/gochujang/web-to-markdown) — Just the body, no LLM
- [HTML Metadata Extractor](https://apify.com/gochujang/html-metadata-extractor) — Cheaper for OG / Twitter / JSON-LD
- [JSON Schema Generator](https://apify.com/gochujang/json-schema-generator) — Bootstrap a schema from samples

***

### Feedback

A short review helps engineers find it: [Leave a review on Apify Store](https://apify.com/gochujang/web-structured-extractor#reviews)

# Actor input Schema

## `url` (type: `string`):

URL of the page to extract from.

## `schema` (type: `object`):

If provided, Claude will fill an object matching this schema. Use 'goal' instead/alongside for free-form requests.

## `goal` (type: `string`):

Natural-language description (e.g. 'price, in stock, top review'). Used if 'schema' is not provided.

## `model` (type: `string`):

Claude model.

## `anthropicApiKey` (type: `string`):

Get one at console.anthropic.com.

## `userAgent` (type: `string`):

Custom UA for page fetch.

## Actor input object example

```json
{
  "url": "https://www.apify.com/store/actors/gochujang/smart-money-tracker",
  "schema": {},
  "goal": "",
  "model": "claude-opus-4-7",
  "userAgent": ""
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://www.apify.com/store/actors/gochujang/smart-money-tracker"
};

// Run the Actor and wait for it to finish
const run = await client.actor("gochujang/web-structured-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "url": "https://www.apify.com/store/actors/gochujang/smart-money-tracker" }

# Run the Actor and wait for it to finish
run = client.actor("gochujang/web-structured-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://www.apify.com/store/actors/gochujang/smart-money-tracker"
}' |
apify call gochujang/web-structured-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=gochujang/web-structured-extractor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Web Structured Data Extractor (Claude, JSON Schema)",
        "description": "Pass a URL + JSON schema (or natural-language goal). Claude reads the page and returns a strict JSON object matching your schema. Product / news / hotel / real-estate / job-board extraction. BYO Anthropic API key. $0.01 per page.",
        "version": "0.1",
        "x-build-id": "HAb66fvjWmWDQg4dE"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/gochujang~web-structured-extractor/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-gochujang-web-structured-extractor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/gochujang~web-structured-extractor/runs": {
            "post": {
                "operationId": "runs-sync-gochujang-web-structured-extractor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/gochujang~web-structured-extractor/run-sync": {
            "post": {
                "operationId": "run-sync-gochujang-web-structured-extractor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "url",
                    "anthropicApiKey"
                ],
                "properties": {
                    "url": {
                        "title": "Page URL",
                        "type": "string",
                        "description": "URL of the page to extract from."
                    },
                    "schema": {
                        "title": "Target JSON schema (optional)",
                        "type": "object",
                        "description": "If provided, Claude will fill an object matching this schema. Use 'goal' instead/alongside for free-form requests.",
                        "default": {}
                    },
                    "goal": {
                        "title": "Extraction goal (optional)",
                        "type": "string",
                        "description": "Natural-language description (e.g. 'price, in stock, top review'). Used if 'schema' is not provided.",
                        "default": ""
                    },
                    "model": {
                        "title": "Claude model",
                        "enum": [
                            "claude-opus-4-7",
                            "claude-sonnet-4-6",
                            "claude-haiku-4-5"
                        ],
                        "type": "string",
                        "description": "Claude model.",
                        "default": "claude-opus-4-7"
                    },
                    "anthropicApiKey": {
                        "title": "Anthropic API Key",
                        "type": "string",
                        "description": "Get one at console.anthropic.com."
                    },
                    "userAgent": {
                        "title": "User-Agent",
                        "type": "string",
                        "description": "Custom UA for page fetch.",
                        "default": ""
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```