# HTML to JSON Smart Parser (`muhammadafzal/html-to-json-smart-parser`) Actor

Convert any HTML page, pasted HTML, or HTML file into clean structured JSON. Works without an API key via smart heuristics; optional AI mode for schema-driven extraction. Returns titles, headings, links, images, tables, emails, and custom fields.

- **URL**: https://apify.com/muhammadafzal/html-to-json-smart-parser.md
- **Developed by:** [Muhammad Afzal](https://apify.com/muhammadafzal) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 result parseds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## HTML to JSON Smart Parser

**HTML to JSON Smart Parser** converts any HTML page, pasted HTML snippet, or HTML file into clean, structured JSON — no CSS selectors, no XPath, no API key required. Point it at a URL (or paste raw HTML) and get back a consistent JSON record with the page's title, headings, paragraphs, links, images, tables, emails, phones, meta tags, and JSON-LD structured data. Optional AI mode restructures the page into a schema you define with OpenAI.

Perfect for developers, data analysts, and AI/LLM pipelines that need HTML transformed into structured data without writing a single parser.

### What is the HTML to JSON Smart Parser?

A general-purpose HTML parser that turns messy, layout-specific markup into predictable JSON. Where traditional scrapers need custom selectors for every site, this actor uses a **smart heuristic engine** to find the data automatically, plus an **optional AI mode** that uses OpenAI to extract exactly the fields you ask for.

Two engines, one actor:

- **Heuristic mode (free, no API key):** rule-based extraction of title, description, headings, paragraphs, links, images, tables, emails, phones, meta tags, and JSON-LD. Fast, deterministic, works out of the box.
- **AI mode (bring your OpenAI key):** the LLM reads the page and returns JSON matching your `fieldsToExtract` list or an exact `outputSchema` — ideal for heterogenous or complex documents.

### Use cases for HTML to JSON conversion

- **RAG and LLM pipelines** — index HTML archives and web pages as clean, token-efficient JSON for embeddings and retrieval.
- **Data migration** — convert legacy HTML exports, reports, and file dumps into structured database-ready records.
- **Price and product monitoring** — pull title, price, availability, and SKU from e-commerce pages into a spreadsheet or API.
- **Lead generation** — extract business pages for emails, phones, and contact details.
- **Document automation** — turn HTML tables and articles into JSON for downstream tools and Zapier/Make flows.
- **Web scraping without code** — replace hand-written parsers with one configurable extraction tool.

### What data can I extract?

| Group | Fields returned |
|---|---|
| Identity | `url`, `sourceType`, `title`, `description`, `canonical`, `favicon` |
| Structure | `headings[]` (level + text), `paragraphs[]`, `links[]` (text, href, internal), `images[]`, `tables[]` (headers + rows) |
| Contact | `emails[]`, `phones[]` |
| Metadata | `metaTags{}`, `jsonLd[]` (parsed JSON-LD blocks) |
| Custom | `custom{}` — one value per name in `fieldsToExtract` (e.g. `price`, `author`, `availability`, `rating`) |
| AI (optional) | `extracted{}` — the LLM's schema-matched JSON object |
| Audit | `mode` (`heuristic`/`ai`), `error`, `scraped_at` |

Example heuristic output:

```json
{
  "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
  "title": "A Light in the Attic",
  "description": "A beautiful children's poetry book.",
  "headings": [{ "level": "h1", "text": "A Light in the Attic" }],
  "paragraphs": ["It's hard to imagine a world without A Light in the Attic."],
  "links": [{ "text": "Home", "href": "https://books.toscrape.com/", "internal": true }],
  "emails": ["sales@books.toscrape.com"],
  "jsonLd": [{ "@type": "Product", "name": "A Light in the Attic" }],
  "custom": { "title": "A Light in the Attic", "price": "34.95" },
  "mode": "heuristic",
  "scraped_at": "2026-08-03T12:00:00.000Z"
}
```

### How much will it cost?

This actor uses **pay-per-event pricing**. The heuristic engine needs no external API — the only cost is the actor itself.

| Event | Price |
|---|---|
| Actor start | $0.005 (one-time per run) |
| Result parsed | $0.003 per JSON record |

**Cost examples:**

- Parse 10 URLs (heuristic): ~**$0.035** — one start + 10 records.
- Parse 100 URLs (heuristic): ~**$0.30**.
- AI mode adds OpenAI token costs billed to your own API key — you stay in control.

### How to use the HTML to JSON Smart Parser

#### 1. By URL

1. In the **Page URLs** field, paste one or more URLs (each page becomes one JSON record).
2. Optionally add a comma-separated `fieldsToExtract` list such as `title, price, availability`.
3. Click **Start**.

#### 2. By pasted HTML or HTML file

1. Paste raw HTML into **HTML Content (Paste)**, or upload files / provide download URLs in **HTML File URLs**.
2. Run. No API key needed.

#### 3. By AI schema (advanced)

1. Set `mode` to `ai` (or just add your OpenAI key — auto mode picks it up).
2. Provide `fieldsToExtract` and/or an `outputSchema` such as `{"title":"string","price":"number","inStock":"boolean"}`.
3. Run — each page comes back as an `extracted` object matching your schema.

#### Input

```json
{
  "startUrls": [{ "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" }],
  "fieldsToExtract": "title, price, availability",
  "mode": "heuristic",
  "maxResults": 100
}
```

#### Output

Each source produces one JSON record in the actor's default dataset: identity fields, structural arrays (headings, paragraphs, links, images, tables), contact fields (emails, phones), metadata (metaTags, jsonLd), your `custom` fields, and — in AI mode — the `extracted` schema result. Export as JSON, CSV, XML, or Excel, or read it via the Apify API.

### Scrape data with dedicated actors

Need structured data from a specific platform? Try these focused extractors:

- [AI Web Scraper — Structured Data Extraction](https://apify.com/muhammadafzal/ai-web-extractor)
- [Google Maps Scraper](https://apify.com/muhammadafzal/google-maps-scraper)
- [Google News Scraper](https://apify.com/muhammadafzal/google-news-scraper)
- [Google Scholar Scraper](https://apify.com/muhammadafzal/google-scholar-scraper)
- [Google Shopping Scraper](https://apify.com/muhammadafzal/google-shopping-scraper)
- [Domain Scraper](https://apify.com/muhammadafzal/domain-scraper)
- [Craigslist Scraper](https://apify.com/muhammadafzal/craigslist-scraper)
- [Ebay Scraper](https://apify.com/muhammadafzal/ebay-scraper)

### FAQ

#### How many results can I parse in one run?

You can parse up to **10,000 sources** per run (each URL, pasted HTML block, or file = one record). Control cost with `maxResults`.

#### Can I integrate this with other apps?

Yes. Export results as JSON/CSV/Excel, schedule recurring runs, and connect to Zapier, Make, or your own system via the [Apify API](https://docs.apify.com/api/v2).

#### Can I use this through an MCP server?

Yes — the actor is MCP-optimized and works with the Apify MCP server in Claude, ChatGPT, Cursor, and other AI agents. Agents can discover it, pass URLs, and receive clean JSON.

#### Do I need an OpenAI API key?

Only for **AI mode**. The heuristic engine runs fully free with no external API key. If you provide a key, `mode: auto` upgrades extraction to AI automatically.

#### Is it legal to scrape this data?

Web scraping legality depends on the target site's terms of service and your jurisdiction. This tool only fetches and parses pages you point it at — you are responsible for respecting a site's ToS, robots.txt, and applicable laws. See [Apify's legal guidance](https://apify.com/legal) for details.

#### Your feedback

Found a site that parses poorly, or want a new extraction feature? Open an issue or leave a review — it directly shapes the roadmap.

# Actor input Schema

## `startUrls` (type: `array`):

Web page URLs to fetch and parse into JSON. Use this when the user provides specific pages (products, articles, listings, documents). Each URL is fetched and parsed; one structured JSON record is returned per page. Leave empty if pasting HTML or uploading files.

## `htmlContent` (type: `string`):

Raw HTML to convert to JSON, pasted directly into this field. Use this when the HTML is not publicly reachable (emails, local files, authenticated pages). Takes precedence over nothing — it is processed in addition to any URLs. Leave empty if using URLs or file URLs.

## `htmlFileUrls` (type: `array`):

URLs of HTML files to fetch and parse (e.g., exported HTML archives or static files). You can upload files in the Apify Console using the file upload button, or provide direct download URLs. Leave empty if using page URLs or pasted HTML.

## `fieldsToExtract` (type: `string`):

Comma-separated list of custom fields to pull out of the HTML, e.g. 'title, price, author, availability'. Each field is extracted heuristically using common selectors (itemprop, meta tags, data attributes, class names). Leave empty to auto-detect all important fields. AI mode uses these names as the extraction contract.

## `outputSchema` (type: `string`):

A JSON schema or sample object that the AI should return in AI mode, e.g. {"title": "string", "price": "number", "inStock": "boolean"}. Only used when mode is 'ai' or an OpenAI API key is provided. When set, the AI restructures the page into exactly this shape. Ignored in heuristic mode.

## `mode` (type: `string`):

How to extract data. 'auto' uses AI when an OpenAI API key is provided, otherwise falls back to heuristics. 'heuristic' always uses the fast, free rule-based engine. 'ai' always calls the LLM to restructure the page into your fieldsToExtract or outputSchema.

## `openaiApiKey` (type: `string`):

Your OpenAI API key, used only when mode is 'ai' or 'auto'. Without it, extraction is heuristic and free. Get a key at https://platform.openai.com/api-keys. The key is stored securely as a secret and never returned in results.

## `model` (type: `string`):

The OpenAI model used for AI-mode extraction. Defaults to gpt-4o-mini, the best cost/quality balance for HTML extraction. Use a stronger model (e.g. gpt-4o) for complex or ambiguous documents.

## `systemPrompt` (type: `string`):

Custom instructions that override the default AI extraction prompt. Use to add domain rules (e.g. 'extract prices in EUR only', 'normalize dates to YYYY-MM-DD'). Only used in AI mode. Leave empty to use the smart default prompt.

## `maxContentChars` (type: `integer`):

How much of the page text to send to the AI model in AI mode. Larger pages are truncated to this limit to control token cost. Heuristic mode ignores this.

## `maxResults` (type: `integer`):

Maximum number of JSON records to produce across all sources (cost control). One record is produced per URL, pasted HTML block, or HTML file.

## `proxyConfiguration` (type: `object`):

Proxy settings for fetching page and file URLs. Datacenter proxy is enabled by default; switch to residential for sites that block datacenter IPs. Not used for pasted HTML content.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
    }
  ],
  "htmlFileUrls": [],
  "fieldsToExtract": "title, price, description",
  "outputSchema": "",
  "mode": "auto",
  "model": "gpt-4o-mini",
  "systemPrompt": "",
  "maxContentChars": 20000,
  "maxResults": 100,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Link to the dataset containing all parsed JSON records.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
        }
    ],
    "htmlContent": "",
    "htmlFileUrls": [],
    "fieldsToExtract": "title, price, description",
    "outputSchema": "",
    "mode": "auto",
    "openaiApiKey": "",
    "model": "gpt-4o-mini",
    "systemPrompt": ""
};

// Run the Actor and wait for it to finish
const run = await client.actor("muhammadafzal/html-to-json-smart-parser").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" }],
    "htmlContent": "",
    "htmlFileUrls": [],
    "fieldsToExtract": "title, price, description",
    "outputSchema": "",
    "mode": "auto",
    "openaiApiKey": "",
    "model": "gpt-4o-mini",
    "systemPrompt": "",
}

# Run the Actor and wait for it to finish
run = client.actor("muhammadafzal/html-to-json-smart-parser").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
    }
  ],
  "htmlContent": "",
  "htmlFileUrls": [],
  "fieldsToExtract": "title, price, description",
  "outputSchema": "",
  "mode": "auto",
  "openaiApiKey": "",
  "model": "gpt-4o-mini",
  "systemPrompt": ""
}' |
apify call muhammadafzal/html-to-json-smart-parser --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,muhammadafzal/html-to-json-smart-parser"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/q47VXMdg0CyoPt3cx/builds/n4S7c4j7iCk2CXlgD/openapi.json