Website HTML Scraper 🌐

🌐 One request, one complete record

Export the returned information and check the actual result.

🔎 What is the Website HTML Scraper — and when should you use it?

The Website HTML Scraper turns static HTML, final URL, page title, response status and available response headers into clean, structured rows you can filter, export and feed straight into a spreadsheet, database or AI agent. No account, no portal API key, no browser automation to maintain.

Use it when you need: the static HTML from one public page for your own parsing or saved comparison.

Use something else when: you need extracted organic search results rather than a page document. Use Google Search Scraper for extracted organic search results rather than a page document. Ordinary public HTTP(S) HTML pages only. No JavaScript rendering, login, cookies, custom request headers, challenge bypass, proxy settings or automatic retries. HTML is capped at 2 MiB and unsupported responses fail without a page-result charge.

🤖 Use with AI agents

Already connected to the Apify MCP server? Just ask for this Actor by name: sian.agency/website-html-scraper

Agentic payments: Availability of x402 and Skyfire depends on this Actor's current Store eligibility. Check the published listing for supported payment methods. One completed response record is charged. Failed retrievals have no result-event charge; a separate run-start fee applies.

Otherwise copy this prompt into Claude, ChatGPT, Cursor or any MCP-enabled assistant:

I want Website HTML research using the Apify Actor `sian.agency/website-html-scraper`.

Use it when I need: the static HTML from one public page for your own parsing or saved comparison.

Don't use it when: you need extracted organic search results rather than a page document — use google-serp-scraper instead.

How to call it: set operation to fetch and provide url. Read the full completed record.

Start with this input:
{
  "operation": "fetch",
  "url": "https://example.com/"
}

Ask me which public page URL I want to retrieve, then run the Actor and summarise the results as a table.

Things you can ask your agent for:

• Fetch https://example.com/ and save the returned HTML as JSON.
• Show the final URL, available title and HTML byte length of a public page.
• Keep a dated HTML observation for my own comparison.

Machine-readable API, MCP config and OpenAPI definition for this Actor are published at apify.com/sian.agency/website-html-scraper.md.

📋 Overview

Website HTML Scraper saves one ordinary public page as an HTML record in your Apify dataset. Use the html field when you want the markup for your own parser or a saved page comparison. Check finalUrl and httpStatus before using it.

The page must work without JavaScript, login or cookies. Each successful run returns one document within the 2 MiB limit. Retrieval failures have no completed-page charge; a run-start fee still applies.

---

✨ Features

• 🌐 Complete record: One successful request creates one exportable observation.
• 🌐 Static HTML: Retain the returned HTML document with its final URL and response status.
• 🌐 Focused scope: Ordinary public HTTP(S) HTML pages only.
• 🌐 JSON export: Preserve nested fields and arrays without flattening away information.
• 🌐 Run report: Review results, billing and retry advice in the Output tab.
• 🌐 Input validation: Unsupported fields and invalid inputs stop before retrieval.
• 🌐 Scheduled observations: Use Apify scheduling to build your own saved history.

---

🎬 Quick Start

Choose fetch, supply the url and run the actor. Open the dataset when the request finishes. The Console example is a prefill; API callers supply their own work input.

curl -X POST 'https://api.apify.com/v2/acts/sian.agency~website-html-scraper/runs?token=YOUR_TOKEN' \
-H 'Content-Type: application/json' -d '{"operation": "fetch", "url": "https://example.com/"}'

🚀 Getting Started

Step 1: Enter the subject

Provide a specific url to retrieve.

Step 2: Run the actor

Review the live price and start one request.

Step 3: Inspect the result

Download JSON to preserve the complete nested record. Check status and any source values before using the data.

---

📥 Input Configuration

Field	Type	Required	Description
operation	select	No	fetch
url	string	Yes at runtime	Subject of this request; no API default

{
  "operation": "fetch",
  "url": "https://example.com/"
}

Ordinary public HTTP(S) HTML pages only. No JavaScript rendering, login, cookies, custom request headers, challenge bypass, proxy settings or automatic retries. HTML is capped at2MB and unsupported responses fail without a page-result charge.

---

📤 Output

Field	Type	Description
url	string	Requested URL
finalUrl	string	Final URL
pageTitle	string or null	Page title
httpStatus	integer	HTTP status
html	string	HTML source
bodyLength	integer	Body bytes
responseHeaders	object	Response headers
complete	boolean	Whether retrieval completed
_fetchedAt	string	Observation time
processedAt	string	UTC ISO observation timestamp
scrapingDate	string	Same UTC ISO timestamp as processedAt
status	string	success or error

{
  "status": "success",
  "complete": true,
  "url": "https://example.com/",
  "finalUrl": "https://example.com/",
  "httpStatus": 200,
  "bodyLength": 577,
  "html": "<!doctype html><html lang=en><head><meta charset=utf-8><link rel=icon href=data:,><meta name=viewport content=\"width=device-width,initial-scale=1\"><title>Example Domain</title><style>html{color-scheme:light dark;background:light-dark(#eee,#222)}body{font:16px/1.6 system-ui,sans-serif;max-width:26em;margin:auto;padding:25vh 2em 2em;text-align:center}</style></head><body><p>This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.</p><script src=/s.js></script></body></html>\n"
}

The example shows a recorded response. Optional values can be absent, and results can change between runs.

---

💼 Use Cases & Examples

1. Web developers

Fetch a public page after a deployment and inspect the html field for the expected server-delivered markup. If content appears only after JavaScript runs in a browser, this actor will not render it.

2. SEO analysts

Inspect the returned HTML for title or canonical tags with your own parser, then keep the finalUrl beside the observation. The actor returns the document; it does not calculate SEO scores or extract a complete metadata inventory.

3. Content researchers

Save the HTML for one public article page so you can examine its markup and extract the sections relevant to your research. Check that the text is present in the returned document before relying on it.

4. Quality analysts

Compare httpStatus, finalUrl and bodyLength with your expected public page response. A titleless page can still succeed and has pageTitle: null; non-HTML and unsuccessful responses are reported as failures.

5. Archiving teams

Keep a dated copy of a small public HTML page in an Apify dataset. This preserves the returned markup within the 2 MiB limit; it does not archive images, scripts, stylesheets or linked pages.

6. Automation teams

Pass a public URL from a Make, Zapier or N8N workflow and send the completed HTML record to your own parser or storage. Route a retrieval failure for review; the actor does not retry it or change rendering settings.

---

🔗 Integration Examples

JavaScript

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('sian.agency/website-html-scraper').call({"operation": "fetch", "url": "https://example.com/"});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient
client = ApifyClient('YOUR_TOKEN')
run = client.actor('sian.agency/website-html-scraper').call(run_input={'operation': 'fetch', 'url': 'https://example.com/'})
for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)

cURL

curl -X POST 'https://api.apify.com/v2/acts/sian.agency~website-html-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN' \
-H 'Content-Type: application/json' -d '{"operation": "fetch", "url": "https://example.com/"}'

Automation Workflows (N8N / Zapier / Make)

1. Start from a schedule or webhook.
2. Run the actor with the intended url.
3. Check the record's status and content.
4. Save the result or pass it to your next workflow step.

---

📊 Performance & Pricing

One completed response record is charged once. Failed retrievals have no result-event charge. A separate platform run-start fee applies, including invalid or stopped runs.

Non-HTML responses, access-verification pages and documents over the 2MB limit are unsupported. They have no page-result charge.

Paid Apify plans receive tier discounts. Free-plan users can test one record per run in the Console or MCP; direct API access requires a paying plan. Check the live price before starting.

---

❓ Frequently Asked Questions

Q: What is charged?

One completed request record. Failed retrievals have no result-event charge. The separate platform run-start fee applies.

Q: What is supported?

Ordinary public HTTP(S) HTML pages only. No JavaScript rendering, login, cookies, custom request headers, challenge bypass, proxy settings or automatic retries. HTML is capped at2MB and unsupported responses fail without a page-result charge.

Q: Can I use the actor through the API?

Yes, using a paying Apify plan. Free-plan users can test from the Console or MCP.

Q: Does an empty API input run the example?

No. The work input has a Console prefill and no API default.

Q: Which export keeps full detail?

JSON preserves nested data. CSV and Excel encode nested structures as text.

---

🐛 Troubleshooting

Request did not complete

• Check the input, then retry later. A failed retrieval has no result-event charge.

Request stopped before retrieval

• Check your maximum charge, available credits and free-run allowance.

A returned field is missing

• Optional fields depend on the response. Do not infer a value that was not returned.

Data differs between runs

• Keep the observation time and compare the actual returned records. Results may change.

---

⚖️ Is it legal to scrape data?

Use public information for a lawful purpose and respect applicable rights and source terms. Public availability does not itself grant a redistribution license. This actor does not grant permission to reuse third-party content.

Results may contain personal data. Establish an appropriate legal basis under GDPR or other applicable rules before processing it. Read Apify's overview of web scraping legality and obtain advice for your use case when needed.

Website HTML names belong to their respective owners. SIÁN Agency is independent and is not affiliated with or endorsed by those owners.

---

🤝 Support

Open an actor issue with the run link and input. Send questions to apify@sian-agency.online.

Built by SIÁN Agency | More tools