Web Page Extractor: HTTPS pages to Markdown, text or HTML

Extract readable page content from a batch of public HTTPS pages that you own or are authorized to collect. Get one dataset record per URL with the title, canonical URL, outgoing links, HTTP status, and the selected Markdown, plain text or HTML fragment representation. A typed failure record explains each page that could not be extracted.

This Actor fetches static HTML. It does not run JavaScript, sign in, follow redirects, solve CAPTCHA challenges or evade site blocks. It checks each site's robots.txt before requesting a page and refuses disallowed paths.

Input

Provide 1 to 20 HTTPS URLs and confirm that you own or are authorized to collect each page and that its current site terms permit automated access and use of the content. Do not submit personal profiles, contact lists, regulated personal data, authenticated pages or URLs containing credentials. The Actor does not accept cookies, custom headers or proxies.

Choose a content format and extraction scope:

• Markdown, text, or HTML for the selected page content. HTML is an untrusted fragment; it is not a sanitizer.
• Full body is the default and preserves multi-section documentation, service pages, code and tables while removing scripts, styles, templates and interactive form controls.
• Main content uses Readability's article-focused extraction. It can omit navigation and secondary sections.

HTML output is the extracted body or main-content fragment, not the original document DOM. Links are collected from the source HTML, up to 100 per page.

Examples

This input uses the Actor's actual prefill values for the Althor Labs public site. It also writes the schema defaults explicitly so it can be pasted as a complete input object:

{
  "urls": ["https://althorlabs.com/"],
  "authorizedUse": true,
  "outputFormat": "markdown",
  "extractionMode": "full_body"
}

The authorizedUse confirmation is for this owner-authorized example. Set it to true only when you are authorized to collect every submitted page and the site terms permit that use.

An HTTP URL fails before the Actor makes a page request:

{
  "schema_version": "1",
  "input_index": 0,
  "url": "http://example.com/[invalid URL redacted]",
  "status": "failure",
  "rendering_mode": "static_html",
  "extraction_mode": "full_body",
  "output_format": "markdown",
  "http_status": null,
  "title": null,
  "canonical_url": null,
  "links": [],
  "links_truncated": false,
  "content": null,
  "content_bytes": null,
  "failure": {
    "code": "invalid_url",
    "message": "Only absolute HTTPS URLs are supported."
  }
}

Limits and behavior

• Maximum 20 pages per run.
• Maximum 1 MiB response per page and 10 MiB total response bytes including robots files.
• Maximum 15 seconds per request and 120 seconds for the whole run.
• Maximum 8 MiB serialized result per page. Oversized pages return a typed failure instead of truncated content.
• HTTPS port 443 only. Private or special-use IPs, URL credentials and all redirects are refused.
• robots.txt errors fail closed. A published 404 or 410 has no rules.
• Requests are sequential and identify this Actor with its User-Agent.

Data and rights

Use this Actor only for pages you own or are authorized to collect. You are responsible for checking site terms, rate limits, and rights to retain or redistribute extracted content. The Actor stores output in your run dataset and does not publish it. Incidental personal information in a page is not removed automatically.

Output

Each dataset item represents one input URL. status is success or failure; failure.code contains a stable reason such as robots_disallowed, redirect_refused, response_too_large or request_timeout. rendering_mode is always static_html.

Successful result shape (synthetic)

{
  "schema_version": "1",
  "input_index": 0,
  "url": "https://example.com/",
  "status": "success",
  "rendering_mode": "static_html",
  "extraction_mode": "full_body",
  "output_format": "markdown",
  "http_status": 200,
  "title": "Example page",
  "canonical_url": "https://example.com/",
  "links": ["https://example.com/services/"],
  "links_truncated": false,
  "content": "Example content",
  "content_bytes": 15,
  "failure": null
}

Pricing

Each successfully extracted page costs $0.002. Typed failure rows remain in the dataset and are free. There is no Actor-start fee; standard Apify platform usage is included in the event price.

n8n workflow

The Web Page Extractor to Vector Store workflow JSON connects this Actor to Google Gemini embeddings and n8n's Simple Vector Store. See the template setup notes before importing. Configure Apify and Google Gemini credentials in n8n; page text is sent to Gemini for embedding. The Simple Vector Store is in-memory, shared within the n8n instance, and cleared on execution or restart, so use this workflow for demos rather than persistent storage. This workflow JSON is not listed in the n8n template library.