Website to Markdown — Content Extractor for LLMs & RAG

Point it at a domain and get every page back as clean Markdown. This Actor reads a site's robots.txt, walks its sitemap — nested indexes, gzip children and relative URLs included — or crawls the links when there is no sitemap, and turns each page into boilerplate-free Markdown and plain text: navigation, cookie bars, sidebars, share widgets, related-post strips and footers removed, headings and lists and tables and fenced code kept. Every row also carries the page's title, description, author, dates, language, canonical, headings, links and images, and — when you ask for it — an array of chunks ready to embed.

No API key. No browser. No JavaScript execution. $0.60 per 1,000 pages ($2.00 from October 20, 2026), proxy and chunking included. robots.txt and sitemaps are free, pages that had no content are free, and a run that extracts nothing costs nothing at all.

Try it in 30 seconds

This is the Store prefill; paste it and run:

{
  "startUrls": ["https://docs.apify.com/platform/actors"],
  "crawlLinks": true,
  "maxPages": 20,
  "maxDepth": 2,
  "output": "both",
  "chunkSize": 0
}

A URL with a path means that page plus the pages it links to. A bare domain — basecamp.com — means the whole site: robots.txt, sitemap, and up to maxPages of it. Want exactly the URLs you listed and nothing else? Set crawlLinks: false.

Output — one row per page; the fields you will use most are url, title, markdown, text, headings, links and chunks (full list under Output reference), plus one free site summary row per website. Anything that produced no content comes back as a free diagnostic row (ok: false, errorType, error) instead of a charge.

Price — $0.60 per 1,000 pages (from October 20, 2026: $2.00 per 1,000) on the FREE tier ($0.0006 a page, + $0.001 per site summary and $0.001 per run); robots.txt, sitemaps, disallowed URLs, pages with no content, chunking and the metadata are all free, and a run that extracted nothing costs nothing. Pay-per-event, no API key, no browser, limited permissions — works over the Apify MCP server (mcp.apify.com) and with agentic (x402) payments.

From code — client.actor("insight.solutions/website-content-extractor").call(run_input={"startUrls": ["https://docs.apify.com/platform/actors"], "maxPages": 20}) with apify-client, or POST https://api.apify.com/v2/acts/insight.solutions~website-content-extractor/run-sync-get-dataset-items.

What comes back

One page row per URL that had content. This one is real — read from docs.apify.com/actors on 2026-09-11, trimmed only where the text runs long:

{
  "ok": true,
  "rowType": "page",                       // "page" | "site" | "diagnostic"

  "input": "https://docs.apify.com/platform/actors",
  "website": "apify.com",
  "url": "https://docs.apify.com/actors",
  "finalUrl": "https://docs.apify.com/actors",
  "statusCode": 200,
  "contentType": "text/html; charset=utf-8",
  "responseTimeMs": 817,

  "title": "Actors | Platform | Apify Documentation",
  "description": "Learn how to develop, run and share serverless cloud programs. Create your own web scraping and automation tools and publish them on the Apify platform.",
  "siteName": null,
  "author": null,
  "publishedAt": null,
  "modifiedAt": null,
  "language": "en",
  "canonical": "https://docs.apify.com/actors",
  "image": "https://apify.com/og-image/docs-article?title=Actors",
  "keywords": [],
  "jsonLdTypes": [],

  "markdown": "# Actors\n\nActors are serverless cloud programs that take a structured JSON input, perform a task (web scraping, browser automation, data processing, and more), and optionally produce a structured output. Run them manually in [Apify Console](https://console.apify.com/actors), through the [API](https://docs.apify.com/api) or [CLI](https://docs.apify.com/cli)…\n\n## Actor lifecycle\n\n…",
  "text":     "Actors\n\nActors are serverless cloud programs that take a structured JSON input, perform a task (web scraping, browser automation, data processing, and more), and optionally produce a structured output…",
  "html": null,                            // filled only when includeHtml is on

  "wordCount": 295,
  "charCount": 1865,
  "tokenEstimate": 619,                    // characters ÷ 4
  "readingTimeMin": 1,

  "headings": [
    { "level": 1, "text": "Actors" },
    { "level": 2, "text": "Actor lifecycle" },
    { "level": 2, "text": "Actor components" },
    { "level": 2, "text": "Public and private Actors" }
  ],
  "links": [
    { "text": "Apify Console", "url": "https://console.apify.com/actors", "internal": true },
    { "text": "API", "url": "https://docs.apify.com/api", "internal": true }
  ],
  "internalLinkCount": 10,
  "externalLinkCount": 1,
  "images": [
    { "src": "https://docs.apify.com/assets/images/apify-actor-drawing-9e5b2c6bbe7a85acac72e5c7a13290a4.png",
      "alt": "Apify Actor diagram" }
  ],

  "chunks": null,                          // filled when chunkSize > 0
  "chunkCount": null,

  "extractionMethod": "article",           // how the content region was found
  "boilerplateRatio": 0.294,               // 29% of the page's text was chrome

  "depth": 0,
  "discoveredVia": "input",                // "input" | "sitemap" | "link"
  "sitemapLastmod": null,
  "scrapedAt": "2026-09-11T21:11:57.000Z"
}

Then one site row per website, written after its pages:

{
  "ok": true,
  "rowType": "site",
  "website": "apify.com",
  "url": "https://docs.apify.com",
  "pagesExtracted": 20,
  "discovery": "sitemap",
  "sitemapUrlCount": 20,
  "robotsBlocked": 0,
  "totalWords": 18450,
  "totalTokens": 26400,                    // budget an embedding run from this
  "avgBoilerplateRatio": 0.31,
  "failures": 3
}

And a free diagnostic row for anything that produced no content, saying which of the eight things happened.

Static HTML only — read this before you run it

This Actor fetches the HTML a server sends and reads that. It does not run JavaScript, and there is no browser anywhere in it. That is the whole reason it is fast and costs $0.0006 a page.

For the large majority of the web — documentation, blogs, news, government sites, e-commerce product pages, marketing sites, wikis, forums — the server sends the content and this works perfectly. For a single-page app that renders its content in the browser, the server sends a shell, and a shell is what you get. The Actor does not pretend otherwise: a 200 OK page with fewer than 20 words after extraction produces a free empty diagnostic row that says so in plain language, and it is not billed.

If a site you need is JavaScript-rendered, use a browser-based Actor for that site. This one is the cheap, fast tool for everything else.

How the extraction works

Four stages, and every one of them is visible in the output so a surprising row can be explained rather than guessed at.

1. Remove what is never content. <script>, <style>, <noscript>, <svg>, <iframe>, forms and form controls, comments; <nav>, <header>, <footer> and <aside>; anything with role="navigation" | "banner" | "contentinfo" | "complementary" | "search"; anything marked aria-hidden or display:none; and anything whose id or class contains a whole word from nav, menu, sidebar, footer, header, cookie, consent, banner, breadcrumb, share, social, comment(s), related, promo, advert, ad(s), popup, modal, newsletter, subscribe, toc.

That last rule has two guards, because it is the one that can go wrong. It is never applied to the element that was chosen as the content root — a site whose article lives in <div class="content-header"> keeps its article. And a matched element survives anyway when it is a link-light majority of its own container, because an element that is most of what contains it is the page rather than an ornament on it. That is what keeps a Hacker News thread's <div class="comment"> bodies while still deleting a blog's comment widget, a share bar and a table of contents.

2. Find the content root, in this order, each falling through when what it finds holds less than 150 characters:

extractionMethod	What was used
article	The page's <article> element. With several of them it must hold half the page's text, or they are cards on a listing page and this strategy is skipped
main	<main> or [role=main]
selector	A well-known content container: #content, #main, .post-content, .entry-content, .article-body, .article__body, .post-body, .markdown-body, .mw-parser-output, .prose, .content, .documentation, .doc-content
density	Every block scored by text length × (1 − link density), then up to the smallest ancestor holding 60% of the page's non-boilerplate text. This is what reads a table-laid-out forum thread
body	Last resort: the whole body, boilerplate still stripped

3. Convert to Markdown. h1–h6 → #…######; paragraphs; <br> → a line break; nested ul/ol with correct indentation and start honoured; pre → a fenced block with the language read from class="language-x", lang-x, highlight-x or brush: x; inline code → backticks; blockquote → >; table → a GitHub pipe table with | escaped in cells and colspan filled with empty cells so the columns line up; a → [text](absolute url); img → ![alt](absolute src); strong/b → **, em/i → *; hr → ---; dl/dt/dd, figure/figcaption and details/summary all kept. Entities are decoded, whitespace runs collapse, and there is never more than one blank line between blocks.

A <table> used for page layout — no <th> anywhere, cells holding <div>s and paragraphs — is rendered as the container it is being used as, not as a pipe table. A five-column row containing a whole comment is not a table to anybody except the HTML.

4. Read the metadata: og:title → <title> → first <h1>; meta description → og:description; author from meta[name=author], article:author, rel=author or JSON-LD author.name; dates from article:published_time / article:modified_time, JSON-LD datePublished / dateModified, or <time datetime>; plus <html lang>, rel=canonical, og:site_name, og:image, meta keywords and every JSON-LD @type.

Chunking for RAG

Set chunkSize between 500 and 10,000 and every row gains a chunks array:

"chunks": [
  {
    "index": 0,
    "text": "## 3.1. Using Python as a Calculator\n\nLet's try some simple Python commands…",
    "headingPath": "3. An Informal Introduction to Python > 3.1. Using Python as a Calculator",
    "charCount": 1492,
    "tokenEstimate": 373
  }
]

Four guarantees, all tested against real captured pages:

• Chunks break at block boundaries first. A paragraph is only cut open when it is longer than chunkSize by itself, and then at a sentence end.
• A fenced code block is never split, and neither is a table. They go into one chunk whole.
• headingPath is exact — it comes from the <h1>–<h6> elements themselves, not from counting # characters, so a chunk from halfway down a page knows which section it is in.
• chunkOverlap repeats the tail of the previous chunk on a word boundary, and never opens a code fence it does not close.

The one documented exception to the size bound: a single code block or table longer than chunkSize becomes a chunk of its own rather than being cut in half. Everything else stays within chunkSize + chunkOverlap.

Use cases

• Build a RAG index over a documentation site. One run, chunkSize: 1500, and every chunk arrives with its heading path already attached.
• Feed an LLM agent the current state of a site without a browser in the loop and without paying per token for navigation menus.
• Archive a blog or a handbook as Markdown — one file per row, finalUrl as the filename.
• Competitive and content research at a level a search snippet cannot reach: the actual words on 500 pages, with word counts and reading times.
• Migrate a CMS. Extract the old site to Markdown, import it into the new one.
• Train or evaluate on a corpus you control, with sitemapLastmod telling you what changed since the last crawl.

How it works, and why it keeps working

• Discovery ladder. Every domain is tried on https://, then https://www., then plain http://, using robots.txt as the probe — it is needed anyway, it is tiny, and any HTTP answer to it proves the host is there.
• Sitemaps as sites really publish them. Sitemap: lines from robots.txt, then /sitemap.xml, /sitemap_index.xml, /sitemap-index.xml, /sitemap/sitemap.xml. Indexes are followed three levels deep, *.xml.gz children are inflated whether the server pre-inflated them or not, and relative <loc> values are resolved against the sitemap's own URL. Collection stops as soon as there are enough URLs to fill maxPages.
• Link crawl as the fallback, and as an option. Links are read from the whole page — including the navigation this Actor strips out of the text, because that is where a site keeps the links to the rest of itself. Assets, feeds, API paths and PDFs are not followed.
• One polite crawler. Requests to one host are spaced 250–600 ms apart no matter how high maxConcurrency goes; each parallel worker holds its own proxy session; every request has a 20-second ceiling and the whole run has maxRunSecs.
• Bot walls are retried once, from a different exit IP, and then reported. Cloudflare's "Just a moment…" interstitial arrives as HTTP 200, so it is recognised from the page itself rather than from the status. A second refusal is about the URL, not the address, and hammering it would earn the run a blocklist entry.
• Redirects are deduplicated on the final URL, so two queued URLs that land on one page produce one row and one charge.

Input reference

Field	Type	Default	What it does
startUrls	array	—	Domains (the whole site) and/or URLs with a path (that page). Entries on the same host are one website: one crawl, one set of robots rules, one summary row. An email address or a bare IP is rejected with a free invalid-input diagnostic row
crawlLinks	boolean	true	Follow same-host links found on extracted pages
maxPages	integer	50	Pages per website. Max 5,000. This is the price of a run
maxDepth	integer	3	How far from the start the link crawl may travel. 0 = follow nothing
includeUrlPatterns	array	[]	Globs (*/docs/*) or /regex/i. A discovered URL must match one if any are given
excludeUrlPatterns	array	[]	Same spelling, and it wins. Never applied to a URL you typed
output	string	both	both, markdown or text
includeHtml	boolean	false	Also return the extracted region's HTML
keepLinks	boolean	true	[text](url) in the Markdown, or just the text
keepImages	boolean	true	 in the Markdown, or just the alt
chunkSize	integer	0	0 = off. 500–10,000 turns on the chunks array
chunkOverlap	integer	0	0–200 characters repeated between chunks
respectRobots	boolean	true	Obey the User-agent: * group
maxSitemapUrls	integer	5000	Ceiling on URLs collected from sitemaps
maxConcurrency	integer	8	Pages of one site in flight. 1–16
maxRunSecs	integer	240	Wall-clock budget. 30–3,600
proxyConfiguration	object	{"useApifyProxy": true}	Datacenter by default; switch to RESIDENTIAL for hostile sites

Output reference

rowType tells you what a row is. Every row carries every column, so a CSV export is one rectangle.

page rows — the response (url, finalUrl, statusCode, contentType, responseTimeMs), what the page says about itself (title, description, siteName, author, publishedAt, modifiedAt, language, canonical, image, keywords, jsonLdTypes), the content (markdown, text, html, wordCount, charCount, tokenEstimate, readingTimeMin, headings, links, internalLinkCount, externalLinkCount, images, chunks, chunkCount, extractionMethod, boilerplateRatio) and where the URL came from (depth, discoveredVia, sitemapLastmod).

site rows — pagesExtracted, discovery, sitemapUrlCount, robotsBlocked, totalWords, totalTokens, avgBoilerplateRatio, failures.

diagnostic rows — ok: false, never charged, one per URL that produced nothing:

errorType	What it means
unreachable	Nothing answered on https or http
blocked	Refused, or an anti-bot interstitial, from two different exit IPs
timeout	maxRunSecs ran out before this could be read
invalid-input	Not a website or page URL — an email address, an IP, a bad URL pattern
not-html	A PDF, an image, a JSON endpoint. Nothing to extract
http-error	A 4xx or 5xx. An error page is not content
too-large	Over 5 MB of markup
empty	200 OK HTML, but under 20 words after extraction — almost always a JavaScript-rendered shell

Five dataset views — Content, Metadata, Chunks, Site summaries and What could not be read — are set up in the Console so each shape exports as its own clean table.

Pricing

Event	When	FREE	BRONZE	SILVER	GOLD
Page extracted (primary)	One page of content in your dataset	$0.0006	$0.0006	$0.00048	$0.00036
Site summary	One website that produced at least one page	$0.001	$0.001	$0.001	$0.001
Run started	Once per run, after the first page is extracted	$0.001	$0.001	$0.001	$0.001

$0.60 per 1,000 pages, proxy included, chunking included.

Price change notice. From October 20, 2026 (UTC) the page event is $0.002 / $0.002 / $0.0016 / $0.0012 (FREE / BRONZE / SILVER / GOLD), with site and actor-start unchanged; the table above applies until then.

What you are never charged for

• robots.txt and every sitemap file, including a fifty-file gzip index tree.
• Any URL that produced no content — a dead host, a PDF, a 404, a bot wall, a document over 5 MB, or a JavaScript-only shell. Each gets a free diagnostic row saying which.
• URLs robots.txt disallows. They are counted on the site row and never fetched.
• Chunking, the text field, the html field and the metadata. They are part of the page, not extras.
• A run that extracted nothing. A run whose URLs were all usable but produced nothing — a domain that does not resolve, a site that refused us, a JavaScript-only shell — finishes SUCCEEDED with zero results, a status message that says so, and a free diagnostic row per URL telling you why; nothing is charged, start fee included. A run finishes FAILED only when there was nothing usable to attempt (no start URL, or none this Actor can fetch), or the Actor itself hit an error.

Bound a run's spend with ACTOR_MAX_TOTAL_CHARGE_USD. The Actor stops fetching while it can still bill for what it holds, so a budget is never exceeded and never leaves a paid summary standing over unpaid detail.

Limits, and the ones that might bite

• No JavaScript rendering. Said once above and repeated here because it is the limit that matters. Server-rendered HTML only.
• 5,000 pages per website per run, and 5,000 sitemap URLs collected. Raise maxRunSecs to match: at roughly two to three pages a second per host, 1,000 pages is around seven minutes.
• One <article> wins; a dozen do not. On a listing page where each card is an <article>, the Actor falls through to <main> and returns the whole list. That is usually what you want from a listing page, and extractionMethod tells you it happened.
• Boilerplate removal is a heuristic, not a parser. It is tested against eleven real captured pages, and every one of them has its result written down in TEST_RESULTS.md. When a row looks wrong, extractionMethod and boilerplateRatio are the two columns that explain it: body plus a ratio near zero means no strategy found a content region.
• tokenEstimate is characters ÷ 4. A rule of thumb for budgeting, not a tokenizer. Your model's count will differ, usually by under 15% on English prose and by more on code.
• A table that is genuinely tabular but has no <th> and holds block content is rendered as prose rather than as a pipe table. That is the safe direction to be wrong in.
• A page behind a login is not fetched. There is no authentication in this Actor and there is not going to be one.
• The upstream format may change. Sites are rebuilt, sitemaps move, and a site that published a clean sitemap.xml last month can serve an index of gzip children this month. When that happens the discovery path changes rather than breaking — the discovery and sitemapUrlCount columns on the site row show which path was taken.

Use it from an AI agent, or from code

One JSON object in, one flat array out — the shape agent runtimes want. The Actor runs with limited permissions, uses pay-per-event pricing and never enters Standby, so it works over the Apify MCP server and with x402 agentic payments. The Integrations tab pushes results to Slack, a webhook, Zapier, Make, Google Sheets, Snowflake or BigQuery.

curl -X POST "https://api.apify.com/v2/acts/insight.solutions~website-content-extractor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":["https://docs.apify.com/platform/actors"],"maxPages":20,"chunkSize":1500}'

# pip install apify-client
from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("insight.solutions/website-content-extractor").call(run_input={
    "startUrls": ["docs.apify.com"],
    "maxPages": 200,
    "chunkSize": 1500,
    "chunkOverlap": 150,
    "excludeUrlPatterns": ["*/changelog/*"],
})

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["rowType"] == "page":
        for chunk in row["chunks"] or []:
            embed(chunk["text"], metadata={
                "url": row["finalUrl"],
                "title": row["title"],
                "section": chunk["headingPath"],
            })
    elif row["rowType"] == "site":
        print("crawled", row["pagesExtracted"], "pages,", row["totalTokens"], "tokens")

FAQ

How does it find the pages? robots.txt first, for the Sitemap: lines and the crawl rules. Then those sitemaps, or /sitemap.xml and three other usual locations. Sitemap indexes are followed to their children, gzip children are inflated, and relative <loc> values are resolved. If none of that yields anything, it crawls from the home page. When you give a URL with a path, that page is read first and its links are followed.

Does it render JavaScript? No. It reads the HTML the server sends. A page that renders its content in the browser comes back as whatever the shell contains, and when that is under 20 words the Actor files a free empty diagnostic row saying so rather than billing you for an empty row.

How good is the boilerplate removal, really? Good enough that the Apify docs page comes back as its H1 and four sections with no sidebar and no footer, the Python tutorial keeps all 42 code samples with their >>> prompts, and MDN's <table> reference keeps both of its data tables and all 24 code blocks with the right languages. Every one of those is an assertion in the test suite, run against the captured bytes. The full list, including what is not perfect, is in TEST_RESULTS.md.

What is extractionMethod for? Telling you which of the five strategies found the content, so a surprising row can be explained. article and main are the confident ones; density means the page had no semantic structure to work with; body means nothing was found and you are looking at the whole page minus its chrome.

Can I get just the text, without Markdown syntax? Yes — output: "text". Both formats come from one walk of the page, so they can never disagree about what it said; choosing one only leaves the other column null.

Do the chunks overlap correctly across a code block? Yes. An overlap that would start inside a fenced block is trimmed past the fence, so a chunk never opens a fence it does not close. That is tested against the Python, MDN and GitHub captures at four chunk sizes.

Why did a page I know exists not come back? Five possibilities, in order of likelihood: it is not in the sitemap and nothing links to it within maxDepth; maxPages was reached first; an excludeUrlPatterns entry matched it; robots.txt disallows it; or it produced no content. The site row's sitemapUrlCount, pagesExtracted and robotsBlocked distinguish the first four, and there is a free diagnostic row for the fifth.

Does it follow links to other domains? No. A crawl stays on the host it started from, with www. folded away. Links to other domains are still reported in the links column with internal: false.

Can I extract 5,000 pages? Yes, that is the per-website ceiling. Raise maxRunSecs to match and bound the spend with ACTOR_MAX_TOTAL_CHARGE_USD. At $0.0006 per page the whole site is $3.

What happens if one site in a list fails? The others still run. The failed one produces a free diagnostic row and the run finishes SUCCEEDED. If every site fails, the run still finishes SUCCEEDED, with zero pages, a status message that says so ("0 pages extracted. N diagnostic row(s) explain why …") and nothing billed at all. Only an input with nothing fetchable in it finishes FAILED.

Is the data fresh? Live. Every run reads the site at that moment; nothing is cached, and there is no index behind this.

Legal and data-protection notes

• Public pages only. Every URL fetched is one a browser would load without signing in. The Actor never logs in, never accepts cookies or session tokens, never takes an API key belonging to anyone else, and never touches anything behind authentication.
• It obeys the site's own instructions by default. robots.txt is read before anything else and its Disallow rules are respected unless you deliberately turn that off for a site you control.
• It is a polite crawler. Requests to one host are spaced 250–600 ms apart, each has a 20-second ceiling, and the whole run has a budget. A refusal is retried once from a different address and then left alone.
• Content belongs to whoever published it. This Actor reformats a public web page; it does not license it. Copyright, terms of use and any database rights are unchanged by passing the page through a Markdown converter, and what you may do with the output is between you and the publisher.
• Comment threads and forums can contain personal data. The extractor keeps the text a page shows a reader, which on a forum includes user names and what they wrote. If you crawl that kind of site, that is personal data and your own obligations apply to it.
• Not affiliated with any website read or named in these docs. All trademarks belong to their respective owners.

Our other Actors

Every Insight Solutions Actor is pay-per-result with no browser, no login and no API key, and every one of them returns free diagnostic rows instead of billing for failures. Prices are per 1,000 results.

Video, audio & social

• YouTube Transcript API — captions as timed segments, text, SRT or VTT, with language fallback and translation.
• YouTube Comments API — comments and replies with likes, pinned and hearted flags, newest or top sort.
• YouTube Channel API — a channel's videos, Shorts and live streams, plus YouTube search.
• Podcast Search, Episodes & Charts API — Apple Podcasts search, charts and full episode feeds.
• Bluesky Scraper — profiles, posts, followers and follows from the public AT Protocol API.
• Telegram Channel Scraper — posts, views and channel stats from public Telegram channels.
• Substack Scraper — posts with full free text, comments and publication profiles.
• Hacker News API — stories, comments, users, front page and a structured "Who is hiring?" parser from the official HN APIs.
• GitHub Repos, Trending & Releases API — repository facts, trending lists, releases and search, no token needed for small runs.
• Discourse Forum API — topics, posts and categories from any Discourse community via its own JSON endpoints, usernames only.

News, documents & the web

• Google News Search, Topics & Real Article URLs — news search and topic feeds with the publisher's real URL decoded.
• WordPress Posts & Pages API — posts and pages from any WordPress site as clean text or Markdown, with categories, tags and images.
• Website Metadata API — any URL's meta tags, Open Graph, Twitter cards, hreflang, feeds, JSON-LD and microdata, with normalised Product, Article, Event and FAQ entities.
• Subdomain Finder API — every subdomain of a domain from certificate-transparency logs, resolved over DNS, with an optional live HTTP check.
• Internet Archive API — archive.org search, item metadata, files and reviews.
• Wayback Machine Toolkit — archived URL inventories, snapshots and text diffs between dates.
• Website Technology Detector — the tech stack behind any site, with the evidence for each detection.
• Domain Intelligence API — DNS, RDAP registration, TLS certificate and HTTP facts in one row per domain.
• SEO Page Audit — sitemap crawl with on-page checks, structured data and broken-link reports.
• Keyword Suggestions API — Google, YouTube, Bing, Amazon and eBay autocomplete with alphabet and question expansions.
• Website Contact Extractor — emails, phone numbers and social profiles from any list of websites.
• Web Search Results API — Bing and DuckDuckGo organic results with snippets, no key, no browser.
• Company Enrichment API — a domain in, a company profile out: firmographics, contacts, tech stack, DNS and hiring signal.
• Company Dossier API — one company in, twelve sections out: profile, tech, contacts, DNS, open roles, news, SEC filings, federal awards, recalls, YC batch and apps.
• Press Releases API — GlobeNewswire and PR Newswire releases plus any newsroom feed, by keyword, company, ticker or subject.
• Federal Register API — rules, proposed rules, notices and the Public Inspection desk with dockets, comment deadlines and CFR references.
• Academic Papers Search API — OpenAlex, Crossref, arXiv and PubMed in one row per paper: abstract, citations, open-access PDF, authors and venue.
• RSS & Atom Feed Monitor — any RSS, Atom or JSON feed (or an OPML file) in, only the new items out, with keyword filters and a webhook.
• Website Change Monitor — watch any pages, diff the text between runs, get change rows with added/removed lines, keyword alerts and a webhook.
• Wikipedia & Wikidata API — article text, search, daily pageviews and Wikidata entity facts, any language edition.

Business, finance & jobs

• Congress & Insider Trades API — STOCK Act periodic transaction reports and SEC Form 4 insider trades in one schema.
• Federal Contracts, Grants & Lobbying API — SAM.gov opportunities, USAspending awards, Grants.gov notices and Senate lobbying filings in one schema.
• SEC EDGAR API — filings, XBRL financials and full-text search by ticker or CIK.
• Clinical Trials & FDA API — ClinicalTrials.gov studies plus openFDA recalls, labels, approvals, 510(k)s and adverse-event reports.
• Product & Vehicle Recalls API — CPSC, NHTSA, FDA and USDA recalls, vehicle complaints and ratings, plus a VIN decoder.
• Y Combinator Companies, Batches & Founders — the YC directory with founders and social links, filterable by batch, industry and hiring status.
• Career Site Jobs API — jobs straight from Greenhouse, Lever, Ashby, Workable and 10+ other ATS career sites.
• Workday Jobs API — every posting on any Workday career site, with facets, search, posted dates and full descriptions.
• New Job Postings Monitor — new, closed and changed postings on the career sites you watch.
• Hiring Signals API — Open Roles & Hiring Surge by Company — one row per company per run: open roles, what opened and closed, department and seniority breakdowns, and a hiring-surge flag.
• Remote Jobs API — RemoteOK, Remotive, We Work Remotely, Himalayas, Jobicy and more in one schema, deduplicated.
• Shopify Products API — any Shopify store's catalogue, variants, prices and stock signals.
• WooCommerce Products API — any WooCommerce store's catalogue, prices, variants and stock through its public Store API.
• Shopify App Store API — Shopify App Store listings by category, collection or developer: pricing plans, ratings and merchant reviews.
• Luma Events API — upcoming Luma events by city, category or calendar: dates, venue, price, capacity and organiser calendar, no host names.
• Shopify Store Monitor — price drops, sales, restocks, sell-outs and new products on any Shopify store, one row per change.
• Public Tenders API — EU TED, UK Find a Tender and Contracts Finder notices by keyword, CPV code, country, stage and deadline.
• Nonprofit & IRS 990 Lookup API — search US nonprofits and get EIN, NTEE code and multi-year Form 990 financials.
• OpenStreetMap Places API — businesses and points of interest by category and area from OpenStreetMap: name, address, coordinates, website, phone, opening hours.

Apps & games

• App Store & Google Play Reviews API — reviews from both stores with ratings, versions and developer replies.
• App Store Top Charts & App Search API — Apple top charts by country and genre, plus app search and details.
• App Store Keyword Rank Tracker — where any app ranks for any keyword on the App Store and Google Play, with rank changes and ASO suggestions.
• Steam Reviews API — Steam reviews with playtime, helpfulness and game details.
• Steam Game Data API — prices, tags, review scores, live player counts and top charts.
• Sports Scores & Schedules API — scores, schedules, standings, teams and headlines for the NFL, NBA, MLB, NHL, soccer and more, no key.
• App Store Monitor — App Store and Google Play listings watched between runs: new versions with release notes, price, rating and listing changes, one row per change.