Website Metadata API — Schema.org, Open Graph & JSON-LD
Pricing
from $0.60 / 1,000 pages
Website Metadata API — Schema.org, Open Graph & JSON-LD
Read any list of URLs over plain HTTP and get one row per page: title, meta tags, Open Graph, Twitter cards, canonical, hreflang, feeds, icons, JSON-LD and microdata with person names removed, plus normalised Article, Product, Organization, Event, JobPosting and FAQ data. No browser, no API key.
Pricing
from $0.60 / 1,000 pages
Rating
0.0
(0)
Developer
Insight Solutions
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
9 days ago
Last modified
Categories
Share
Website Metadata API — Open Graph, JSON-LD & Schema.org
Give it a list of URLs, get back what each page says about itself. One row per page: the title and every meta tag, Open Graph and Twitter card data, canonical and hreflang links, RSS/Atom feeds, icons, every JSON-LD block and microdata item — and, on top of that, the things people actually want out of structured data in one predictable shape: the article's dates, the product's price and rating, the organisation's profiles, the event's venue, the job's salary, the FAQ, the breadcrumbs.
Plain HTTP, no browser, no API key. Person names are removed from everything structured before it is written — see Privacy. $1.00 per 1,000 pages, proxy included; a page that could not be read is a free diagnostic row, and a run that returns nothing costs nothing at all.
Try it in 30 seconds
Input — this is the Store prefill; paste it and run:
{"urls": ["https://apify.com/store","https://github.com/apify/crawlee","https://en.wikipedia.org/wiki/Web_scraping","https://www.theguardian.com/international","https://docs.python.org/3/"]}
A bare domain — apify.com — works too and means its home page. Or give a website and its sitemap supplies the URLs.
Output — one row per URL. The fields you will use most are title, description, canonical, openGraph, schemaTypes and entities (full list under Output reference). Anything that could not be read comes back as a free diagnostic row (ok: false, errorType, error) instead of a charge.
Price — $1.00 per 1,000 pages on the FREE tier (+ $0.001 per run, charged only after the first page); Scale $0.80 and Business $0.60 per 1,000. Blocked pages, 404s, PDFs, URLs robots.txt disallows and anything that timed out are free. Pay-per-event, limited permissions — works over the Apify MCP server (mcp.apify.com) and with agentic (x402) payments.
From code — client.actor("insight.solutions/website-metadata-api").call(run_input={…}) with apify-client, or POST https://api.apify.com/v2/acts/insight.solutions~website-metadata-api/run-sync-get-dataset-items.
What comes back
A real row (blog.apify.com, shortened where marked):
{"ok": true,"rowType": "page", // "page" | "diagnostic""input": "https://blog.apify.com/introducing-apify-ai/","url": "https://blog.apify.com/introducing-apify-ai/","finalUrl": "https://blog.apify.com/introducing-apify-ai/","statusCode": 200,"redirectCount": 0,"contentType": "text/html; charset=utf-8","truncated": false, // true when the page passed 2 MB and only 2 MB were read"title": "Meet Apify AI: describe a task, get the data","description": "Describe what you want in plain English, and Apify AI finds the right Actor, …","lang": "en","charset": "utf-8","canonical": "https://blog.apify.com/introducing-apify-ai/","robotsMeta": "index,follow","generator": "Ghost 6.64","author": null, // <meta name="author"> only when it is an organisation"publishedAt": "2026-07-21T11:00:00.000Z","modifiedAt": "2026-07-22T07:12:11.000Z","wordCount": 1595,"headings": { "h1": ["Meet Apify AI: describe a task, get the data"], "h2": ["What Apify AI does", "…"] },"openGraph": {"type": "article", "siteName": "Apify Blog", "title": "Meet Apify AI: describe a task, get the data","images": [{ "url": "https://storage.ghost.io/…/Apify-AI-launch.png", "width": 1200, "height": 676, "secureUrl": null, "type": null, "alt": null }],"article": { "publishedTime": "2026-07-21T11:00:00.000Z", "tags": ["AI", "Apify updates", "Thought leadership"], "authors": [] }// … locale, url, description, videos, audio, product, other},"twitter": { "card": "summary_large_image", "site": "@apify", "label1": "Written by", "data1": null, "…": "…" },"metaTags": [{ "name": "robots", "property": null, "httpEquiv": null, "content": "index,follow" }, "… 29 in all"],"links": {"canonical": "https://blog.apify.com/introducing-apify-ai/","alternates": [], // [{ hreflang, href }] — stripe.com has 89"feeds": [{ "type": "application/rss+xml", "href": "https://blog.apify.com/rss/", "title": "RSS feed" }],"icons": [{ "rel": "icon", "href": "https://storage.ghost.io/…/apify-favicon.svg", "sizes": null, "type": "image/png" }],"manifest": null, "amphtml": null, "prev": null, "next": null, "me": [], "license": null},"schemaTypes": ["Article", "Organization", "ImageObject", "Person"],"jsonLd": [{"@type": "Article","headline": "Meet Apify AI: describe a task, get the data","publisher": { "@type": "Organization", "name": "Apify Blog", "…": "…" },"author": { "@type": "Person", "redacted": true }, // the byline is gone"contributor": [{ "@type": "Person", "redacted": true }],"datePublished": "2026-07-21T11:00:00.000Z", "…": "…"}],"jsonLdInvalid": 0,"microdata": [],"redactions": 2,"entities": {"article": {"source": "json-ld", "type": "Article", "headline": "Meet Apify AI: describe a task, get the data","datePublished": "2026-07-21T11:00:00.000Z", "dateModified": "2026-07-22T07:12:11.000Z","publisher": "Apify Blog", "authorType": "Person", "tags": ["AI", "Apify updates", "Thought leadership"],"section": null, "image": "https://storage.ghost.io/…/Apify-AI-launch.png", "wordCount": null, "isAccessibleForFree": null},"organization": { "source": "json-ld", "name": "Apify Blog", "url": "https://blog.apify.com/", "…": "…" },"product": null, "localBusiness": null, "event": null, "jobPosting": null, "faq": null,"breadcrumbs": null, "video": null, "recipe": null, "website": null, "softwareApplication": null},"error": null, "errorType": null,"scrapedAt": "2026-10-07T12:00:00.000Z","source": "blog.apify.com","sourceUrl": "https://blog.apify.com/introducing-apify-ai/"}
And what a product page's entity looks like — Steam's store page, which marks its game up in microdata rather than JSON-LD:
{"source": "microdata", "type": "Product", "name": "Stardew Valley","offers": [{ "type": "Offer", "price": 10.99, "priceCurrency": "GBP", "availability": null, "…": "…" }],"aggregateRating": { "value": 10, "count": 393881, "best": 10, "worst": 1 }}
Use cases
- Link previews — title, description,
og:imagewith its dimensions, site name and icon, for any URL a user pastes. - Structured-data audits — which schema.org types each page declares (
schemaTypes), which blocks do not even parse (jsonLdInvalid), and whether the Product has an offer and a rating. - Price and rating extraction —
entities.product.offers[]andaggregateRating, whichever markup the shop used. - Article dates and sections —
publishedAt,modifiedAt,entities.article.sectionandtagsfor a list of news or blog URLs. - International SEO — the full hreflang table per page, and whether the canonical points where it should.
- Feed and icon discovery — every RSS/Atom/JSON feed a site advertises, every icon size, the web-app manifest.
- Event, job and local-business data — venue, dates and ticket price; title, hiring organisation, salary range and remote flag; address, geo and opening hours.
- Feeding agents — one flat JSON row per URL with a fixed shape, so a model does not have to read raw HTML to know what a page is.
How it works
One GET per URL (never HEAD), with an ordinary browser accept header, through Apify's datacenter proxy by default.
- Reading stops at 2 MB. Everything this Actor reads — the head, Open Graph, almost all JSON-LD — sits at the top of a document, and real home pages run past 3 MB. The first 2 MB are parsed and the row says
truncated: true, rather than the page becoming an error. - What is not a page is not downloaded. When the headers say PDF, image or JSON, the connection is closed before the body arrives, and the row is a free
not-htmldiagnostic. - Redirects are followed (up to 10) and recorded:
finalUrlandredirectCount. Relative links resolve against the final URL, or against<base href>when the page has one. - robots.txt is honoured by default. Each origin's file is read once per run; a URL its
User-agent: *rules disallow is not fetched and gets a freerobots-disallowedrow. Rules written for named bots (Googlebot, Facebot, …) are not ours and are ignored. ACrawl-delayin the*group is honoured, up to 30 seconds. - Polite pacing. Requests to one host are spaced 300–500 ms apart however many run in parallel;
maxConcurrencyspeeds up a list of many sites, not one site. - One retry, never a bypass. A bot wall — Cloudflare's "Just a moment", AWS WAF's "Human Verification", DataDome, PerimeterX, Incapsula, or a 403/429 — is asked for once more from a fresh proxy session, because the refusal is usually about the exit IP. If the second answer is a wall too, the row is a free
blockeddiagnostic. Challenges are never solved, waited out or worked around. A transient fault (a reset, a 500/502/503/504) gets one more try from the same session. A timeout is not retried. - Parsing is targeted readers, not a browser: the head is read quote-aware with script and comment bodies masked first (so a
<meta>spelled inside JavaScript is not read), JSON-LD is parsed leniently (comment and CDATA wrappers, trailing commas,//comments, raw newlines in strings, several roots in one block, an HTML-escaped block) and@graphis flattened, and microdata is read from a tolerant element tree so nesting follows the markup.
Privacy: what is removed, and what is kept
Structured data is full of people: article bylines, company founders, testimonial authors with their job titles, event hosts. This Actor describes pages, not people, so before anything is written:
| Where | Rule |
|---|---|
| A Person node anywhere in JSON-LD or microdata | Replaced by {"@type": "Person", "redacted": true} — no @id, name, givenName, familyName, email, telephone, url, image, sameAs, jobTitle or address |
An {"@id": …} reference that points at a Person node | The same marker (a Person's @id is often their profile URL) |
A string in a person-valued property — author, creator, editor, contributor, founder, employee, director, actor, performer, musicBy, reviewedBy, accountablePerson, copyrightHolder, organizer, member and their plural forms, review.author included | "[person]". An Organization node in those properties is kept; an untyped object becomes {"redacted": true} |
<meta name="author"> (the author column, and its entry in metaTags) | Kept only when it is the site's own name (og:site_name, application-name, the publisher's JSON-LD name) or reads as an organisation — a single word, an acronym, or a name with a word like Team, News, Inc, Foundation. Anything that reads as a person's name, and any URL, is null |
Other author-like meta tags — article:author, parsely-author, cXenseParse:author, citation_author, dc.creator, byl, copyright, a twitter:dataN labelled "Written by" | Same rule; the tag stays in metaTags with content: null |
twitter:creator | Kept only when it is the site's own handle |
profile:* Open Graph tags | Never read |
What is kept, so you still know what was there: entities.article.authorType ("Person", "Organization", or null when the author was a bare string), schemaTypes (which still lists Person), redactions (how many values were replaced), and every meta tag's name.
What this does not cover: words the page itself shows. A title such as "Interview with <a writer's name> | Example News", a heading, a description or a review's text is the page's own content and is returned as written. The rule is about structured fields that identify people, not about prose.
Normalised entities
Each entity is null when the page has none, and otherwise always has the same keys. Source order: JSON-LD first (top-level nodes before nested ones), then microdata, then — for article and product only — Open Graph. When JSON-LD and microdata describe the same thing (same name, or one has none), the second fills the first's empty fields; two different things are never mixed. source says where the entity came from.
| Entity | Keys |
|---|---|
article | source, type, headline, description, datePublished, dateModified, section, tags[], image, publisher (organisation name), authorType, wordCount, isAccessibleForFree |
product | source, type, name, brand, sku, gtin, mpn, description, image, url, offers[] (type, name, price, highPrice, priceCurrency, availability, condition, url, seller (organisation), offerCount), aggregateRating (value, count, best, worst) |
organization | source, type, name, legalName, url, logo, sameAs[], contactPoint[] (type, telephone, email, areaServed) |
localBusiness | source, type, name, url, address (streetAddress, addressLocality, addressRegion, postalCode, addressCountry, text), geo (lat, lng), telephone, openingHours[], priceRange, aggregateRating |
event | source, type, name, url, startDate, endDate, location (name, address, url), organizer (organisations only), offers[], eventStatus, attendanceMode, image |
jobPosting | source, type, title, hiringOrganization, datePosted, validThrough, employmentType[], jobLocation (name, address), baseSalary (currency, value, minValue, maxValue, unitText), remote, url |
faq | [{ question, answer }] — answers as plain text, HTML removed; up to 100 |
breadcrumbs | [{ name, url }] in position order |
video | source, name, description, thumbnail, uploadDate, duration, embedUrl, contentUrl |
recipe | source, name, yield, cookTime, prepTime, totalTime, calories, ingredientsCount, rating |
website | source, name, url, searchActionUrlTemplate |
softwareApplication | source, type, name, applicationCategory, operatingSystem, url, image, publisher (organisation), offers[], aggregateRating |
Values are cleaned but not reinterpreted: prices become numbers ("1.299,00" → 1299), enumerations lose their prefix (https://schema.org/InStock → InStock), currencies are upper-cased, dates are returned exactly as the page wrote them.
Input reference
| Field | Type | Default | What it does |
|---|---|---|---|
urls | array of strings | prefilled with five pages | Full URLs or bare domains. The #fragment is dropped; duplicates are read, and billed, once. { "url": … } objects are accepted too |
website | string | empty | A site whose sitemap adds URLs: robots.txt's Sitemap: lines, else /sitemap.xml, /sitemap_index.xml, /sitemap-index.xml, /sitemap/sitemap.xml. Indexes and .xml.gz files are followed. robots.txt is read for its Sitemap: lines even when respectRobots is off |
maxUrlsFromSitemap | integer | 100 | How many URLs the sitemap may add. Reading stops as soon as this many are found |
includeJsonLd | boolean | true | The scrubbed JSON-LD nodes in jsonLd |
includeMicrodata | boolean | true | The scrubbed microdata items in microdata |
includeOpenGraph | boolean | true | openGraph |
includeTwitter | boolean | true | twitter |
includeMetaTags | boolean | true | metaTags, every meta tag in page order |
includeLinks | boolean | true | links (canonical is also its own column either way) |
includeHeadings | boolean | true | headings, the first 50 <h1> and <h2> texts |
includeEntities | boolean | true | entities |
followRedirects | boolean | true | Off: a redirect is a free redirected row naming its target |
respectRobots | boolean | true | Off: robots.txt rules are not applied. Only for sites you are entitled to read regardless |
requestTimeoutSecs | integer | 20 | Per request, 5–60 |
maxConcurrency | integer | 8 | URLs in flight at once, 1–20. Each worker keeps its own proxy session |
maxRunSecs | integer | 240 | Whole-run budget. When it runs out the Actor keeps what it has and files a free timeout row for each URL it never reached |
proxyConfiguration | object | Apify datacenter | Switch to the RESIDENTIAL group for sites whose bot wall refuses datacenter addresses |
Switching an include… option off sets that column to null; every row still has every column.
Output reference
Every row carries the same keys. ok: true is a page; ok: false is a free diagnostic row.
| Field | What it is |
|---|---|
rowType | page or diagnostic |
input, discoveredVia | The entry you supplied (or the website), and input / sitemap |
url, finalUrl, statusCode, redirectCount | What was requested, where it ended, how it answered |
contentType, charset, bytes, truncated, fetchMs | The response's type and declared charset, bytes read, whether reading stopped at 2 MB, time spent |
title, description, keywords, author, lang | The head basics; author only when it is an organisation |
canonical, robotsMeta, viewport, generator, themeColor | Single-value meta and link facts |
publishedAt, modifiedAt | From the article entity, else the date meta tags, else another JSON-LD node's datePublished/dateModified (a WebPage or FAQPage) — as written |
hasAmp, isPaywalled, wordCount | AMP link or AMP document; schema.org isAccessibleForFree (null when unstated); visible words |
headings, openGraph, twitter, metaTags, links | See What comes back |
jsonLd, jsonLdInvalid, schemaTypes, microdata, redactions | Structured data after the scrub, blocks that could not be parsed, every type present, items, replacements made |
entities | See Normalised entities |
ok, error, errorType | Whether the row is a page, and if not, why not |
scrapedAt, source, sourceUrl | When, which host, which URL |
errorType on a diagnostic row is one of:
| Value | Meaning | Charged? |
|---|---|---|
invalid-url | Not an http(s) URL — an email address, an IP address, ftp:, junk — or no input at all | No |
robots-disallowed | The site's robots.txt disallows this URL for anonymous clients | No |
blocked | A bot wall or a 403/429 from two different proxy sessions. Never bypassed | No |
http-error | The site answered 4xx or 5xx (the status is in statusCode) | No |
not-html | The URL is a PDF, image, JSON or other non-page; its body was not downloaded | No |
redirected | The URL redirects and followRedirects is off; error names the target | No |
unreachable | The domain does not resolve, the connection failed, or the redirects loop | No |
timeout | No answer within requestTimeoutSecs, or the run's maxRunSecs ran out first | No |
no-sitemap | The website produced no URL: no sitemap, an empty one, or one that could not be read | No |
parse-error | The page arrived but could not be parsed (not seen in testing; reported rather than guessed) | No |
Pricing
$1.00 per 1,000 pages. Pay-per-event, with the proxy already inside that number.
| Event | What triggers it | FREE | Starter | Scale | Business |
|---|---|---|---|---|---|
| Page (primary) | One page row written to your dataset | $0.001 | $0.001 | $0.0008 | $0.0006 |
| Run started | Once per run, after the first page | $0.001 | $0.001 | $0.001 | $0.001 |
Worked example. 1,000 product URLs, of which 40 sit behind a bot wall and 10 are now 404s:
- 950 pages × $0.001 = $0.95
- 1 run start = $0.001
- 50 diagnostic rows = $0.00
- Total: $0.951
What you are never charged for: a blocked page, a 4xx or 5xx, a PDF or any other non-page, a URL robots.txt disallows, a redirect you asked not to follow, a dead domain, a timeout, a website with no sitemap, robots.txt and sitemap requests, and the retry a refused page gets. If a whole run comes back empty it finishes SUCCEEDED with zero results and a status message that says why ("0 pages. N diagnostic row(s) explain why …"), and bills nothing at all, start fee included.
Set ACTOR_MAX_TOTAL_CHARGE_USD on a run and the Actor stops fetching once the next page would not be covered, rather than handing you rows it cannot bill. It finishes SUCCEEDED with the ceiling named in its status message, and everything already delivered is yours.
Limits, and the ones that might bite
Server HTML only. No browser runs, so metadata that a page's JavaScript injects after load is not seen. Most sites put their meta tags and JSON-LD in the server's HTML because that is what search engines and link-preview bots read — but a single-page app that renders its head client-side will come back thin.
Bot walls are not bypassed. Sites that refuse datacenter addresses come back blocked after one retry from a fresh session. The RESIDENTIAL proxy group sometimes gets through; nothing gets through a challenge that requires a browser, and this Actor does not try.
2 MB per page. Anything after the first 2 MB — a JSON-LD block at the very bottom of an enormous page — is not seen. truncated tells you when that happened.
Microdata, simplified; no RDFa. itemref is not followed, and items inside <noscript> or <template> are not read. RDFa (typeof / property) is not read at all.
robots.txt is checked for the URL you gave, not for each hop of a redirect chain.
The privacy scrub is deliberately strict. A company that a site marks up as a Person (Google Play does this for app developers) is redacted like a person, and a bare-string author is "[person]" even when it is a newsroom. twitter:creator survives only when it is the site's own handle. You lose a little organisation data so that no individual's name gets through.
The upstream format may change. Sites change their markup without notice. When they do you get different (or fewer) fields for that site, never a crash for the run, and a page that cannot be read is a free row.
Use it from an AI agent, or from code
One JSON object in, one flat array out — the shape agent runtimes want. The Actor runs with limited permissions, uses pay-per-event pricing and never enters Standby, so it works over the Apify MCP server and with x402 agentic payments. The Integrations tab pushes results to Slack, a webhook, Zapier, Make, Google Sheets, Snowflake or BigQuery.
curl -X POST "https://api.apify.com/v2/acts/insight.solutions~website-metadata-api/run-sync-get-dataset-items?token=$APIFY_TOKEN" \-H "Content-Type: application/json" \-d '{"urls":["https://www.bbc.com/news","store.steampowered.com/app/413150/Stardew_Valley/"],"includeMetaTags":false}'
# pip install apify-clientfrom apify_client import ApifyClientclient = ApifyClient("<APIFY_TOKEN>")run = client.actor("insight.solutions/website-metadata-api").call(run_input={"urls": ["https://www.bbc.com/news", "https://luma.com/init-conf"],"includeJsonLd": False, # keep the rows small: entities only"includeMetaTags": False,})for row in client.dataset(run["defaultDatasetId"]).iterate_items():if not row["ok"]:print("skipped:", row["url"], row["errorType"])continueentities = row["entities"] or {}event = entities.get("event")print(row["title"], row["schemaTypes"], event and (event["startDate"], event["location"]))
FAQ
Does it render JavaScript? No. It reads the HTML the server sends, which is where meta tags and JSON-LD almost always are. See Limits.
Why is author null on a page that clearly names its writer?
Because the writer is a person, and this Actor does not return people's names. entities.article.authorType tells you the byline was a Person. See Privacy.
Why does jsonLd say "[person]" where the page had a name?
Same rule: a plain-text value in a person-valued property such as author cannot be told apart from a person's name, so it is replaced.
Why is my product's price missing?
Either the page does not publish one in its markup (many shops render the price with JavaScript only), or the shop's markup is in a block this Actor could not parse — jsonLdInvalid counts those. schemaTypes shows what the page did declare.
Can it crawl a whole site?
Give it a website and up to maxUrlsFromSitemap URLs come from its sitemap (default 100, up to 5,000). It does not follow links from page to page.
What does a blocked page cost? Nothing. It costs us two requests (the retry from a fresh session) and you a free diagnostic row.
What happens if every URL fails?
The run finishes SUCCEEDED with zero pages, a status message that names the diagnostic rows, and nothing billed at all. Only an input with no usable URL and no usable website finishes FAILED.
Is the data fresh? Live. Every run reads the pages at that moment; nothing is cached.
Legal and data-protection notes
- Public pages only. The Actor never logs in, never sends cookies, never takes anyone's API key, and honours robots.txt by default.
- No personal names in structured output. Person nodes, person-valued strings and person-like author tags are removed before anything is written, as described above. The page's own visible text (titles, headings, descriptions) is returned as published; if you store it, you are the controller of whatever personal data it contains.
- The content is the site's. Metadata describes pages that belong to their owners. Reusing their text or images is subject to their terms and to the law where you operate.
- Not affiliated with any website whose pages you read. All product names and trademarks belong to their respective owners and are used only to describe the public pages this Actor can read.
Our other Actors
Every Insight Solutions Actor is pay-per-result with no browser, no login and no API key, and every one of them returns free diagnostic rows instead of billing for failures. Prices are per 1,000 results.
Video, audio & social
- YouTube Transcript API — captions as timed segments, text, SRT or VTT, with language fallback and translation.
- YouTube Comments API — comments and replies with likes, pinned and hearted flags, newest or top sort.
- YouTube Channel API — a channel's videos, Shorts and live streams, plus YouTube search.
- Podcast Search, Episodes & Charts API — Apple Podcasts search, charts and full episode feeds.
- Bluesky Scraper — profiles, posts, followers and follows from the public AT Protocol API.
- Telegram Channel Scraper — posts, views and channel stats from public Telegram channels.
- Substack Scraper — posts with full free text, comments and publication profiles.
- Hacker News API — stories, comments, users, front page and a structured "Who is hiring?" parser from the official HN APIs.
- GitHub Repos, Trending & Releases API — repository facts, trending lists, releases and search, no token needed for small runs.
- Discourse Forum API — topics, posts and categories from any Discourse community via its own JSON endpoints, usernames only.
News, documents & the web
- Google News Search, Topics & Real Article URLs — news search and topic feeds with the publisher's real URL decoded.
- Website to Markdown — Content Extractor for LLMs & RAG — any site as clean Markdown, text and heading-aware chunks.
- WordPress Posts & Pages API — posts and pages from any WordPress site as clean text or Markdown, with categories, tags and images.
- Subdomain Finder API — every subdomain of a domain from certificate-transparency logs, resolved over DNS, with an optional live HTTP check.
- Internet Archive API — archive.org search, item metadata, files and reviews.
- Wayback Machine Toolkit — archived URL inventories, snapshots and text diffs between dates.
- Website Technology Detector — the tech stack behind any site, with the evidence for each detection.
- Domain Intelligence API — DNS, RDAP registration, TLS certificate and HTTP facts in one row per domain.
- SEO Page Audit — sitemap crawl with on-page checks, structured data and broken-link reports.
- Keyword Suggestions API — Google, YouTube, Bing, Amazon and eBay autocomplete with alphabet and question expansions.
- Website Contact Extractor — emails, phone numbers and social profiles from any list of websites.
- Web Search Results API — Bing and DuckDuckGo organic results with snippets, no key, no browser.
- Company Enrichment API — a domain in, a company profile out: firmographics, contacts, tech stack, DNS and hiring signal.
- Company Dossier API — one company in, twelve sections out: profile, tech, contacts, DNS, open roles, news, SEC filings, federal awards, recalls, YC batch and apps.
- Press Releases API — GlobeNewswire and PR Newswire releases plus any newsroom feed, by keyword, company, ticker or subject.
- Federal Register API — rules, proposed rules, notices and the Public Inspection desk with dockets, comment deadlines and CFR references.
- Academic Papers Search API — OpenAlex, Crossref, arXiv and PubMed in one row per paper: abstract, citations, open-access PDF, authors and venue.
- RSS & Atom Feed Monitor — any RSS, Atom or JSON feed (or an OPML file) in, only the new items out, with keyword filters and a webhook.
- Website Change Monitor — watch any pages, diff the text between runs, get change rows with added/removed lines, keyword alerts and a webhook.
- Wikipedia & Wikidata API — article text, search, daily pageviews and Wikidata entity facts, any language edition.
Business, finance & jobs
- Congress & Insider Trades API — STOCK Act periodic transaction reports and SEC Form 4 insider trades in one schema.
- Federal Contracts, Grants & Lobbying API — SAM.gov opportunities, USAspending awards, Grants.gov notices and Senate lobbying filings in one schema.
- SEC EDGAR API — filings, XBRL financials and full-text search by ticker or CIK.
- Clinical Trials & FDA API — ClinicalTrials.gov studies plus openFDA recalls, labels, approvals, 510(k)s and adverse-event reports.
- Product & Vehicle Recalls API — CPSC, NHTSA, FDA and USDA recalls, vehicle complaints and ratings, plus a VIN decoder.
- Y Combinator Companies, Batches & Founders — the YC directory with founders and social links, filterable by batch, industry and hiring status.
- Career Site Jobs API — jobs straight from Greenhouse, Lever, Ashby, Workable and 10+ other ATS career sites.
- Workday Jobs API — every posting on any Workday career site, with facets, search, posted dates and full descriptions.
- New Job Postings Monitor — new, closed and changed postings on the career sites you watch.
- Hiring Signals API — Open Roles & Hiring Surge by Company — one row per company per run: open roles, what opened and closed, department and seniority breakdowns, and a hiring-surge flag.
- Remote Jobs API — RemoteOK, Remotive, We Work Remotely, Himalayas, Jobicy and more in one schema, deduplicated.
- Shopify Products API — any Shopify store's catalogue, variants, prices and stock signals.
- WooCommerce Products API — any WooCommerce store's catalogue, prices, variants and stock through its public Store API.
- Shopify App Store API — Shopify App Store listings by category, collection or developer: pricing plans, ratings and merchant reviews.
- Luma Events API — upcoming Luma events by city, category or calendar: dates, venue, price, capacity and organiser calendar, no host names.
- Shopify Store Monitor — price drops, sales, restocks, sell-outs and new products on any Shopify store, one row per change.
- Public Tenders API — EU TED, UK Find a Tender and Contracts Finder notices by keyword, CPV code, country, stage and deadline.
- Nonprofit & IRS 990 Lookup API — search US nonprofits and get EIN, NTEE code and multi-year Form 990 financials.
- OpenStreetMap Places API — businesses and points of interest by category and area from OpenStreetMap: name, address, coordinates, website, phone, opening hours.
Apps & games
- App Store & Google Play Reviews API — reviews from both stores with ratings, versions and developer replies.
- App Store Top Charts & App Search API — Apple top charts by country and genre, plus app search and details.
- App Store Keyword Rank Tracker — where any app ranks for any keyword on the App Store and Google Play, with rank changes and ASO suggestions.
- Steam Reviews API — Steam reviews with playtime, helpfulness and game details.
- Steam Game Data API — prices, tags, review scores, live player counts and top charts.
- Sports Scores & Schedules API — scores, schedules, standings, teams and headlines for the NFL, NBA, MLB, NHL, soccer and more, no key.
- App Store Monitor — App Store and Google Play listings watched between runs: new versions with release notes, price, rating and listing changes, one row per change.