WebMCP Scraper

WebMCP Scraper inventories the agent-facing catalog of a WebMCP-enabled website. Give it page URLs and get one dataset row per tool, resource, resource template, or prompt, including names, descriptions, JSON schemas, and where the entry came from.

Start with the public Chrome Labs demo: WebMCP explainer.

Need a pass/fail score instead of a catalog? Use WebMCP Validator for a per-URL readiness report.

---

What does WebMCP Scraper do?

WebMCP lets a page register structured tools on document.modelContext so an in-browser agent can call them instead of clicking through a human UI. Sites can also advertise a same-origin MCP HTTP catalog (Cloudflare’s WebMCP packs do this).

This Actor:

1. Opens each start URL in a headless browser
2. Captures tools registered with registerTool
3. Reads declarative HTML tools (<form toolname="…">)
4. Lists tools, resources, resource templates, and prompts from the MCP HTTP catalog when one responds
5. Saves a flat dataset you can filter by kind and source

Use it to build agent directories, snapshot schemas for QA, or watch WebMCP adoption across a URL list.

---

Why scrape WebMCP catalogs?

A WebMCP page is an in-browser tool server. Scraping the catalog — not the DOM — is the reliable way to learn what an agent is allowed to do:

• Agent directories that only list sites with callable tools
• QA snapshots of tool names and JSON schemas after each deploy
• Competitive intel on which products already expose agent APIs
• Cloudflare pack inventory after enabling WebMCP in the dashboard
• RAG / eval datasets built from tool descriptions
• Prompt libraries harvested from MCP prompts/list
• Resource maps for docs, manifests, and signed media catalogs
• Schema diffs between weekly crawls
• Sales enablement for agent platforms pitching publishers
• Research on WebMCP adoption across the public web
• Routing — send an agent only to sites that expose the tool it needs
• Incident checks when an integration reports missing tools

---

How to scrape WebMCP tools and resources

1. Open the Actor in Apify Console
2. Paste page URLs that should expose WebMCP (or the Chrome Labs demo)
3. Leave Probe MCP HTTP catalog on to also collect resources and prompts
4. Raise Extra wait time for SPAs that register tools after hydration
5. Click Start and export Overview

Schedule the Actor or call it from the Apify API. Combine it with WebMCP Validator when you first need a score, then dump the catalog for failing URLs.

Input example

{
  "startUrls": [
    { "url": "https://googlechromelabs.github.io/webmcp-tools/demos/explainer/" }
  ],
  "maxItems": 5,
  "waitUntil": "networkidle",
  "waitForMillis": 4000,
  "probeMcpHttp": true,
  "includeSiteSummary": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

---

Output

Each row is one catalog entry.

{
  "url": "https://googlechromelabs.github.io/webmcp-tools/demos/explainer/",
  "kind": "tool",
  "name": "getAvailability",
  "title": "Get booking availability",
  "description": "List bookable consultation times within a date range.",
  "source": "webmcp-document",
  "hasExecute": true,
  "inputSchema": {
    "type": "object",
    "properties": {
      "startDate": { "type": "string", "format": "date" },
      "endDate": { "type": "string", "format": "date" }
    },
    "required": ["startDate", "endDate"]
  },
  "uri": null,
  "mimeType": null
}

kind is tool, resource, resource_template, prompt, or site (only when nothing else was found). source tells you whether the row came from in-page WebMCP, a declarative form, or MCP HTTP.

Download the dataset as JSON, CSV, or Excel.

---

How much does it cost?

This Actor uses pay-per-event pricing. Platform compute is included in the event price — you are not billed extra Playwright usage.

Event	FREE plan
Actor start	$0.00005
Catalog item (one tool, resource, prompt, or site row)	$9.99 / 1,000 ($0.00999 each)

Paid Apify Store plans get volume discounts (GOLD from $7.48 / 1,000). No proxy is required by default. The Chrome Labs demo (3 tools) is about $0.03 on FREE. Check the Pricing tab for current rates.

---

FAQ

What is WebMCP?

A proposed web API from the W3C Web Machine Learning Community Group. Pages expose tools to agents via document.modelContext. Read the GitHub explainer and Cloudflare’s preview.

Does it execute tools?

No. It only lists tools, resources, and prompts. It does not call execute / tools/call, so it will not submit forms or change account state.

Will it log in?

No. Only public pages. Cookie banners and bot challenges are fine; account login is not supported.

Related Actors

Actor	Job
WebMCP Validator	Per-URL readiness score and spec checks
WebMCP Scraper	One dataset row per tool, resource, or prompt (this Actor)

Open an issue on the Actor page if a public demo URL stops returning catalog rows.

---

Image Credit

Logo and banner use the official Model Context Protocol mark from the MCP docs logo.