Website to OpenAPI Spec (Hidden API Reverse-Engineering)

Point it at a website or web app. It opens the pages in a real Chromium browser, records every API call the front end makes (XHR, fetch, GraphQL), and writes an OpenAPI 3.1 spec of the API behind the site, ready for Swagger UI, Postman, Insomnia, code generators or an AI agent.

What you get for each endpoint:

• Path templates with named parameters: /api/v1/products/8812/reviews and /api/v1/products/9031/reviews become /api/v1/products/{productId}/reviews; slug URLs like /catalog/blue-cotton-shirt are recognized too
• Query, header and path parameters, typed (integer, boolean, date, uuid…) and marked required when every call had them
• Request and response JSON Schemas merged across every call seen: optional vs required fields, nullable fields, string formats (date-time, email, uri, uuid), id-keyed maps
• Every status code seen (200, 201, 404, 401…) with its own schema
• Auth detected and declared as security schemes: Bearer/JWT, Basic, API keys in headers or query, session cookies
• GraphQL: each operation (name, query, variables schema, result schema), including persisted queries
• Redacted examples and a ready-to-run curl command; tokens, passwords, API keys, cookies, JWTs and emails are masked

Also converts HAR files (DevTools > Network > Save all as HAR, Charles, Proxyman, mitmproxy) into the same spec, with no browsing needed.

What it's for

• Documenting an undocumented or legacy API your client's front end already uses, as a Swagger spec in minutes instead of days of DevTools work
• Building an integration or scraper against the JSON API behind a page instead of parsing HTML
• Competitive and technical research: what endpoints, backends (Algolia, Supabase, Shopify, Contentful, Firebase…) and auth a product uses
• Giving an AI agent a tool definition for a site: the spec drops straight into agent frameworks that load OpenAPI tools

How it compares

	This actor	API & Endpoint Discovery (Store)	mitmproxy2swagger / har-to-openapi (CLI)	Manual DevTools
Output	OpenAPI 3.1 spec + endpoint table + curl	Endpoint list with field names	OpenAPI spec	Notes
Captures traffic for you	Yes, real browser, scrolls and clicks "Load more"	Yes	No, you record it yourself	You click around
Path parameters	Automatic, with names ({productId})	No	Manual: you mark them in a config and run it again	Manual
Request/response schemas	Full JSON Schema, merged across calls	Response field names only	Basic	Manual
Auth, GraphQL operations	Detected	Auth flag only	No	Manual
Secrets in output	Redacted	n/a	Up to you	n/a
Runs in the cloud / MCP	Yes, both	Cloud	Local	Local

Pricing

Pay per event, only for work that produced something:

Event	Price	When
page-captured	$0.005	One page opened and browsed (failed page loads are free)
endpoint-documented	$0.01	One distinct endpoint (method + path template) in the spec

A typical run (10 pages of a web app, 15 endpoints found) costs $0.20. HAR conversion has no page charges: a HAR with 40 endpoints costs $0.40.

Runs default to 4 GB of memory (2 GB minimum): Chromium needs it, and more memory also means more CPU on Apify. Compute is included in the event prices.

If you set a maximum charge for the run, browsing stops when it's reached and the spec keeps the endpoints that were called most often. Free-plan runs browse up to 10 pages.

Input

Field	Default	Notes
startUrls		Pages to start from
maxPages	10	Pages to open in total. Links on the same host are followed, preferring pages of a kind not seen yet (after one product page, the next pick is a category or account page, not another product)
crawlScope	sameHostname	sameSite also follows subdomains; none opens only the start URLs
harUrls		HAR files to convert, by URL
apiScope	site	site: the site's own API hosts and hosted backends its front end calls directly. all: every host except analytics, ads, chat widgets and other trackers
includeUrlPatterns, excludeUrlPatterns		Regexes to narrow down which API URLs are documented
clickSelectors		CSS selectors to click on every page (tabs, filters, "details") to trigger API calls that only happen on interaction
clickLoadMore	true	Clicks up to 2 "Load more / Show more / Next" buttons per page. Never clicks anything that looks like delete, buy, submit or log out; log-out and delete links are never followed
scrolls, waitSecs	3, 3	Scroll steps (lazy loading, infinite scroll) and extra wait per page
cookies, headers		For apps behind a login: cookies exported with Cookie-Editor (JSON), or an Authorization header
title, includeExamples, redactPersonalData		Spec title; examples on/off; email redaction (secrets are always redacted)
proxyConfiguration	None	Pages load fastest from Apify's own IPs. Turn on a proxy (residential for strict sites) if the site blocks the run; if nothing comes through the proxy, the run browses again without it

Example: a public web app

{
  "startUrls": [{ "url": "https://crates.io/crates?sort=downloads" }],
  "maxPages": 8
}

This run opens the listing and six crate pages and documents the API behind crates.io: GET /api/v1/crates (with page, per_page, sort), /api/v1/crates/{crateId}, /api/v1/crates/{crateId}/downloads, /api/v1/crates/{crateId}/owners, /api/v1/summary and /api/v1/site_metadata, for $0.10.

Example: an app behind a login, only its /api/ calls

{
  "startUrls": [{ "url": "https://app.example.com/dashboard" }, { "url": "https://app.example.com/settings" }],
  "maxPages": 20,
  "cookies": [{ "name": "session", "value": "…", "domain": ".example.com" }],
  "includeUrlPatterns": ["/api/"],
  "clickSelectors": ["[role=tab]", ".filters button"]
}

Example: convert a HAR file

{ "harUrls": ["https://api.apify.com/v2/key-value-stores/<store>/records/my-session.har"] }

Output

• openapi.yaml and openapi.json in the run's key-value store (linked from the run's Output tab)
• Dataset: one row per endpoint, for a quick look or a spreadsheet
• SUMMARY: pages browsed, API calls per page, warnings

Dataset row:

{
  "method": "GET",
  "path": "/api/v1/products/{productId}",
  "url": "https://shop.example.com/api/v1/products/{productId}",
  "operationId": "getProductsByProductId",
  "calls": 6,
  "statusCodes": [200, 404],
  "pathParams": ["productId"],
  "queryParams": [],
  "auth": ["Bearer token (JWT)"],
  "responseContentType": "application/json",
  "responseFields": ["id", "slug", "name", "price", "tags", "updatedAt", "discount", "stock"],
  "graphqlOperations": [],
  "exampleUrl": "https://shop.example.com/api/v1/products/101",
  "curl": "curl -X GET 'https://shop.example.com/api/v1/products/101' \\\n  -H 'authorization: Bearer <redacted>' \\\n  -H 'x-client-version: 4.2.0'",
  "seenOnPages": ["https://shop.example.com/product/blue-cotton-shirt"]
}

Spec excerpt:

paths:
  /api/v1/products/{productId}:
    get:
      tags: [products]
      operationId: getProductsByProductId
      parameters:
        - { name: productId, in: path, required: true, schema: { type: integer }, example: 101 }
        - { name: x-client-version, in: header, required: true, schema: { type: string }, example: 4.2.0 }
      security:
        - bearerAuth: []
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema: { $ref: "#/components/schemas/GetProductsByProductIdResponse" }
        "404":
          description: Not Found
          content:
            application/json:
              schema:
                type: object
                properties: { error: { type: string }, message: { type: string } }
                required: [error, message]
components:
  securitySchemes:
    bearerAuth: { type: http, scheme: bearer, bearerFormat: JWT }

The spec documents what the site was seen doing: an endpoint the pages never called, or a field that never appeared, is not in it, and required means "present in every observed response". Browse more pages, click more (clickSelectors), or merge in a HAR of a manual session to cover more.

For AI agents (Standby + MCP)

The actor also runs as a Standby web server:

• MCP: https://jixiaohao--network-to-openapi.apify.actor/mcp with the tool synthesize_openapi (url, maxPages default 3, harUrl, apiScope, clickSelectors, cookies, includeUrlPatterns). It returns the endpoint list and the spec as YAML. Add it to Claude or any MCP client with your Apify token (?token=... or a Bearer header).
• REST: GET /openapi?url=https://example.com&maxPages=3 (endpoints + spec as JSON), &format=yaml for the spec alone, or POST /openapi with a JSON body.

How it works

1. Capture. Each page loads in headless Chromium (images, video, fonts and tracker scripts blocked to save time; a page that's slow to finish loading is still captured). After the network goes quiet the actor scrolls, clicks "Load more" and any selectors you gave, and records every request and response the page's scripts make. Service workers are bypassed so no call is hidden.
2. Filter. Analytics, ads, consent banners, chat widgets, error trackers, feature-flag polling, static assets and CORS preflights are dropped; first-party API hosts and known hosted backends are kept.
3. Template paths. Segments that are clearly ids (numbers, UUIDs, hashes, version numbers like 1.0.229, mixed ids like B08N5WRWNW) become parameters. Sibling URLs that differ in one segment are merged when the values look like slugs or the responses share a structure, but kept apart when they are different resources (/api/users vs /api/orders).
4. Infer schemas. All bodies for an endpoint are merged into one JSON Schema per status code and media type.
5. Redact and write the spec, the endpoint table and curl commands.

Only document APIs you are allowed to access, and respect the site's terms. The actor sends the same requests a normal visitor's browser would; it does not fuzz or probe endpoints.

Development

npm install
npm test        # unit tests plus an end-to-end run in Chromium against the demo shop in test/demo-app.mjs
node test/demo-app.mjs 4321   # the demo shop on its own