API Docs Scraper & Postman Generator

Scrape API documentation pages and generate Postman collections or OpenAPI specs. Built for developers, QA teams, and technical writers who need to work with API documentation.

What It Extracts

• Endpoints: API paths, HTTP methods, descriptions
• Parameters: Query params, path params, headers, request bodies
• Examples: Request/response examples from documentation
• Generated output: Markdown docs, Postman collections, or OpenAPI specs
• Auth signals: Authentication requirements and notes

Why It's Better

Not just scraping - generates ready-to-use Postman collections and OpenAPI specs from documentation pages. Handles multiple documentation formats (REST, GraphQL, code examples).

Input

Field	Type	Default	Description
apiDocUrls	string[]	required	URLs of API documentation pages
outputFormat	string	"markdown"	Output format: markdown, postman, openapi-lite
includeExamples	boolean	true	Include request/response examples

Output Example

{
  "sourceUrl": "https://docs.example.com/api",
  "endpoints": [
    {
      "method": "GET",
      "path": "/api/v1/users",
      "description": "List all users",
      "parameters": [
        { "name": "page", "type": "integer", "required": false, "description": "Page number" }
      ],
      "authNote": "Requires Bearer token"
    }
  ],
  "generatedDocs": "# API Documentation\n\n## GET /api/v1/users\nList all users...",
  "endpointCount": 15,
  "pagesScraped": 8,
  "scrapedAt": "2025-01-15T10:30:00Z"
}

Use Cases

• API integration: Quickly understand an API before coding
• Postman setup: Generate Postman collections from docs
• API documentation: Create standardized docs from existing APIs
• QA testing: Generate test cases from API documentation
• Technical writing: Extract API specs for documentation projects

PPE Pricing

Event	Description	Suggested Price
api-doc-processed	One documentation page processed	$0.005
endpoint-extracted	Per endpoint extracted	$0.001

Limitations

• Endpoint detection is pattern-based, may miss unconventional formats
• Generated OpenAPI specs are "lite" - not full OpenAPI 3.0 compliance
• Cannot access authenticated documentation pages
• Rate limited to respect documentation sites

Legal/Ethical Use

This actor only accesses publicly available API documentation. It does not access private APIs, bypass authentication, or scrape rate-limited endpoints. Users are responsible for compliance with documentation site terms of service.

Local Run

cd actors/api-docs-scraper-generator
apify run --input-file .actor/sample_input.json

Deploy

cd actors/api-docs-scraper-generator
apify push

FAQ

Q: What documentation formats does it support? A: It handles common patterns: headings with method+path, code blocks with HTTP methods, table-based endpoint listings.

Q: Can it generate a complete OpenAPI spec? A: The "openapi-lite" output generates a basic spec. Full OpenAPI 3.0 compliance requires manual refinement.

Q: Does it work with GraphQL documentation? A: Limited support. It can extract query/mutation names but not full GraphQL schemas.

Tags

API documentation scraper, Postman collection generator, OpenAPI generator, API docs, REST API, developer tools, API integration, technical documentation