Pricing
Pay per usage
Developers and AI agents need to read documentation (e.g. Stripe Docs, Next.js Docs), but standard scrapers return noisy HTML that includes: navigation bars headers / footers ads / cookie banners This Actor must return pure content-only Markdown, suitable for vectorization and semantic search.
startUrlstringRequired
The root URL of the documentation site to crawl (e.g., https://docs.example.com/api).
maxDepthintegerOptional
Maximum link depth to crawl from the start URL. Leave empty for unlimited depth.
maxPagesintegerOptional
Maximum number of pages to scrape. Useful for limiting large documentation sites.
Default value of this property is 1000
includeGlobsarrayOptional
Glob patterns for URLs to include (e.g., '/docs/'). Leave empty to include all URLs on the same domain.
excludeGlobsarrayOptional
Glob patterns for URLs to exclude (e.g., '/changelog/', '/blog/').
excludeElementsstringOptional
CSS selectors for elements to strip before converting to Markdown. Separate multiple selectors with commas.
Default value of this property is "nav, footer, .sidebar, script, style, .ads, header, .navigation, .menu, .toc, .breadcrumb, .edit-page, .feedback, .newsletter, aside, [role=\"navigation\"], [role=\"banner\"], [role=\"contentinfo\"]"
contentSelectorstringOptional
CSS selector for the main content area. If not found, the entire body is used after removing excluded elements.
Default value of this property is "main, article, .content, #content, .documentation, .docs-content, .markdown-body, [role=\"main\"]"
