Website Navigation Category Scraper

Extract category-like navigation links from website headers, menus, sidebars, footers, and dropdowns. This Actor is designed for messy real websites where categories are not always stored in clean /category/ or /collections/ URLs.

What it extracts

• Link name
• Link URL
• Source area: header, sidebar, footer, sitemap, or body
• Dropdown parent when visible
• Keywords inferred from link text and URL
• Confidence score

Filtering

The Actor removes common non-category pages such as about us, contact us, privacy policy, terms, login, cart, checkout, blog, careers, case studies, FAQ, support, search, and similar utility pages.

Best use cases

• Website category discovery
• Navigation taxonomy extraction
• Scraper setup before product/service scraping
• Competitor site structure mapping
• Header/menu/sidebar link extraction

Example input

{
  "startUrls": ["https://books.toscrape.com/"],
  "includeFooter": true,
  "includeSitemaps": true,
  "maxLinksPerWebsite": 50
}

Quick start and pricing

Use dryRun: true to preview a sample row without fetching a site. For a live test, enter one public website, keep includeBodyLinks: false, and start with maxLinksPerWebsite: 50. Enable sitemap scanning for category URLs that are not visible in the navigation. The Actor charges per category URL saved; a page can return fewer records than its cap if links are absent, blocked, or filtered as non-categories. confidenceScore is a heuristic, not proof that a URL is an ecommerce category.

The newer maxLinksPerWebsite setting takes precedence over the legacy maxPages alias. To skip sitemap requests, turn off includeSitemaps.

Workflow and operating limits

Catalog and crawl planning: collect category navigation from a public website, export category URLs, then feed the selected URLs to your own page or product crawler. This Actor discovers links, not product records or article text.

Limits and result interpretation

Navigation labels and URL patterns are heuristics, not an authoritative category taxonomy. JavaScript-only menus and protected sites can be incomplete. maxLinksPerWebsite is the current limit; maxPages is a legacy alias.

Result price versus total workflow cost

Base pricing is $2 per 1,000 saved category links. Multiple links from one website count separately. Use maxLinksPerWebsite: 5 for a small test and disable unnecessary sitemap scans.

Actor-start charges may apply even if no paid result is saved. Verify current prices in the Pricing tab; discounts depend on your plan. Budget settings cap charges rather than guaranteeing a quantity of results.

API and n8n workflow

Use the Console API menu to generate the run request for this Actor. Keep your Apify token in a credential, not in a public URL or shared workflow. In n8n, start a run with HTTP Request or the Apify integration, poll until it ends, then download the default dataset as JSON or CSV. Check status fields and error rows before importing data; SUCCEEDED alone does not prove that live data was collected.

Troubleshooting and support

Start with one valid public input. Check Runs, the run status and Logs, then inspect the dataset. If a paid source is used, inspect its run separately. Account concurrency limits can temporarily prevent a child run from starting; submit batches sequentially on small plans. Report the run ID and a redacted input through this Actor's Issues tab. Never post tokens, LinkedIn cookies, or other session credentials. Share a customer run only through Apify's supported opt-in sharing controls.