n8n web scraping agent
n8n web scraping agent follows links across entire sites, extracting LLM-ready content. It handles JavaScript rendering and outputs clean Markdown, plain text, or HTML.
Trusted by industry leaders all over the world
What data you can get with n8n web scraping agent
Get structured content from entire websites with headings, lists, and code blocks preserved. The Actor strips navigation, footers, ads, and cookie banners.
Output
{ "url": "https://docs.apify.com/academy/web-scraping-for-beginners", "html": null, "text": "Skip to main content\nOn this page\nWeb scraping for beginners\nLearn how to develop web scrapers with this comprehensive and practical course. Go from beginner to expert, all in one place.\nWelcome to Web scraping for beginners...", "crawl": { "depth": 0, "loadedUrl": "https://docs.apify.com/academy/web-scraping-for-beginners", "loadedTime": "2023-04-05T16:26:51.030Z", "referrerUrl": "https://docs.apify.com/academy" }, "markdown": " Web scraping for beginners | Apify Documentation \n\n[Skip to main content](#docusaurus_skipToContent_fallback)\n\nOn this page\n\n# Web scraping for beginners...", "metadata": { "title": "Web scraping for beginners | Apify Documentation", "author": null, "keywords": null, "description": "Learn how to develop web scrapers with this comprehensive and practical course. Go from beginner to expert, all in one place.", "canonicalUrl": "https://docs.apify.com/academy/web-scraping-for-beginners", "languageCode": "en" }, "screenshotUrl": null}How to set up n8n web scraping agent with Apify
Point the Actor at a starting URL and set crawl depth. It follows internal links automatically, choosing headless Firefox for JS-heavy pages or raw HTTP for speed.
Sign up for Apify account01
Creating an account is quick and free. No credit card required. Your account gives you access to more than 20,000+ scrapers and APIs.
Get your Apify API token02
Go to Settings in Apify Console and navigate to the API & Integrations tab. There, create a new token and save it for later.
Test run web scraping agent03
Open web scraping agent in Apify Console and configure your input parameters. Click Start to run the Actor and preview the data structure you receive in your n8n workflow.
Integrate web scraping agent via n8n04
Add the Apify node to your n8n workflow. Select Run Actor as the operation, choose your Actor, and pass your input configuration as JSON. Enable Wait for finish to retrieve results directly in subsequent nodes.
Never get blocked
Every plan (free included) comes with Apify Proxy, which is great for avoiding blocking and giving you access to geo-specific content.
Customers love us
We truly care about the satisfaction of our users and thanks to that we're one of the best-rated data extraction platforms on both G2 and Capterra.
Monitor your runs
With our latest monitoring features, you always have immediate access to valuable insights on the status of your web scraping tasks.
Use the HTTP Request node to call the Apify API. Authenticate with your API token, specify the web scraping agent Actor ID and input parameters, then trigger the run. For short scrapes, you can wait for results synchronously. For longer runs, use webhooks to notify your workflow when data is ready.
Apify offers a free tier with prepaid platform usage - enough for testing and small-scale scraping. Paid plans are available for larger data volumes. No credit card required to try the free tier.
No. Most Apify Actors work through a simple web interface where you configure URLs and extraction settings. Connect the web scraping agent Actor to n8n using the HTTP Request node with no coding required. For advanced customizations, you can modify Actor inputs or write custom extraction logic.
Apify Actors automatically rotate proxies, manage browser fingerprints, solve CAPTCHAs, and handle rate limiting. When a site adds new protections, the Apify team updates the Actors. You get reliable data extraction without managing infrastructure or chasing site changes.