🤖 Semantic Web Crawler & Schema-Enhanced Extractor

The Semantic Web Crawler is the ultimate tool for transforming arbitrary websites into structured, analytics-ready datasets—without requiring custom code per site. It performs a depth-controlled crawl, intelligently renders pages, and extracts meaningful semantic signals and structured data (schema.org) from every page.

It's designed to give your team consistent, rich, and measurable data about a website’s structure and content quality for use in SEO, data science, and research.

---

✨ Why Use This Actor? (The Value Proposition)

This Actor moves beyond basic text scraping to provide context and structure, which is vital for modern data workflows.

• SEO & Content Strategy: Map information architecture, internal links, and content depth. Identify thin/filler content and improve topical coverage.
• Data & Analytics: Build site-wide corpora with consistent features for dashboards, trend analysis, and ML/NLP tasks. Benchmark and compare competing sites.
• Product & Research: Power search and recommendations with clean text and semantic cues. Validate the presence and quality of schema.org structured data.

---

🚀 Main Features & Data Extraction

The core value lies in the rich, standardized JSON record output for every successfully crawled page.

🧭 Controlled & Flexible Crawling

The Actor is built for resilience and control:

• Depth-Limited Crawling: Starts from one or more startUrls and follows internal links up to a configurable maxDepth.
• Resilience & Performance: Includes granular controls for maxConcurrency, maxRetries, and timeouts.
• Proxy Support: Full integration with Apify Proxy (via groups like RESIDENTIAL) and custom proxy URLs.

📊 Rich Extraction Per Page

The final output JSON record contains the following comprehensive data:

• Structured Data: schema_json_ld: Automatically collected, merged, and cleaned schema.org JSON-LD data.
• Content: markdown_content: Clean, structured content output in Markdown format. clean_text: Noise-reduced text content and a unique content_hash for change detection.
• Semantic Structure: Detailed link graph, headings hierarchy, tables, and lists.
• Content Blocks: Categorization of content into per-block types (e.g., heading, paragraph, list, table, quote) with word counts and link/image presence.
• Metrics: text_metrics: Word, sentence, and paragraph counts, averages, and normalized top keywords. technical_metrics: HTTP status code and HTML size.

---

⚙️ How to Use (Quick Start)

The Actor requires minimal setup. The only mandatory setting is the list of websites to start crawling.

1. Set Start URLs

Specify the entry point(s) for the crawl.

2. Configure Crawl Depth

Set maxDepth to define how many link clicks deep the crawler should go (e.g., 1 for just the homepage links, 5 for deep site analysis).

3. Run with Example Input

Use the following JSON structure to crawl apify.com up to depth 5 using Residential proxies:

{
    "maxConcurrency": 5,
    "maxDepth": 5,
    "maxLinksPerPage": 5,
    "maxRetries": 3,
    "proxy": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    },
    "startUrls": [
        "https://www.apify.com/"
    ]
}

---

📋 Input Schema (Full Parameters)

Field	Type	Default	Description
startUrls	Array of URLs	N/A	REQUIRED. The starting point(s) for the crawl.
maxDepth	Integer	1	Maximum link depth to crawl from the start URL.
maxLinksPerPage	Integer	50	Maximum number of internal links to queue from any single page.
maxConcurrency	Integer	5	Maximum number of pages to process simultaneously. Lower this for smaller sites.
maxRetries	Integer	2	Maximum number of times to retry a failed request.
proxy	Object	N/A	Proxy settings (useApifyProxy, groups, countryCode).

---

📁 Data Output Structure (Sample Record)

The Actor saves one JSON record per page to the Dataset. This output is standardized for seamless downstream integration.

[
  {
    "url": "https://apify.com/",
    "domain": "apify.com",
    "title": "Apify: Full-stack web scraping and data extraction platform",
    "clean_content": "New Join the Apify $1M Challenge. Build to win! Get real-time web data for your AI Apify Actors scrape up-to-date web data from any website for AI apps and agents,\n social media monitoring, competitive intelligence, lead generation, and product research. Try it now TikTok Scraper clockworks / tiktok-scraper Extract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools. Clockworks 92K 4.7 Google Maps Scraper compass / crawler-google-places Extract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools. Compass 201K 4.8 Instagram Scraper apify / instagram-scraper Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools. Apify 147K 4.6 Website Content Crawler apify / website-content-crawler Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem. Apify 86K 4.6 Amazon Scraper junglee / free-amazon-product-scraper Gets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions, and ASIN. Junglee 9K 5.0 Facebook Posts Scraper apify / facebook-posts-scraper Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports. Apify 36K 4.6 TikTok Scraper clockworks / tiktok-scraper Extract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools. Clockworks 92K 4.7 Google Maps Scraper compass / crawler-google-places Extract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools. Compass 201K 4.8 Instagram Scraper apify / instagram-scraper Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools. Apify 147K 4.6 Website Content Crawler apify / website-content-crawler Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem. Apify 86K 4.6 Amazon Scraper junglee / free-amazon-product-scraper Gets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions, and ASIN. Junglee 9K 5.0 Facebook Posts Scraper apify / facebook-posts-scraper Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports. Apify 36K 4.6 Browse 7,000+ Actors Trusted by global technology leaders Not just a web scraping API Marketplace of 7,000+ Actors Apify has Actors for scraping websites, automating the web, and feeding AI with web data. Visit Apify Store Build and deploy your own Have a new use case? Start building new Actors with our code templates and extensive guides. Start building Or we can build it for you Rely on our experts to deliver and maintain custom web scraping solutions for you. Learn more Easily integrate Zapier GitHub Google Sheets Pinecone any app Airbyte MCP clients Google Drive Slack Zapier with Actors Browse integrations View SDKs Build and deploy reliable scrapers Open-source tools Proxies Unblocking Cloud deployment Monitoring Data processing We love open source Apify works great with both Python and JavaScript, as well as Playwright, Puppeteer, Selenium, Scrapy, and Crawlee - our own web crawling and browser automation library. JavaScript Python 1 import { PuppeteerCrawler , Dataset } from \"crawlee\" ; 2 3 const crawler = new PuppeteerCrawler ( { 4 async requestHandler ( { request , page , enqueueLinks } ) { 5 await Dataset . pushData ( { 6 url : request . url , 7 title : await page . title ( ) , 8 } ) ; 9 await enqueueLinks ( ) ; 10 } , 11 } ) ; 12 13 await crawler . run ( [ \"https://crawlee.dev\" ] ) ; LlamaIndex Langchain Playwright Puppeteer Cheerio Selenium Scrapy BeautifulSoup Learn. Web Scraping Academy Classes for beginners and experts. Learn about web scraping and automation with our free courses. Visit Academy Code. Code templates JavaScript, TypeScript, and Python templates to quick-start your web scraping projects. Get started Connect. Discord community Get help from the Apify developer community of more than 11,500 members. Join community Publish Actors. Get paid. Reach thousands of new customers Building and running a SaaS is hard. Building an Actor and selling it on Apify Store is 10x easier. Get users from day one. Learn more No upfront costs Publishing your Actor is free of charge—the customers pay for the computing resources. New creators get $500 free platform credits. Rely on Apify infra Actors scale automatically as you gain new users. You don’t need to worry about compute, storage, proxies, or authentication. Billing is on us Handling payments, taxes, and invoicing is a painful part of running a SaaS. Apify does all that and sends you a net payout every month. Enterprise-grade solution Secure and reliable web data extraction provider for any scale. 99.95% uptime. SOC2, GDPR, and CCPA compliant. Contact sales Learn more We looked at several providers, and Apify was the most complete, reliant solution we found. It was miles ahead of everything else we reviewed. Pranav Singh Engineering Manager at Intercom We selected Apify because of their vast experience with web data collection to empower our sales team with fresh, unique leads. Filip Popovic COO at Groupon Our collaboration with Apify proves that advanced IT tools leveraging AI can be the key in detecting infringements of consumer protection legislation. Marie-Paule Benassi Consumer Affairs Director at EU Read more customer stories Apify Professional Services Our experienced team can help you design, implement, and successfully execute your web scraping project. Learn more It's time to run \nyour first Actor. Get started Get a demo",
    "markdown": "## Apify Professional Services\n\nOur experienced team can help you design, implement, and successfully execute your web scraping project.",
    "depth": 0,
    "content_hash": "3b1a6db9da882c1194acc3e9bd36469f29ff02e510612b71c0c8563894200b47",
    "extraction_timestamp": "2025-11-14T17:30:31.308234Z",
    "schema_org_data": {
      "@graph": []
    },
    "semantic_structure": {
      "headings": [
        {
          "level": 1,
          "text": "Get real-time web data for your AI",
          "subsections": [
            {
              "level": 3,
              "text": "TikTok Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Google Maps Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Instagram Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Website Content Crawler",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Amazon Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Facebook Posts Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "TikTok Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Google Maps Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Instagram Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Website Content Crawler",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Amazon Scraper",
              "subsections": []
            },
            {
              "level": 3,
              "text": "Facebook Posts Scraper",
              "subsections": []
            },
            {
              "level": 2,
              "text": "Not just a web scraping API",
              "subsections": [
                {
                  "level": 3,
                  "text": "Marketplace of 7,000+ Actors",
                  "subsections": []
                },
                {
                  "level": 3,
                  "text": "Build and deploy your own",
                  "subsections": []
                },
                {
                  "level": 3,
                  "text": "Or we can build it for you",
                  "subsections": []
                }
              ]
            },
            {
              "level": 2,
              "text": "Easily integrate ZapierGitHubGoogle SheetsPineconeany appAirbyteMCP clientsGoogle DriveSlackZapierwith Actors",
              "subsections": []
            },
            {
              "level": 2,
              "text": "Build and deploy reliable scrapers",
              "subsections": [
                {
                  "level": 3,
                  "text": "We love open source",
                  "subsections": []
                }
              ]
            },
            {
              "level": 2,
              "text": "Learn.",
              "subsections": [
                {
                  "level": 3,
                  "text": "Web Scraping Academy",
                  "subsections": []
                }
              ]
            },
            {
              "level": 2,
              "text": "Code.",
              "subsections": [
                {
                  "level": 3,
                  "text": "Code templates",
                  "subsections": []
                }
              ]
            },
            {
              "level": 2,
              "text": "Connect.",
              "subsections": [
                {
                  "level": 3,
                  "text": "Discord community",
                  "subsections": []
                }
              ]
            },
            {
              "level": 2,
              "text": "Publish Actors. Get paid.",
              "subsections": [
                {
                  "level": 3,
                  "text": "Reach thousands of new customers",
                  "subsections": [
                    {
                      "level": 4,
                      "text": "No upfront costs",
                      "subsections": []
                    },
                    {
                      "level": 4,
                      "text": "Rely on Apify infra",
                      "subsections": []
                    },
                    {
                      "level": 4,
                      "text": "Billing is on us",
                      "subsections": []
                    }
                  ]
                }
              ]
            },
            {
              "level": 2,
              "text": "Enterprise-grade solution",
              "subsections": []
            },
            {
              "level": 2,
              "text": "Apify Professional Services",
              "subsections": []
            },
            {
              "level": 2,
              "text": "It's time to run \nyour first Actor.",
              "subsections": []
            }
          ]
        }
      ],
      "tables": [],
      "lists": {
        "ordered": [],
        "unordered": [
          {
            "items": 41,
            "text": [
              "ProductBackStart here!Get data with ready-made web scrapers for popular websitesBrowse 8,000+ ActorsApify platformApify StorePre-built web scraping toolsActorsBuild and run serverless programsIntegrationsConnect with apps and servicesAnti-blockingAnti-blockingScrape without getting blockedProxyRotate scraper IP addressesOpen sourceCrawleeWeb scraping and crawling library",
              "Apify StorePre-built web scraping tools",
              "ActorsBuild and run serverless programs",
              "IntegrationsConnect with apps and services",
              "Anti-blockingScrape without getting blocked",
              "ProxyRotate scraper IP addresses",
              "CrawleeWeb scraping and crawling library",
              "SolutionsBackMCP server configurationConfigure your Apify MCP server with Actors and tools for seamless integration with MCP clients.Start buildingWeb data forEnterpriseStartupsUniversitiesNonprofitsUse casesData for generative AIData for AI agentsLead generationMarket researchView more →ConsultingApify Professional ServicesApify Partners",
              "Enterprise",
              "Startups",
              "Universities",
              "Nonprofits",
              "Data for generative AI",
              "Data for AI agents",
              "Lead generation",
              "Market research",
              "View more →",
              "Apify Professional Services",
              "Apify Partners",
              "DevelopersBackDocumentationFull reference for the Apify platformGet startedCode templatesPython, JavaScript, and TypeScriptWeb scraping academyCourses for beginners and expertsMonetize your codePublish your scrapers and get paidLearnAPI referenceCLISDKCrawleePublish tools on Apify and win big prizesJoin the challenge",
              "DocumentationFull reference for the Apify platform",
              "Code templatesPython, JavaScript, and TypeScript",
              "Web scraping academyCourses for beginners and experts",
              "Monetize your codePublish your scrapers and get paid",
              "API reference",
              "CLI",
              "SDK",
              "Crawlee",
              "ResourcesBackHelp and supportAdvice and answers about ApifyActor ideasGet inspired to build ActorsChangelogSee what’s new on ApifyCustomer storiesFind out how others use ApifyCompanyAbout ApifyContact usBlogLive eventsPartnersJobsWe're hiring!Join our DiscordTalk to scraping experts",
              "Help and supportAdvice and answers about Apify",
              "Actor ideasGet inspired to build Actors",
              "ChangelogSee what’s new on Apify",
              "Customer storiesFind out how others use Apify",
              "About Apify",
              "Contact us",
              "Blog",
              "Live events",
              "Partners",
              "JobsWe're hiring!",
              "Pricing",
              "Contact sales"
            ]
          },
          {
            "items": 3,
            "text": [
              "Apify StorePre-built web scraping tools",
              "ActorsBuild and run serverless programs",
              "IntegrationsConnect with apps and services"
            ]
          },
          {
            "items": 2,
            "text": [
              "Anti-blockingScrape without getting blocked",
              "ProxyRotate scraper IP addresses"
            ]
          },
          {
            "items": 1,
            "text": [
              "CrawleeWeb scraping and crawling library"
            ]
          },
          {
            "items": 4,
            "text": [
              "Enterprise",
              "Startups",
              "Universities",
              "Nonprofits"
            ]
          },
          {
            "items": 5,
            "text": [
              "Data for generative AI",
              "Data for AI agents",
              "Lead generation",
              "Market research",
              "View more →"
            ]
          },
          {
            "items": 2,
            "text": [
              "Apify Professional Services",
              "Apify Partners"
            ]
          },
          {
            "items": 1,
            "text": [
              "DocumentationFull reference for the Apify platform"
            ]
          },
          {
            "items": 3,
            "text": [
              "Code templatesPython, JavaScript, and TypeScript",
              "Web scraping academyCourses for beginners and experts",
              "Monetize your codePublish your scrapers and get paid"
            ]
          },
          {
            "items": 4,
            "text": [
              "API reference",
              "CLI",
              "SDK",
              "Crawlee"
            ]
          },
          {
            "items": 4,
            "text": [
              "Help and supportAdvice and answers about Apify",
              "Actor ideasGet inspired to build Actors",
              "ChangelogSee what’s new on Apify",
              "Customer storiesFind out how others use Apify"
            ]
          },
          {
            "items": 6,
            "text": [
              "About Apify",
              "Contact us",
              "Blog",
              "Live events",
              "Partners",
              "JobsWe're hiring!"
            ]
          },
          {
            "items": 4,
            "text": [
              "Apify Store",
              "Integrations",
              "Proxy",
              "Crawlee"
            ]
          },
          {
            "items": 4,
            "text": [
              "Documentation",
              "Code templates",
              "API reference",
              "Get paid on Apify"
            ]
          },
          {
            "items": 2,
            "text": [
              "Professional Services",
              "Apify Partners"
            ]
          },
          {
            "items": 3,
            "text": [
              "Help & Support",
              "Submit your ideas",
              "Forum"
            ]
          },
          {
            "items": 5,
            "text": [
              "APIs",
              "What is web scraping?",
              "Best web scraping tools",
              "Python web scraping libraries",
              "Scrapers"
            ]
          },
          {
            "items": 10,
            "text": [
              "About Apify",
              "Contact us",
              "Events",
              "Blog",
              "Become an affiliate",
              "Customer stories",
              "Changelog",
              "JobsWe're hiring!",
              "Brand",
              "Impressum"
            ]
          },
          {
            "items": 6,
            "text": [
              "",
              "",
              "",
              "",
              "",
              ""
            ]
          },
          {
            "items": 2,
            "text": [
              "",
              ""
            ]
          },
          {
            "items": 6,
            "text": [
              "",
              "",
              "",
              "",
              "",
              ""
            ]
          }
        ]
      },
      "links": {
        "all": [
          {
            "text": "Skip to content",
            "url": "https://apify.com/#main",
            "type": "internal",
            "rel": []
          },
          {
            "text": "",
            "url": "https://apify.com/",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Get started",
            "url": "https://console.apify.com/sign-up",
            "type": "external",
            "rel": []
          },
          {
            "text": "Log in",
            "url": "https://console.apify.com/sign-in",
            "type": "external",
            "rel": []
          },
          {
            "text": "Start here!Get data with ready-made web scrapers for popular websitesBrowse 8,000+ Actors",
            "url": "https://apify.com/store",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Apify StorePre-built web scraping tools",
            "url": "https://apify.com/store",
            "type": "internal",
            "rel": []
          },
          {
            "text": "ActorsBuild and run serverless programs",
            "url": "https://apify.com/actors",
            "type": "internal",
            "rel": []
          },
          {
            "text": "IntegrationsConnect with apps and services",
            "url": "https://apify.com/integrations",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Anti-blockingScrape without getting blocked",
            "url": "https://apify.com/anti-blocking",
            "type": "internal",
            "rel": []
          },
          {
            "text": "ProxyRotate scraper IP addresses",
            "url": "https://apify.com/proxy",
            "type": "internal",
            "rel": []
          },
          {
            "text": "CrawleeWeb scraping and crawling library",
            "url": "https://crawlee.dev/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "MCP server configurationConfigure your Apify MCP server with Actors and tools for seamless integration with MCP clients.Start building",
            "url": "https://mcp.apify.com/",
            "type": "external",
            "rel": []
          },
          {
            "text": "Enterprise",
            "url": "https://apify.com/enterprise",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Startups",
            "url": "https://apify.com/resources/startups",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Universities",
            "url": "https://apify.com/resources/universities",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Nonprofits",
            "url": "https://apify.com/resources/nonprofits",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Data for generative AI",
            "url": "https://apify.com/use-cases/data-for-generative-ai",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Data for AI agents",
            "url": "https://apify.com/use-cases/data-for-ai-agents",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Lead generation",
            "url": "https://apify.com/use-cases/lead-generation",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Market research",
            "url": "https://apify.com/use-cases/market-research",
            "type": "internal",
            "rel": []
          },
          {
            "text": "View more →",
            "url": "https://apify.com/use-cases",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Apify Professional Services",
            "url": "https://apify.com/professional-services",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Apify Partners",
            "url": "https://apify.com/partners",
            "type": "internal",
            "rel": []
          },
          {
            "text": "DocumentationFull reference for the Apify platform",
            "url": "https://docs.apify.com/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Code templatesPython, JavaScript, and TypeScript",
            "url": "https://apify.com/templates",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Web scraping academyCourses for beginners and experts",
            "url": "https://docs.apify.com/academy",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Monetize your codePublish your scrapers and get paid",
            "url": "https://apify.com/partners/actor-developers",
            "type": "internal",
            "rel": []
          },
          {
            "text": "API reference",
            "url": "https://docs.apify.com/api",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "CLI",
            "url": "https://docs.apify.com/cli/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "SDK",
            "url": "https://docs.apify.com/sdk",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Crawlee",
            "url": "https://crawlee.dev/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Publish tools on Apify and win big prizesJoin the challenge",
            "url": "https://apify.com/challenge",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Help and supportAdvice and answers about Apify",
            "url": "https://help.apify.com/en/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Actor ideasGet inspired to build Actors",
            "url": "https://apify.com/ideas",
            "type": "internal",
            "rel": []
          },
          {
            "text": "ChangelogSee what’s new on Apify",
            "url": "https://apify.com/change-log",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Customer storiesFind out how others use Apify",
            "url": "https://apify.com/success-stories",
            "type": "internal",
            "rel": []
          },
          {
            "text": "About Apify",
            "url": "https://apify.com/about",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Contact us",
            "url": "https://apify.com/contact",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Blog",
            "url": "https://blog.apify.com/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Live events",
            "url": "https://lu.ma/apify",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "Partners",
            "url": "https://apify.com/partners",
            "type": "internal",
            "rel": []
          },
          {
            "text": "JobsWe're hiring!",
            "url": "https://apify.com/jobs",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Join our DiscordTalk to scraping experts",
            "url": "https://discord.com/invite/jyEM2PRvMU",
            "type": "external",
            "rel": []
          },
          {
            "text": "Pricing",
            "url": "https://apify.com/pricing",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Contact sales",
            "url": "https://apify.com/contact-sales",
            "type": "internal",
            "rel": []
          },
          {
            "text": "NewJoin the Apify $1M Challenge. Build to win!",
            "url": "https://apify.com/challenge",
            "type": "internal",
            "rel": []
          },
          {
            "text": "TikTok Scraperclockworks/tiktok-scraperExtract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Clockworks92K4.7",
            "url": "https://apify.com/clockworks/tiktok-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Google Maps Scrapercompass/crawler-google-placesExtract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.Compass201K4.8",
            "url": "https://apify.com/compass/crawler-google-places",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Instagram Scraperapify/instagram-scraperScrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Apify147K4.6",
            "url": "https://apify.com/apify/instagram-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Website Content Crawlerapify/website-content-crawlerCrawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.Apify86K4.6",
            "url": "https://apify.com/apify/website-content-crawler",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Amazon Scraperjunglee/free-amazon-product-scraperGets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions, and ASIN.Junglee9K5.0",
            "url": "https://apify.com/junglee/free-amazon-product-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Facebook Posts Scraperapify/facebook-posts-scraperExtract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.Apify36K4.6",
            "url": "https://apify.com/apify/facebook-posts-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "TikTok Scraperclockworks/tiktok-scraperExtract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Clockworks92K4.7",
            "url": "https://apify.com/clockworks/tiktok-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Google Maps Scrapercompass/crawler-google-placesExtract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.Compass201K4.8",
            "url": "https://apify.com/compass/crawler-google-places",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Instagram Scraperapify/instagram-scraperScrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Apify147K4.6",
            "url": "https://apify.com/apify/instagram-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Website Content Crawlerapify/website-content-crawlerCrawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.Apify86K4.6",
            "url": "https://apify.com/apify/website-content-crawler",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Amazon Scraperjunglee/free-amazon-product-scraperGets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions, and ASIN.Junglee9K5.0",
            "url": "https://apify.com/junglee/free-amazon-product-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Facebook Posts Scraperapify/facebook-posts-scraperExtract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.Apify36K4.6",
            "url": "https://apify.com/apify/facebook-posts-scraper",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Browse 7,000+ Actors",
            "url": "https://apify.com/store",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Marketplace of 7,000+ ActorsApify has Actors for scraping websites, automating the web, and feeding AI with web data.Visit Apify Store",
            "url": "https://apify.com/store",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Build and deploy your ownHave a new use case? Start building new Actors with our code templates and extensive guides.Start building",
            "url": "https://apify.com/actors",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Or we can build it for youRely on our experts to deliver and maintain custom web scraping solutions for you.Learn more",
            "url": "https://apify.com/professional-services",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Browse integrations",
            "url": "https://apify.com/integrations",
            "type": "internal",
            "rel": []
          },
          {
            "text": "View SDKs",
            "url": "https://docs.apify.com/api",
            "type": "external",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "",
            "url": "https://crawlee.dev",
            "type": "external",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "LlamaIndex",
            "url": "https://apify.com/templates/python-llamaindex-agent",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Langchain",
            "url": "https://apify.com/templates/js-langchain",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Playwright",
            "url": "https://apify.com/templates/ts-crawlee-playwright-chrome",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Puppeteer",
            "url": "https://apify.com/templates/ts-crawlee-puppeteer-chrome",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Cheerio",
            "url": "https://apify.com/templates/ts-crawlee-cheerio",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Selenium",
            "url": "https://apify.com/templates/python-selenium",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Scrapy",
            "url": "https://apify.com/templates/python-scrapy",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "BeautifulSoup",
            "url": "https://apify.com/templates/python-crawlee-beautifulsoup",
            "type": "internal",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Web Scraping AcademyClasses for beginners and experts. Learn about web scraping and automation with our free courses.Visit Academy",
            "url": "https://docs.apify.com/academy",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Code templatesJavaScript, TypeScript, and Python templates to quick-start your web scraping projects.Get started",
            "url": "https://apify.com/templates",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Discord communityGet help from the Apify developer community of more than 11,500 members.Join community",
            "url": "https://discord.com/invite/jyEM2PRvMU",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "Learn more",
            "url": "https://apify.com/partners/actor-developers",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Contact sales",
            "url": "https://apify.com/contact-sales",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Learn more",
            "url": "https://apify.com/enterprise",
            "type": "internal",
            "rel": []
          },
          {
            "text": "We looked at several providers, and Apify was the most complete, reliant solution we found. It was miles ahead of everything else we reviewed.Pranav SinghEngineering Manager at Intercom",
            "url": "https://blog.apify.com/intercom-customer-support-ai-chatbot-web-scraping",
            "type": "external",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "We selected Apify because of their vast experience with web data collection to empower our sales team with fresh, unique leads.Filip PopovicCOO at Groupon",
            "url": "https://blog.apify.com/groupon-reaches-new-merchants-with-web-data-collection",
            "type": "external",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Our collaboration with Apify proves that advanced IT tools leveraging AI can be the key in detecting infringements of consumer protection legislation.Marie-Paule BenassiConsumer Affairs Director at EU",
            "url": "https://blog.apify.com/how-web-scraping-ai-and-the-eu-have-come-together-to-sweep-away-fake-discounts-in-europe",
            "type": "external",
            "rel": [
              "noopener"
            ]
          },
          {
            "text": "Read more customer stories",
            "url": "https://apify.com/success-stories",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Learn more",
            "url": "https://apify.com/professional-services",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Get started",
            "url": "https://console.apify.com/sign-up",
            "type": "external",
            "rel": []
          },
          {
            "text": "Get a demo",
            "url": "https://apify.com/contact-sales/demo",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Apify Store",
            "url": "https://apify.com/store",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Integrations",
            "url": "https://apify.com/integrations",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Proxy",
            "url": "https://apify.com/proxy",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Crawlee",
            "url": "https://crawlee.dev/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Documentation",
            "url": "https://docs.apify.com/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Code templates",
            "url": "https://apify.com/templates",
            "type": "internal",
            "rel": []
          },
          {
            "text": "API reference",
            "url": "https://docs.apify.com/api",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Get paid on Apify",
            "url": "https://apify.com/partners/actor-developers",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Professional Services",
            "url": "https://apify.com/professional-services",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Apify Partners",
            "url": "https://apify.com/partners",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Help & Support",
            "url": "https://help.apify.com/en/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Submit your ideas",
            "url": "https://apify.com/ideas",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Forum",
            "url": "https://discord.apify.com/",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "APIs",
            "url": "https://apify.com/api",
            "type": "internal",
            "rel": []
          },
          {
            "text": "What is web scraping?",
            "url": "https://blog.apify.com/what-is-web-scraping/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Best web scraping tools",
            "url": "https://blog.apify.com/best-web-scraping-tools/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Python web scraping libraries",
            "url": "https://blog.apify.com/what-are-the-best-python-web-scraping-libraries/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Scrapers",
            "url": "https://apify.com/scrapers",
            "type": "internal",
            "rel": []
          },
          {
            "text": "About Apify",
            "url": "https://apify.com/about",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Contact us",
            "url": "https://apify.com/contact",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Events",
            "url": "https://lu.ma/apify",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "Blog",
            "url": "https://blog.apify.com/",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Become an affiliate",
            "url": "https://apify.com/partners/affiliate",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Customer stories",
            "url": "https://apify.com/success-stories",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Changelog",
            "url": "https://apify.com/change-log",
            "type": "internal",
            "rel": []
          },
          {
            "text": "JobsWe're hiring!",
            "url": "https://apify.com/jobs",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Brand",
            "url": "https://apify.com/resources/brand",
            "type": "internal",
            "rel": []
          },
          {
            "text": "Impressum",
            "url": "https://docs.apify.com/legal",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "",
            "url": "https://apify.com/",
            "type": "internal",
            "rel": []
          },
          {
            "text": "",
            "url": "http://linkedin.com/company/apify/",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://x.com/apify",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://github.com/apify",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://www.youtube.com/apify",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://discord.com/invite/jyEM2PRvMU",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://www.tiktok.com/@apifytech",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://docs.apify.com/legal/gdpr-information",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "",
            "url": "https://trust.apify.com/",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://www.getapp.com/business-intelligence-analytics-software/a/apify/",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://www.softwareadvice.com/data-extraction/apify-profile/",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://www.capterra.com/p/150854/Apify/",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://www.g2.com/products/apify/reviews",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://www.trustradius.com/products/apify/reviews",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "",
            "url": "https://crozdesk.com/software/apify",
            "type": "external",
            "rel": [
              "external",
              "noopener",
              "nofollow"
            ]
          },
          {
            "text": "Loading status...",
            "url": "https://status.apify.com/",
            "type": "external",
            "rel": []
          },
          {
            "text": "Terms of Use",
            "url": "https://docs.apify.com/legal/general-terms-and-conditions",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Privacy Policy",
            "url": "https://docs.apify.com/legal/privacy-policy",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "Cookie Policy",
            "url": "https://docs.apify.com/legal/cookie-policy",
            "type": "external",
            "rel": [
              "external",
              "noopener"
            ]
          },
          {
            "text": "© 2025 Apify",
            "url": "https://docs.apify.com/legal",
            "type": "external",
            "rel": [
              "noopener"
            ]
          }
        ],
        "internal_count": 82,
        "external_count": 52,
        "domains": {
          "apify.com": 82,
          "console.apify.com": 3,
          "crawlee.dev": 4,
          "mcp.apify.com": 1,
          "docs.apify.com": 15,
          "help.apify.com": 2,
          "blog.apify.com": 8,
          "lu.ma": 2,
          "discord.com": 3,
          "discord.apify.com": 1,
          "linkedin.com": 1,
          "x.com": 1,
          "github.com": 1,
          "www.youtube.com": 1,
          "www.tiktok.com": 1,
          "trust.apify.com": 1,
          "www.getapp.com": 1,
          "www.softwareadvice.com": 1,
          "www.capterra.com": 1,
          "www.g2.com": 1,
          "www.trustradius.com": 1,
          "crozdesk.com": 1,
          "status.apify.com": 1
        }
      }
    },
    "content_blocks": [
      {
        "type": "div",
        "text": "NewJoin the Apify $1M Challenge. Build to win!Get real-time web data for your AIApify Actors scrape up-to-date web data from any website for AI apps and agents,\n social media monitoring, competitive intelligence, lead generation, and product research.Try it nowTikTok Scraperclockworks/tiktok-scraperExtract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Clockworks92K4.7Google Maps Scrapercompass/crawler-google-placesExtract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.Compass201K4.8Instagram Scraperapify/instagram-scraperScrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Apify147K4.6Website Content Crawlerapify/website-content-crawlerCrawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.Apify86K4.6Amazon Scraperjunglee/free-amazon-product-scraperGets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions, and ASIN.Junglee9K5.0Facebook Posts Scraperapify/facebook-posts-scraperExtract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.Apify36K4.6TikTok Scraperclockworks/tiktok-scraperExtract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Clockworks92K4.7Google Maps Scrapercompass/crawler-google-placesExtract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.Compass201K4.8Instagram Scraperapify/instagram-scraperScrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.Apify147K4.6Website Content Crawlerapify/website-content-crawlerCrawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.Apify86K4.6Amazon Scraperjunglee/free-amazon-product-scraperGets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions, and ASIN.Junglee9K5.0Facebook Posts Scraperapify/facebook-posts-scraperExtract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.Apify36K4.6Browse 7,000+ ActorsTrusted by global technology leadersNot just a web scraping APIMarketplace of 7,000+ ActorsApify has Actors for scraping websites, automating the web, and feeding AI with web data.Visit Apify StoreBuild and deploy your ownHave a new use case? Start building new Actors with our code templates and extensive guides.Start buildingOr we can build it for youRely on our experts to deliver and maintain custom web scraping solutions for you.Learn moreEasily integrate ZapierGitHubGoogle SheetsPineconeany appAirbyteMCP clientsGoogle DriveSlackZapierwith ActorsBrowse integrationsView SDKsBuild and deploy reliable scrapersOpen-source toolsProxiesUnblockingCloud deploymentMonitoringData processingWe love open sourceApify works great with both Python and JavaScript, as well as Playwright, Puppeteer, Selenium, Scrapy, and Crawlee - our own web crawling and browser automation library.JavaScriptPython1import { PuppeteerCrawler, Dataset } from \"crawlee\";2\n3const crawler = new PuppeteerCrawler({4  async requestHandler({ request, page, enqueueLinks }) {5    await Dataset.pushData({6      url: request.url,7      title: await page.title(),8    });9    await enqueueLinks();10  },11});12\n13await crawler.run([\"https://crawlee.dev\"]);LlamaIndexLangchainPlaywrightPuppeteerCheerioSeleniumScrapyBeautifulSoupLearn.Web Scraping AcademyClasses for beginners and experts. Learn about web scraping and automation with our free courses.Visit AcademyCode.Code templatesJavaScript, TypeScript, and Python templates to quick-start your web scraping projects.Get startedConnect.Discord communityGet help from the Apify developer community of more than 11,500 members.Join communityPublish Actors. Get paid.Reach thousands of new customersBuilding and running a SaaS is hard. Building an Actor and selling it on Apify Store is 10x easier. Get users from day one.Learn moreNo upfront costsPublishing your Actor is free of charge—the customers pay for the computing resources. New creators get $500 free platform credits.Rely on Apify infraActors scale automatically as you gain new users. You don’t need to worry about compute, storage, proxies, or authentication.Billing is on usHandling payments, taxes, and invoicing is a painful part of running a SaaS. Apify does all that and sends you a net payout every month.Enterprise-grade solutionSecure and reliable web data extraction provider for any scale.99.95% uptime. SOC2, GDPR, and CCPA compliant.Contact salesLearn moreWe looked at several providers, and Apify was the most complete, reliant solution we found. It was miles ahead of everything else we reviewed.Pranav SinghEngineering Manager at IntercomWe selected Apify because of their vast experience with web data collection to empower our sales team with fresh, unique leads.Filip PopovicCOO at GrouponOur collaboration with Apify proves that advanced IT tools leveraging AI can be the key in detecting infringements of consumer protection legislation.Marie-Paule BenassiConsumer Affairs Director at EURead more customer storiesApify Professional ServicesOur experienced team can help you design, implement, and successfully execute your web scraping project.Learn moreIt's time to run \nyour first Actor.Get startedGet a demo",
        "word_count": 939,
        "has_links": true,
        "has_images": true,
        "semantic_type": "form"
      }
    ],
    "text_metrics": {
      "word_count": 1180,
      "unique_words": 505,
      "sentence_count": 84,
      "paragraph_count": 1,
      "avg_word_length": 5.429661016949153,
      "avg_sentence_length": 14.047619047619047,
      "keywords": {
        "apify": {
          "count": 25,
          "frequency": 0.0211864406779661
        },
        "data": {
          "count": 17,
          "frequency": 0.01440677966101695
        },
        "from": {
          "count": 16,
          "frequency": 0.013559322033898305
        },
        "with": {
          "count": 16,
          "frequency": 0.013559322033898305
        },
        "scraper": {
          "count": 14,
          "frequency": 0.011864406779661017
        },
        "extract": {
          "count": 8,
          "frequency": 0.006779661016949152
        },
        "api,": {
          "count": 8,
          "frequency": 0.006779661016949152
        },
        "instagram": {
          "count": 8,
          "frequency": 0.006779661016949152
        },
        "more": {
          "count": 8,
          "frequency": 0.006779661016949152
        },
        "actors": {
          "count": 7,
          "frequency": 0.005932203389830509
        },
        "integrate": {
          "count": 7,
          "frequency": 0.005932203389830509
        },
        "scraping": {
          "count": 7,
          "frequency": 0.005932203389830509
        },
        "your": {
          "count": 6,
          "frequency": 0.005084745762711864
        },
        "tiktok": {
          "count": 6,
          "frequency": 0.005084745762711864
        },
        "hashtags,": {
          "count": 6,
          "frequency": 0.005084745762711864
        },
        "export": {
          "count": 6,
          "frequency": 0.005084745762711864
        },
        "data,": {
          "count": 6,
          "frequency": 0.005084745762711864
        },
        "schedule": {
          "count": 6,
          "frequency": 0.005084745762711864
        },
        "monitor": {
          "count": 6,
          "frequency": 0.005084745762711864
        },
        "other": {
          "count": 6,
          "frequency": 0.005084745762711864
        }
      },
      "language_metrics": {
        "sentences_per_paragraph": 84,
        "words_per_paragraph": 1180
      }
    },
    "technical_metrics": {
      "status_code": 200,
      "html_size_kb": 425.38
    }
  }
]

---

🛠️ Technical Notes

• Compliance: The crawler respects the target site's robots.txt rules and crawl delay directives.
• Efficiency: URLs are normalized, and redirected destinations are logged to ensure no unnecessary reprocessing occurs.
• Content Hash: The content_hash field allows you to easily implement change tracking and avoid reprocessing unchanged documents in subsequent runs.

---

💬 Support and Contact

We are actively developing and improving this Semantic Web Crawler.

General Support & Feedback

If you encounter a bug, have a feature suggestion, or need help integrating the data:

• Please open an Issue Ticket directly on the Apify platform.

Custom Solutions & Enterprise Use

For large-scale projects, custom integrations, or requirements that need bespoke development, consulting, or guaranteed support for this Actor, please contact our team directly at contact@datafusionnow.com for a custom quote and SLA. Contact LinkedIn