⭐ Website Metadata & Social Links Scraper – B2B Enrichment & Domain Intelligence

This Apify Actor is a blazing-fast, all-in-one domain enrichment tool designed to extract social media profiles, SEO metadata, JSON-LD structured data, and technology stacks from any website.

It supports AI-powered Business Classification (powered by lightning-fast Groq models), intelligent deep-link filtering, and multi-page routing to find elusive contact pages. Packaged into a clean, structured dataset, it is built specifically for downstream B2B lead generation, CRM enrichment, and market research workflows.

💲 Pricing Note: This Actor operates on a predictable flat-rate Pay-Per-Event (PPE) model.

• Standard Extraction: $0.004 per domain processed (regardless of how many social links are found).
• AI Enrichment Mode: $0.008 per domain (if the AI successfully classifies the business).
• Example: Scraping 1,000 company websites will cost exactly $4.00 (or $8.00 with AI enabled).

---

🚀 What It Does

For every website URL or domain provided, the actor crawls the homepage and priority subpages (like /about and /contact) to extract a comprehensive business profile.

🏢 Extracted Data

The actor can process and retrieve:

• 🔗 Social Media Links (10+ platforms)
• 🏷️ Brand Identity (Logos & Legal Business Names)
• 🤖 AI Business Classification (Industry, B2B/B2C, Summary)
• ⚙️ Technology Stack (CMS, Frameworks, Analytics)
• 📄 SEO Metadata (Titles, Descriptions, JSON-LD)
• 🗺️ Site Architecture (Robots.txt & Sitemap URLs)

---

🤖 AI Business Categorization (Built-In)

You do not need to provide your own API keys to use the AI features! The Actor has Groq's high-speed LLM infrastructure built directly into it.

To use it, simply enable the AI Classification toggle in the input settings and select your preferred model:

• 🥇 openai/gpt-oss-120b (Most Capable)
• 🥈 openai/gpt-oss-20b (Recommended for speed/accuracy)
• 🥉 qwen/qwen3.6-27b (Best for non-English sites)
• 🚀 groq/compound (Ultra-fast bursts)

---

🏢 Predefined Industries (Standardized B2B List)

In the Actor's input, you can optionally provide a list of Predefined Industries.

• If you leave it blank: The AI will evaluate the website and invent an industry category it thinks fits best.
• If you provide a list: The AI is forced to map the website to one of your exact categories.

To ensure your CRM data remains clean and matches platforms like LinkedIn, Apollo, or Lead411, you can copy and paste categories from this standardized list of 113 B2B Industries into the Actor's input:

Accounting
Aerospace & Defense
Airlines/Aviation
Architecture & Planning
Automotive
Banking
Biotechnology
Broadcast Media
Building Materials
Business Consulting and Services
Chemical Manufacturing
Civil Engineering
Computer Hardware
Computer Networking
Construction
Consumer Electronics
Consumer Goods
Cosmetics
Defense & Space
Design Services
E-Learning
Education Administration Programs
Education Management
Electrical/Electronic Manufacturing
Energy & Utilities
Engineering Services
Entertainment
Environmental Services
Events Services
Facilities Services
Farming, Ranching & Forestry
Financial Services
Food & Beverages
Food Production
Furniture
Government Administration
Graphic Design
Health, Wellness & Fitness
Higher Education
Hospital & Health Care
Hospitality
Human Resources Services
Import & Export
Industrial Automation
Information Services
Information Technology & Services
Insurance
Internet Marketplace Platforms
Investment Banking
Investment Management
IT Services and IT Consulting
Law Enforcement
Law Practice
Legal Services
Leisure, Travel & Tourism
Logistics & Supply Chain
Luxury Goods & Jewelry
Machinery
Management Consulting
Manufacturing & Industrial
Market Research
Marketing & Advertising
Mechanical/Industrial Engineering
Media Production
Medical Device
Mental Health Care
Military and International Affairs
Mining & Metals
Non-Profit Organization Management
Oil & Energy
Online Media
Outsourcing/Offshoring
Package/Freight Delivery
Packaging & Containers
Paper & Forest Products
Performing Arts
Pharmaceuticals
Philanthropy
Photography
Plastics
Primary/Secondary Education
Printing
Professional Training & Coaching
Program Development
Public Policy
Public Relations & Communications
Publishing
Real Estate
Recreational Facilities & Services
Religious Institutions
Renewables & Environment
Research Services
Restaurants
Retail
Security & Investigations
Semiconductors
Social Networking Platforms
Software Development
Sporting Goods
Sports
Staffing & Recruiting
Telecommunications
Textiles
Translation & Localization
Transportation/Trucking/Railroad
Utilities
Venture Capital & Private Equity
Veterinary
Warehousing
Wholesale
Wine & Spirits
Wireless
Writing & Editing

---

📦 Processing Modes

🔹 Fast Extraction (Default)

Provide a list of URLs and the target social platforms. The actor will traverse the domains, parse the HTML, evaluate the DOM context, and extract precise metadata and social profiles purely via code.

🔹 AI Enrichment Mode

Enable AI Classification in the input. The actor will scrape the visible text across the website and pass it to our integrated LLM. It will return a highly accurate, structured JSON object defining the company's exact industry, a 1-sentence executive summary, and whether they are a B2B or B2C entity.

---

🔗 Intelligent Social Link Extraction

The actor goes far beyond simple regex matching. It extracts profiles using:

Supported Platforms

Facebook, Twitter (X), LinkedIn, Instagram, YouTube, TikTok, GitHub, Discord, Reddit, and Pinterest.

Multi-Layered Discovery

• DOM Context Scanning: Looks for social icons inside <header>, <footer>, and elements with social class names.
• JSON-LD sameAs: Explicitly extracts official social links defined in the company's invisible schema.org structured data.
• Raw Regex Fallback: Scans the raw HTML as a final net.

Smart Brand Matching & Deep-Link Rejection

If you scrape apify.com, the scraper intelligently looks for /apify on GitHub, but will actively reject deep repository links like github.com/apify/crawlee. This ensures you get the main corporate profile, not a random sub-page or shared post.

---

🧹 Normalization & Deduplication

Before saving social profiles, the actor sanitizes URLs by:

• Stripping tracking parameters (utm_source, igshid, etc.)
• Forcing secure protocols (https://)
• Removing trailing slashes and www. prefixes
• Automatically removing duplicate links across multiple crawled pages using a strict Set logic.

---

🛡️ Input Validation & Routing

The actor validates and optimizes inputs before processing.

Smart Routing

Instead of wasting compute power crawling random blog posts, the actor explicitly searches for and prioritizes internal links matching /contact, /about, /team, or /support.

Domain Validation

URLs are automatically pre-pended with https:// if no protocol is provided, ensuring seamless connectivity.

---

⚡ It Handles

• ✅ Bulk domain processing
• 🔀 Multi-page contextual routing (Home, About, Contact)
• 🤖 Built-in LLM Integration (No API keys required)
• 🧼 URL normalization and tracking-parameter stripping
• 🧹 Automatic duplicate profile removal
• 🏢 Legal Business Name and Logo extraction via JSON-LD
• ⚙️ Tech stack fingerprinting (Shopify, WordPress, Next.js, etc.)
• 📦 Structured Apify Dataset output with tabular views

---

🧠 How It Works

1. Reads the Actor input (startUrls, platforms, AI configs).
2. Initializes a high-speed CheerioCrawler with proxy support.
3. Requests the domain's robots.txt and locates sitemaps.
4. Scrapes the homepage for metadata, JSON-LD, and tech stack fingerprints.
5. Identifies and prioritizes internal /about and /contact links.
6. Crawls the internal pages (up to maxPagesPerDomain) and aggregates all page text.
7. Extracts social links via Regex, DOM traversal, and Schema.org.
8. Normalizes, filters, and deduplicates all social links.
9. Sends aggregated text to the built-in AI model (if enabled) for business classification.
10. Calculates PPE charges based on successful data extraction.
11. Pushes the rich, structured JSON object to the Apify Dataset.

---

⚙️ Input Configuration

Field	Type	Required	Description
startUrls	Array	Yes	List of website URLs or domains (e.g., apify.com).
platforms	Array	Yes	Select the social media platforms to extract.
maxPagesPerDomain	Integer	Yes	Max pages to crawl per domain (default: 5).
useAiClassification	Boolean	No	Enable LLM to categorize the business industry and summarize it.
aiModel	String	If AI is On	Select the integrated Groq model optimized for speed and accuracy.
predefinedIndustries	Array	No	Specific list of industries for the AI to choose from (Optional).
proxyConfiguration	Object	No	Proxy settings to bypass blocking.

---

🧩 Example Input

{
  "startUrls": ["apify.com", "stripe.com"],
  "platforms": ["linkedin", "twitter", "github", "youtube"],
  "maxPagesPerDomain": 3,
  "useAiClassification": true,
  "aiModel": "openai/gpt-oss-20b",
  "predefinedIndustries": ["SaaS", "Fintech", "E-commerce", "AI"]
}

---

📊 Example Output

A dataset item generated by the Actor follows this structure:

{
  "url": "[https://apify.com](https://apify.com)",
  "domain": "apify.com",
  "brandInfo": {
    "name": "apify",
    "legalName": "Apify Technologies s.r.o.",
    "logo": "[https://apify.com/ext/apify-logo-120px.png](https://apify.com/ext/apify-logo-120px.png)"
  },
  "natureOfBusiness": {
    "industry": "SaaS",
    "summary": "Apify provides a platform to build, run, and scale web scraping and automation tools.",
    "isB2B": true
  },
  "statistics": {
    "pagesScanned": 3,
    "totalProfilesFound": 4
  },
  "metadata": {
    "title": "Apify: The largest marketplace of trusted tools for AI",
    "description": "Thousands of tools to automate your business. Get real-time web data, track competitors, generate leads, and integrate your apps and AI agents.",
    "technologies": []
  },
  "robots": {
    "userAgents": {
      "*": {
        "allow": ["/"],
        "disallow": []
      }
    }
  },
  "sitemaps": [
    "[https://apify.com/sitemap.xml](https://apify.com/sitemap.xml)"
  ],
  "jsonLd": [
    {
      "@context": "[https://schema.org](https://schema.org)",
      "@type": "Organization",
      "name": "Apify",
      "legalName": "Apify Technologies s.r.o.",
      "url": "[https://apify.com/](https://apify.com/)",
      "logo": "[https://apify.com/ext/apify-logo-120px.png](https://apify.com/ext/apify-logo-120px.png)",
      "sameAs": [
        "[https://github.com/apify](https://github.com/apify)",
        "[https://www.linkedin.com/company/apify](https://www.linkedin.com/company/apify)",
        "[https://x.com/apify](https://x.com/apify)",
        "[https://www.youtube.com/apify](https://www.youtube.com/apify)",
        "[https://discord.gg/w3e2v7rWDw](https://discord.gg/w3e2v7rWDw)"
      ],
      "contactPoint": [
        {
          "@type": "ContactPoint",
          "contactType": "customer support",
          "email": "support@apify.com"
        }
      ]
    }
  ],
  "socialProfiles": {
    "twitter": ["[https://x.com/apify](https://x.com/apify)"],
    "linkedin": [
      "[https://linkedin.com/company/apify](https://linkedin.com/company/apify)"
    ],
    "github": ["[https://github.com/apify](https://github.com/apify)"],
    "youtube": ["[https://youtube.com/apify](https://youtube.com/apify)"]
  }
}

---

📈 Output Fields

Field	Description
url	The final executed URL.
domain	The clean target domain string.
brandInfo	Extracted legal name, fuzzy brand name, and corporate logo.
natureOfBusiness	AI-generated industry, B2B/B2C flag, and summary (if enabled).
statistics	Metadata about how many pages were scanned and profiles found.
metadata	Traditional SEO meta tags and detected technology stack.
robots	The parsed robots.txt rules for the domain.
sitemaps	Array of sitemap URLs discovered on the domain.
socialProfiles	Grouped arrays of deduplicated, verified social links.
jsonLd	The raw Schema.org structured data found on the site.

---

⭐ Features

• 🚀 High-Speed CheerioCrawler — Skips heavy browser rendering for blazing-fast execution.
• 🤖 Built-in AI Enrichment — Seamlessly powered by Groq's high-speed infrastructure (no API keys required from users).
• 🎯 Context-Aware Socials — Avoids random social links by checking DOM hierarchy and URL paths.
• 🧹 Strict Deduplication — Merges HTTP/HTTPS and trailing slashes to prevent messy data.
• 🏢 Brand Verification — Automatically detects Logos and Legal Business Names.
• 🕸️ Smart Sub-page Routing — Actively hunts for /contact and /about pages to maximize data yield.
• 💲 Fair PPE Pricing — Predictable, flat-rate pricing per domain processed.

---

💡 Use Cases

• B2B Lead Generation & Scoring
• CRM Database Enrichment (Salesforce, HubSpot)
• Competitive Market Research
• Tech Stack & CMS Prospecting
• Outbound Sales Intelligence
• AI Agent Knowledge Base Population
• Investor Prospecting & Deal Sourcing

---

❓ FAQs

1. Do I need an API key to run this?

No! You do not need to provide any API keys. The core scraper (Socials, Metadata, Tech Stack) works entirely locally, and the AI Business Classification is handled securely behind the scenes. Just flip the toggle on!

2. Does this scrape the entire website?

No. To keep costs low and speeds high, it utilizes maxPagesPerDomain. It scrapes the homepage, then dynamically searches for and scrapes the most relevant internal pages (like /contact).

3. Why are some social links ignored?

The actor uses fuzzy brand matching and path-depth analysis to prevent junk data. For example, if scraping tesla.com, it will extract twitter.com/tesla but will intentionally ignore twitter.com/share?url=... or twitter.com/elonmusk/status/123.

4. How am I charged for this Actor?

This is a Pay-Per-Event (PPE) actor. You are charged a flat rate of $0.004 per domain processed. If you enable AI Business Classification, the charge is doubled to $0.008 per domain. You are not charged for empty results or failed domains.

---

🧑‍💻 Developer Info

Author: codescraper

Email: codescraper011@gmail.com

---

🏷️ Tags

social-links-scraper · website-metadata · domain-enrichment · social-media-scraper · b2b-leads · lead-generation · company-intelligence · tech-stack-detector · json-ld · seo-scraper · groq · business-classification · apify · automation

---

⚠️ Responsible Use

Use this scraper responsibly and in accordance with applicable privacy laws (such as GDPR and CCPA) and the Terms of Service of the targeted websites. This tool is designed for extracting publicly available corporate metadata and business presence profiles, not for scraping protected personal user data.