Company Website Profiler - Enrichment & Tech Stack

Company profile from a domain. Paste a list of company websites and get one clean row per company: name, legal name, logo, description, public contact details, social links, technology stack, sitemap size, SEO and security posture

• 51 columns plus an 18-field SEO block, each labelled with where it came from.

$4 per 1,000 domains, all-in. Nothing extra for compute. $0 for any domain that could not be read - blocked, parked, dead or robots-disallowed rows come back marked ok: false with a plain-English reason, and are never billed.

input   ["stripe.com", "apify.com", "joespizza.example"]
output  one row per domain, every value traceable to its source

---

What you get for each company website

Fill rates are measured, not estimated. Two runs on 2026-09-02, after the latest fixes: 52 large brands and institutions and 18 smaller businesses, agencies and retailers. The raw run is in tests/scale-test-2026-09-02b.txt in the source.

Field	What it is	Filled: big brands	Filled: smaller businesses
fieldSources	where every value above came from	100%	100%
completeness	0-100 score for the whole row	100%	100%
companyName	the company's name	100%	100%
legalName	registered entity, when published	15%	7%
description	the company's own description of itself	89%	100%
logo, favicon	absolute image URLs	91%	93%
email, emails	public business addresses, role desks first	15%	29%
phone, phones	published numbers, de-duplicated	15%	36%
address, country	postal address from structured data	19%	21%
socials	LinkedIn, X, GitHub, YouTube, Facebook, Instagram, TikTok, Discord, Reddit, Mastodon, Bluesky, Crunchbase	81%	93%
techHints	CMS, framework, CDN, analytics, chat, payments	87%	93%
sitemapUrlCount	pages the site publishes, with sitemapChildCount, sitemapLastModified and sitemapCountCapped	70%	79%
seo	title, meta description, canonical, OpenGraph, structured-data types, hreflang, heading counts, images missing alt text, internal vs external links	100%	100%
structured data	schema.org types found on the page	62%	79%
securityTxt, securityContact	RFC 9116 contact	60%	14%
llmsTxt	the emerging AI-readers convention	51%	29%
feeds, feedCount	RSS or Atom feeds that actually resolve	19%	21%
https, securityHeaders, securityHeaderScore	transport and six security headers scored	100%	100%
language, organizationType, foundingDate	from html lang and schema.org	varies	varies
redirectedTo	set when the domain redirects to another host	-	-
performance	Core Web Vitals and Lighthouse (optional, free Google key)	off by default	off by default

emails and phones keep every value found; email and phone are the best single pick. Rows that could not be read still carry all 51 columns, so your spreadsheet never changes shape.

Why the source of every field matters

A company name read from a schema.org Organization block is a fact the company published. A company name guessed from a page title is a guess. Most tools hand you both and let you find out the hard way. This one labels them:

"companyName": "Apify",
"fieldSources": {
  "companyName": "json-ld",
  "description": "json-ld",
  "email": "json-ld",
  "address": "json-ld",
  "country": "json-ld",
  "language": "html lang",
  "socials": "json-ld sameAs + site links",
  "techHints": "markup + headers",
  "securityContact": "security.txt"
}

Sort by fieldSources.companyName and every json-ld row is ready to use. The rows labelled title (least reliable) are the ones worth a human glance.

You are never charged for an empty row. A domain that is parked, dead, blocked or asks robots not to be read still comes back as a row so your list keeps its shape, with ok: false and a reason. It costs nothing. The run summary prints how many rows were actually charged for.

How to profile a list of domains

1. Click Try for free at the top of this page.
2. Paste your domains into Domains or URLs, one per line. stripe.com and https://stripe.com/pricing both work; duplicates are removed.
3. Press Start.
4. Open the Output tab and download CSV, JSON or Excel, or pull the data from the API (below).

A 50-domain list finishes in well under a minute at the default settings: the measured run above did 52 domains in 41 seconds at concurrency 8, roughly 3.8 seconds per domain.

Pricing: $4 per 1,000 domains, nothing for failures

$0.004 per domain profiled. Pay per event.

Domains profiled	Cost
1,000	$4
10,000	$40
100,000	$400

• Only ok: true rows are billed. Blocked, dead, parked and robots-disallowed domains are free.
• No separate compute bill. You are not invoiced for the platform usage this run consumes on top of the price above.
• The run log ends with a charged: line so you can check the number yourself.

Example output

A real row, from a run on 2026-09-02 (trimmed for length - the full row has 51 columns):

{
  "domain": "apify.com",
  "url": "https://apify.com/",
  "ok": true,
  "companyName": "Apify",
  "legalName": "Apify Technologies s.r.o.",
  "description": "Thousands of tools to automate your business...",
  "email": "support@apify.com",
  "address": { "locality": "Prague", "postalCode": "11000", "country": "CZ" },
  "country": "CZ",
  "socials": {
    "github": "https://github.com/apify",
    "linkedin": "https://www.linkedin.com/company/apify",
    "twitter": "https://x.com/apify",
    "tiktok": "https://www.tiktok.com/@apifytech",
    "discord": "https://discord.com/invite/jyEM2PRvMU"
  },
  "socialCount": 5,
  "techHints": ["Next.js", "HubSpot", "Google Tag Manager", "Intercom", "AWS CloudFront"],
  "sitemapFound": true,
  "sitemapUrlCount": 200000,
  "sitemapChildCount": 13,
  "sitemapCountCapped": true,
  "securityTxt": true,
  "llmsTxt": true,
  "https": true,
  "securityHeaderScore": 4,
  "fieldSources": { "companyName": "json-ld", "email": "json-ld", "address": "json-ld" },
  "completeness": 92
}

Every row carries completeness, 0 to 100, so you can sort a delivery instead of eyeballing it. Real scores from the same run:

Domain	Completeness	What was found
apify.com	92%	legal name, Prague address, support@ desk, 5 socials, 5 technologies
kingarthurbaking.com	83%	name from schema.org, toll-free number, 5 socials, 7,591-page sitemap
zingermans.com	79%	service@ desk chosen over bizgifts@, phone, 2 socials
example.com	29%	a stub page, and the score says so

Company enrichment use cases

• Enriching a lead list. Domains from a form, a conference or a CRM export become names, descriptions, technology reads and - where the business publishes them - a contact desk.
• Qualifying prospects. The technology list tells you whether they are a fit, and completeness tells you which rows need a human.
• Site audit and portfolio hygiene. Run your own domains and see which are missing structured data, security headers, HTTPS or a sitemap.
• Feeding an AI agent. One clean JSON row per company, with sources, so a model is not guessing from raw HTML.

Run it from code or your automation tool

JavaScript (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('abusch500/site-intel-company-profile').call({
    domains: ['stripe.com', 'apify.com'],
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("abusch500/site-intel-company-profile").call(
    run_input={"domains": ["stripe.com", "apify.com"]}
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

cURL - run and get the rows back in one call:

curl -X POST "https://api.apify.com/v2/acts/abusch500~site-intel-company-profile/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains": ["stripe.com", "apify.com"]}'

This actor is a standard Apify Actor, so the platform's own integrations are available on the tabs above: n8n, Make, Zapier, MCP for AI agents, webhooks, and scheduling for a recurring refresh. See the Integrations and API tabs of this actor for the exact wiring.

Input

Field	Type	Default	Notes
domains	array	required	"stripe.com" or "https://stripe.com/" both work
maxConcurrency	integer	5	1 to 20
requestTimeoutSecs	integer	20	per request
respectRobotsTxt	boolean	true	leave it on unless you own the domains
includePerformance	boolean	false	adds Core Web Vitals
pageSpeedApiKey	string	empty	free Google key, no billing details needed

{
  "domains": ["stripe.com", "apify.com"],
  "maxConcurrency": 5,
  "includePerformance": false
}

Where the data comes from

Only from places built to be read by machines:

Source	What it is
schema.org JSON-LD	structured data published for machines
OpenGraph and Twitter cards	published so other sites can display a preview
standard <meta> tags	the page's own description of itself
/robots.txt	the site's crawling policy, read first and obeyed
/sitemap.xml	published so crawlers can find pages
/.well-known/security.txt	RFC 9116 public security contact
/llms.txt, /humans.txt, /ads.txt	published conventions
RSS and Atom feeds	published for syndication
HTTP response headers	sent to every visitor
Google PageSpeed Insights	Google's own free public API

Nothing behind a login. Nothing behind a paywall.

On robots.txt. It is fetched before anything else. Allow and Disallow rules are obeyed, including a group naming SiteIntelBot, and a site with a blanket disallow is skipped and not charged for. Crawl-delay is read and reported in the row as robotsTxt.crawlDelay, but not yet applied as a wait. The respectRobotsTxt switch exists for running this against domains you own.

On personal data. This reads only contact routes a business publishes on its own homepage - a mailto: link, a tel: link, a schema.org email. There is no people search, no email-pattern guessing and no personal profiles: linkedin.com/in/<person> URLs are deliberately rejected, and only /company/ and /school/ pages are recorded. Where a company publishes a role desk (info@, support@, sales@) it is preferred over anything else found on the page.

For site owners

This actor identifies itself as SiteIntelBot. To opt out, add this to your robots.txt:

User-agent: SiteIntelBot
Disallow: /

It reads your homepage once, plus the standard published files (robots.txt, sitemap.xml and its children, security.txt, llms.txt, humans.txt, ads.txt and any declared feeds). Measured on 2026-09-02: 18 to 34 requests per domain per run, and never more than about 40 even for a large sitemap index. Requests carry a contactable User-Agent pointing at this page.

FAQ

Why is email or phone empty for many domains? Because most companies do not publish one on the homepage. Measured: 15% of large brands and 29% of smaller businesses publish an email; 15% and 36% a phone. This is a live read of what the site actually says, not a database of guessed addresses, so an empty field means the site did not publish it.

Why was a domain skipped, and was I charged? You were not charged. A skipped row has ok: false and an error saying why: HTTP 403, fetch failed, robots.txt disallows automated access. Only ok: true rows are billed.

How many domains get skipped? In the two measured runs: 5 of 52 large brands (3 of those blocked with HTTP 403 or 429) and 4 of 18 smaller businesses (3 blocked). Roughly one commercial domain in ten refuses datacenter traffic outright, whatever is asking. Those rows are free.

Are there compute or platform charges on top of the price? No. The price per profiled domain is the whole cost of the run to you.

Is this legal? Is it GDPR-safe? It reads only what a site publishes for machines to read - structured data, meta tags and standard files at fixed public paths - and it obeys robots.txt. Nothing behind a login or a paywall. It does not collect personal profiles or guess at individuals' addresses. You are still responsible for how you use business contact details in your own jurisdiction.

How is this different from a contact finder? A contact finder crawls many pages of a site hunting for addresses. This reads one page and the standard files, and returns a whole profile - identity, tech, structure, SEO, security - with a source label on each value. If contacts are the only thing you need, a dedicated contact finder will find more of them.

How is it different from an enrichment database? A database returns what it stored, whenever it stored it. This fetches the site now, and tells you which field came from where. Nothing is cached between runs.

How fresh is the data? It is fetched during your run. fetchedAt on every row is the timestamp.

How fast is it, and how big a list can I send? About 4 seconds per domain, run in parallel. 52 domains took 41 seconds at concurrency 8. Lists are limited only by your own run timeout; memory stays flat regardless of list size.

What does completeness mean? A weighted 0-100 score of how much of the profile actually filled in. Rows in the nineties are ready to use, rows in the thirties need a human. Sort by it.

What does sitemapCountCapped mean? A sitemap index is followed one level, up to 25 child sitemaps, and counting stops at 200,000 URLs. When either limit is reached the flag is true and the count is a floor, not an exact figure. sitemapChildCount tells you how many child sitemaps the index actually declared.

Why is performance null? Core Web Vitals are off by default because they need a free Google PageSpeed API key and add 10-25 seconds per domain. Turn on includePerformance and paste a key to fill it.

Can I turn robots.txt off for my own sites? Yes - respectRobotsTxt: false. It is there for domains you own and control. Leave it on for everything else.

How do I block this bot? See For site owners above. User-agent: SiteIntelBot / Disallow: /.

Does it work on non-English sites? Yes. The name and description are whatever the site publishes, in whatever language it publishes them, and language reports the site's own declared language.

Notes and limits

• Roughly one commercial domain in ten answers HTTP 403 to any datacenter IP. Measured: 3 of 52 large brands, 3 of 18 smaller businesses. It is a web-application firewall rule, not robots.txt, and it is not something this actor tries to defeat. Those rows come back marked and unbilled.
• Technology detection is hints, not certainty. It reads what the page actually loads - script, stylesheet, image and iframe URLs, inline scripts, the generator tag and response headers. A site behind a proxy may hide what it runs. A page that merely mentions a platform in its copy is deliberately not counted.
• Sitemap counts on very large sites are a floor. See sitemapCountCapped above.
• Performance data needs a free key from the Google Cloud Console, no billing details. Without it every other field still works.
• Field data in the performance block is often absent. Google only reports real-user measurements for origins above a traffic threshold. The row says so rather than leaving you guessing.

Support and changelog

Found a wrong value? Open an issue on the Issues tab with the domain. Wrong data gets fixed, and the fix ships to everyone. Issues are read daily.

0.2 - 2026-09-02

• Company names cleaned up: Welcome to ... greetings, Home - Brand titles and Brand -- Tagline separators are handled; PWA labels are no longer read as company names.
• Technology detection now tests only what a page actually loads, so an agency that writes about Squarespace and Webflow is no longer reported as running them. Duplicate spellings (Joomla! / Joomla) merged.
• Social links: platform utility pages (/login, /features, /policies) rejected, and personal linkedin.com/in/ profiles are no longer recorded.
• email now prefers a published role desk over a personal address.
• Duplicate phone numbers written two ways are merged.
• Sitemap indexes are followed 25 children deep instead of 5, fetched in parallel, with relative URLs resolved; sitemapChildCount and sitemapLastModified added.
• Every row - including failures - now carries all 51 columns, so CSV columns never move.
• fieldSources extended to legalName, foundingDate, organizationType, country, language, techHints, securityContact, server, poweredBy.

0.1 - 2026-09-02 - first public release.