Website Tech Stack Scanner | Website Technology Detector

🔎 Website Tech Stack Scanner – Website Technology Detector & Page Metadata Extractor

The Website Tech Stack Scanner visits a list of website URLs, retrieves each page, and returns a structured record containing the page's title and meta description, a scan timestamp, and a technology profile organised into thirty-one named categories — analytics, CMS, hosting, CDN, payments, CRM, security and more — alongside a flattened all_technologies array and a total_technologies count.

⚠️ Please read before using: current build status

In the current version of this Actor, only part of each record is derived from the page it fetches. The url, scannedAt, metadata.title, metadata.description and error fields are produced from the live HTTP response. The technology category fields, all_technologies and total_technologies currently return a fixed placeholder set rather than technologies detected from the page you scanned. Fingerprint-based detection is not yet wired in. Treat this Actor as a working metadata extractor and a stable output contract for tech-stack data — not as a validated technology detector. If you need verified detection results today, please contact us before building on it.

The rest of this README documents the input schema, the complete output contract and the intended purpose of each field, so you can evaluate whether the shape fits your pipeline. Everything described below reflects what the Actor actually emits.

---

📊 What Data Can You Extract with This Website Tech Stack Scanner?

Each dataset item corresponds to one scanned URL. The fields group into six areas.

Category	Fields	What it contains
Scan identity	url, scannedAt	The URL that was scanned and the UTC ISO timestamp of the scan. Both are live values.
Page metadata	metadata	An object with title and description, parsed from the page's <title> tag and <meta name="description">. Live values.
Marketing and analytics stack	Analytics & Product Analytics, Marketing Automation & Journeys, Email & SMS Marketing, Advertising, Pixels & Retargeting, A/B Testing & Experimentation, On-Site Personalization & CRO Widgets, Tag Management & Event Tracking	Arrays of technology names per category. Placeholder values in the current build.
Commerce and customer systems	E-Commerce Platforms & Storefronts, Payments, Billing & Subscriptions, Shipping, Fulfilment & Logistics, Booking & Appointment Systems, Customer Relationship Management (CRM), Customer Data Platforms (CDP) & Profiles, Live Chat, Chatbots & Support, Reviews, UGC & Social Proof, Search, Recommendation & Discovery, Business & Back-Office Tools (Accounting, HR, etc)	Arrays of technology names per category. Placeholder values in the current build.
Engineering and infrastructure	Content Management Systems (CMS), Website Builders & Page Builders, Web Frameworks & Frontend Libraries, Design Systems & UI Components, Media & Video, Hosting, Cloud & Infrastructure, CDN, Caching & Traffic Routing, DevOps, CI/CD & Observability, Developer Tools, SDKs & Documentation, Databases, Security, Bot Protection & WAF, Consent, Privacy & Cookie Management, Localization, Plugins & Miscellaneous	Arrays of technology names per category. Placeholder values in the current build.
Roll-ups and failures	all_technologies, total_technologies, error	A flat list of every technology across all categories, its count, and an error string when a URL could not be fetched.

The genuinely useful field pairing today is metadata together with error. Running a list of URLs through the Actor gives you a fast, proxy-routed check of which sites respond, what each one calls itself in its <title>, and how it describes itself in its meta description — which is a legitimate use on its own for site inventories, redirect checks and domain audits.

---

🌟 Key Features of the Website Tech Stack Scanner

Feature	Description
📝 Live page metadata extraction	Parses the <title> element and <meta name="description"> content from each fetched page into a metadata object.
🗂️ Thirty-one category output contract	The record shape covers analytics, marketing, commerce, CRM, CMS, hosting, CDN, security, privacy and more as named fields.
📦 Batch URL input	startUrls accepts a list of URLs and each one is scanned in turn, producing one dataset row per site.
↪️ Redirect following	HTTP requests follow redirects, so shortened links and domain-level URLs resolve to their final destination before parsing.
🌐 Automatic residential proxying	Requests are routed through residential proxies configured internally; there is no proxy setting to manage in the input.
🧾 Per-URL error records	A URL that cannot be fetched produces a row with url, scannedAt and an error message rather than stopping the run.
🕒 UTC scan timestamps	scannedAt records the exact moment each URL was processed, in ISO 8601 with microsecond precision.
📊 Roll-up fields	all_technologies flattens every category into one array and total_technologies gives the count, simplifying downstream joins.
🔌 Standard Apify API access	Runs, datasets, schedules and webhooks work exactly as they do for any other Actor on the platform.

---

🚀 Why Choose This Website Tech Stack Scanner?

A stable, well-organised output contract. The thirty-one categories map cleanly onto how technology stacks are actually discussed — separating tag management from analytics, CDN from hosting, and consent tooling from security. If you are designing a database schema or a dashboard for technographic data, this record shape is a reasonable starting point that will not need reworking later.

Genuinely useful metadata extraction. The metadata.title and metadata.description values are parsed from the live page, which makes the Actor immediately usable for domain inventories, SEO audits and checking which URLs in a list are reachable at all.

Honest reporting of current limitations. Rather than presenting placeholder detection output as real results, this README states plainly which fields are live and which are not. That lets you evaluate the Actor on what it does today instead of discovering the gap after building around it.

Failure-tolerant batch behaviour. Unreachable hosts, TLS failures and timeouts are captured per URL in the error field. A single bad domain in a list of hundreds does not interrupt the run or lose the rows already collected.

---

📥 Input

{
  "startUrls": [
    { "url": "https://vuejs.org/" },
    { "url": "https://example.com/" }
  ]
}

🔧 Website Tech Stack Scanner Input Fields

Field	Type	Required	Default	Description
startUrls	array	✅ Yes	— (prefilled with [{ "url": "https://vuejs.org/" }])	URLs to scrape. Each entry is a request object containing a url key; entries without a url are skipped.

startUrls is the only input field. Proxy usage is configured internally and is not exposed as an input option.

💡 Input Examples

Single site scan

{
  "startUrls": [{ "url": "https://vuejs.org/" }]
}

Batch scan of a domain list

{
  "startUrls": [
    { "url": "https://example.com/" },
    { "url": "https://example.org/" },
    { "url": "https://example.net/" }
  ]
}

Scanning specific landing pages rather than homepages

{
  "startUrls": [
    { "url": "https://example.com/pricing" },
    { "url": "https://example.com/checkout" }
  ]
}

---

📤 Output

A record from a real run against https://vuejs.org/:

{
  "url": "https://vuejs.org/",
  "scannedAt": "2026-08-08T16:18:49.079035Z",
  "metadata": {
    "title": "Vue.js - The Progressive JavaScript Framework | Vue.js",
    "description": "Vue.js - The Progressive JavaScript Framework"
  },
  "error": null,
  "Analytics & Product Analytics": ["Fathom"],
  "Marketing Automation & Journeys": null,
  "Email & SMS Marketing": null,
  "Advertising, Pixels & Retargeting": null,
  "A/B Testing & Experimentation": null,
  "Tag Management & Event Tracking": null,
  "Customer Relationship Management (CRM)": null,
  "E-Commerce Platforms & Storefronts": null,
  "Content Management Systems (CMS)": null,
  "Website Builders & Page Builders": ["VitePress"],
  "Web Frameworks & Frontend Libraries": ["Vue.js"],
  "Hosting, Cloud & Infrastructure": ["Netlify"],
  "CDN, Caching & Traffic Routing": ["Netlify"],
  "Plugins & Miscellaneous": ["Vite", "Open Graph"],
  "all_technologies": ["VitePress", "Vue.js", "Netlify", "Fathom", "Vite", "Open Graph"],
  "total_technologies": 6
}

🧾 Website Tech Stack Scanner Output Fields — Core Record

Field	Type	Description
url	string | null	The URL that was scanned. Live value.
scannedAt	string | null	UTC ISO 8601 timestamp of the scan. Live value.
metadata	object | null	Object containing title and description parsed from the page HTML. Live value.
error	string | null	Error message when the URL could not be fetched or parsed; null on success. Live value.
all_technologies	array | null	Flattened list of every technology name across all categories. Placeholder in the current build.
total_technologies	integer | null	Count of entries in all_technologies. Placeholder in the current build.

🧾 Website Tech Stack Scanner Output Fields — Technology Categories

Each of the following fields is either an array of technology names or null when the category is empty. All are placeholder values in the current build.

Field	Type
Analytics & Product Analytics	array | null
Marketing Automation & Journeys	array | null
Email & SMS Marketing	array | null
Advertising, Pixels & Retargeting	array | null
A/B Testing & Experimentation	array | null
On-Site Personalization & CRO Widgets	array | null
Tag Management & Event Tracking	array | null
Search, Recommendation & Discovery	array | null
Reviews, UGC & Social Proof	array | null
Customer Relationship Management (CRM)	array | null
Customer Data Platforms (CDP) & Profiles	array | null
Live Chat, Chatbots & Support	array | null
E-Commerce Platforms & Storefronts	array | null
Payments, Billing & Subscriptions	array | null
Shipping, Fulfilment & Logistics	array | null
Booking & Appointment Systems	array | null
Business & Back-Office Tools (Accounting, HR, etc)	array | null
Content Management Systems (CMS)	array | null
Website Builders & Page Builders	array | null
Web Frameworks & Frontend Libraries	array | null
Design Systems & UI Components	array | null
Media & Video	array | null
Hosting, Cloud & Infrastructure	array | null
CDN, Caching & Traffic Routing	array | null
DevOps, CI/CD & Observability	array | null
Developer Tools, SDKs & Documentation	array | null
Databases	array | null
Security, Bot Protection & WAF	array | null
Consent, Privacy & Cookie Management	array | null
Localization	array | null
Plugins & Miscellaneous	array | null

🧾 Failure Records

When a URL cannot be fetched, a shorter record is written:

{
  "url": "https://unreachable.example/",
  "scannedAt": "2026-08-08T16:19:02.441201Z",
  "error": "Connection timed out"
}

---

💻 How to Use the Website Tech Stack Scanner (Step by Step)

Step 1: Prepare Your URL List

Everything the Actor does starts from startUrls. Each entry is a request object with a url key, which is the standard Apify request-list format — you can paste URLs manually, upload them, or generate the list from another Actor's output. Entries without a url key are skipped silently rather than causing an error, so validate your list before a large run if completeness matters.

Step 2: Decide Which Pages to Scan

The Actor fetches exactly the URL you give it, not the whole site. For a general site profile the homepage is the usual choice, since that is where most global scripts and tags load. If you are interested in a particular part of a site, point the scanner at that page directly — checkout and pricing pages, for example, typically load different third-party code from a homepage.

Step 3: Run the Actor

Start the run from the Apify Console or through the API. Each URL is fetched over HTTP with redirects followed, routed through residential proxies configured internally. There is no proxy setting to manage and no browser to configure, which keeps runs lightweight and quick relative to browser-based scanners.

Step 4: Read the Run Log

The log records any URL that fails, along with the underlying exception message. Because failures are also written into the dataset as error rows, the log is mainly useful for spotting systematic problems — a whole batch failing usually points at a network or blocking issue rather than at the individual domains.

Step 5: Extract the Metadata You Can Rely On

Once the run completes, work first with the live fields. Filter for rows where error is not null to find unreachable domains. Then read metadata.title and metadata.description across the successful rows to build a site inventory — this is genuinely extracted from each page and is directly useful for SEO audits, domain portfolio reviews and checking that redirects land where you expect.

Step 6: Understand the Technology Fields Before Using Them

Do not treat the category arrays, all_technologies or total_technologies as detection results in the current build. They return a fixed placeholder set regardless of the site scanned, as the sample above illustrates. Use them to design and test your downstream schema, dashboards and joins, but do not draw conclusions about any website's actual stack from them.

Step 7: Export or Integrate

Export the dataset as JSON, CSV or Excel, or read it through the API. Because the record shape is stable, a pipeline built against it now will continue to work unchanged if and when real fingerprint-based detection is added to the technology fields.

---

🔌 API Access & Integrations

Run the website technology detector and get the dataset back in one call:

curl -X POST "https://api.apify.com/v2/acts/scrapers-hub~website-tech-stack-scanner/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [
      { "url": "https://vuejs.org/" },
      { "url": "https://example.com/" }
    ]
  }'

The same run from Python using the official client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")

run_input = {
    "startUrls": [
        {"url": "https://vuejs.org/"},
        {"url": "https://example.com/"},
    ]
}

run = client.actor("scrapers-hub/website-tech-stack-scanner").call(run_input=run_input)

for site in client.dataset(run["defaultDatasetId"]).iterate_items():
    if site.get("error"):
        print("failed:", site["url"], site["error"])
    else:
        meta = site.get("metadata") or {}
        print(site["url"], "|", meta.get("title"), "|", site["scannedAt"])

Results can also be routed into Zapier, Make, Google Sheets or Slack, or delivered to your own endpoint through Apify webhooks when a run finishes.

---

💡 Best Use Cases for Website Tech Stack Data

The first three use cases below rely only on the live fields and work with the Actor as it stands today. The remainder describe what the output contract is designed for, and depend on real detection being available.

🗂️ Domain Inventory and Reachability Auditing

Feeding a list of owned or acquired domains into startUrls and reading back url, scannedAt and error gives a fast reachability report. Domains that time out, fail TLS negotiation or return connection errors surface immediately as error rows.

🔍 SEO Title and Meta Description Audits

metadata.title and metadata.description are parsed from each live page, which makes bulk auditing straightforward: find pages with missing or empty descriptions, titles that exceed sensible display lengths, or duplicated metadata across a portfolio.

↪️ Redirect and Migration Verification

Because requests follow redirects, scanning a list of legacy URLs after a site migration and comparing the returned metadata.title against expectations reveals redirects that land on the wrong page or on a generic error template.

🛠️ Schema and Pipeline Prototyping

The thirty-one category fields plus all_technologies and total_technologies give you a complete, stable record shape to build against. Warehouse tables, dashboard components and join logic can all be developed and tested now against the placeholder output.

📈 Technographic Segmentation (Intended Use)

The category structure is designed so that sales and marketing teams could segment prospect lists by the systems a site runs — separating E-Commerce Platforms & Storefronts from Payments, Billing & Subscriptions, or Customer Relationship Management (CRM) from Customer Data Platforms (CDP) & Profiles. This requires real detection to be meaningful.

🏁 Competitive Stack Analysis (Intended Use)

Comparing Web Frameworks & Frontend Libraries, Hosting, Cloud & Infrastructure and CDN, Caching & Traffic Routing across a competitor set is the classic technographic exercise the schema supports. Again, this depends on detection results rather than placeholders.

🔐 Privacy and Consent Tooling Review (Intended Use)

Consent, Privacy & Cookie Management, Advertising, Pixels & Retargeting and Tag Management & Event Tracking are separated deliberately so a compliance reviewer could compare declared consent tooling against the tracking actually loaded. This is a genuine need the schema anticipates.

---

⚙️ Tips for Better Website Scanning Results

• Include the scheme in every URL. Entries should be full URLs with https://. Bare domains may fail to fetch and produce error rows.
• Scan the page you actually care about. The Actor fetches one URL per entry and does not crawl. Checkout, pricing and account pages often differ substantially from a homepage.
• Check the error field first. Filtering on non-null error before any analysis separates genuine results from failed fetches immediately.
• Batch in manageable chunks. Very long URL lists make partial retries harder. Several medium runs are easier to manage and resume than one very large one.
• Do not treat the technology fields as detection output. In the current build they are placeholders. Use them for schema work, not for conclusions about any site.
• Store scannedAt with your results. Technology stacks and page metadata both change, so every record needs its own timestamp to remain interpretable later.

---

🛠️ Troubleshooting

Why does every site return the same technologies? Because technology detection is not yet implemented in this build. The category fields, all_technologies and total_technologies return a fixed placeholder set regardless of the URL scanned. Only url, scannedAt, metadata and error reflect the page you requested.

Why is metadata.description empty for some sites? The description is read from a <meta name="description"> tag. Pages that omit that tag, or that supply the description only via Open Graph or JSON-LD, will return an empty string.

Why did a URL produce an error row? The error field carries the underlying exception message — commonly a connection timeout, DNS failure, TLS error or an HTTP client error. Check the URL is correct and publicly reachable, and confirm it includes the scheme.

Why is metadata.title empty on a page that clearly has a title? Some sites render their <title> only after JavaScript executes. This Actor fetches raw HTML without running a browser, so client-side-rendered titles will not appear in the response body.

Why were some entries in startUrls skipped? Entries without a url key are ignored. Check that every item in the array is an object containing a url field, or that plain strings have been converted to that format.

---

❓ Frequently Asked Questions About Website Tech Stack Scanning

What does the Website Tech Stack Scanner do? It fetches each URL you supply, extracts the page title and meta description, records a scan timestamp, and returns a technology profile record organised into thirty-one categories plus roll-up fields.

Does the technology detection actually work right now? No. In the current build the technology category fields, all_technologies and total_technologies return a fixed placeholder set rather than technologies detected from the scanned page. Only url, scannedAt, metadata and error are live.

Which fields can I rely on today? url, scannedAt, metadata.title, metadata.description and error. These are all derived from the actual HTTP response for each URL.

Why publish the full schema if detection is not implemented? So that the output contract is stable and you can build your downstream schema, dashboards and integrations against it now without rework later.

How many URLs can I scan in one run? startUrls is a plain array with no documented cap. Practical limits come from run duration and the responsiveness of the hosts you are scanning.

Does the Actor crawl a whole website? No. It fetches exactly the URLs you provide, one record per URL. Add each page you want scanned to startUrls explicitly.

Does it run a browser? No. It performs plain HTTP requests and parses the returned HTML, which keeps runs fast but means client-side-rendered content is not visible to it.

Can I configure the proxy? No. Residential proxy usage is configured internally and is not exposed as an input field, so there is nothing to set up.

Does it follow redirects? Yes. Requests follow redirects, so shortened links and domain-level URLs resolve to their final destination before the HTML is parsed.

What happens when a site is unreachable? That URL produces a record containing url, scannedAt and an error message. The run continues with the remaining URLs.

What is the difference between all_technologies and the category fields? The category fields group technologies by function; all_technologies is a flattened list of every entry across all categories, and total_technologies is its count. All three are placeholders in the current build.

Can I export the results? Yes. Datasets export as JSON, JSONL, CSV, Excel, XML or HTML from the Apify Console, and can be read directly through the API.

Can I schedule recurring scans? Yes. Apify's scheduler supports any cron expression, and scannedAt on each record keeps successive snapshots distinguishable.

Is website technology scanning legal? Fetching a publicly accessible page and reading its HTML is ordinary web access. How you use the resulting data is your responsibility, including compliance with each site's terms of service.

When will real technology detection be available? Detection is not implemented in this version. If it is important to your use case, contact us at the address below so we can tell you where it stands before you build on it.

---

🆘 Support & Feedback

If you encounter a bug, a URL that fails unexpectedly, or a parsing problem with page metadata, open a ticket in the Issues tab of this Actor on Apify. Reports filed there are tracked and answered directly.

If you need working technology detection, a different output shape, or a custom scanning pipeline, email scraperhubapi@gmail.com and describe what you need. Given the current build status, we would rather have that conversation before you build on the Actor than after.

If the metadata extraction is useful to you as it stands, a review on the Apify page is appreciated, and feedback about which technology categories matter most to you will help prioritise detection work.

---

⚖️ Disclaimer

This website tech stack scanner fetches only publicly accessible pages at URLs you explicitly supply. It does not log in, does not probe for vulnerabilities, does not attempt to access non-public paths, and does not circumvent authentication or technical access controls. It performs a single ordinary HTTP request per URL.

You are responsible for how you use the extracted data, including compliance with the terms of service and robots policies of the sites you scan. Page titles, descriptions and other content may be protected by copyright held by the site operators, and republishing that material may require permission.

Where a scanned page or its metadata contains personal data, you act as the data controller under the GDPR, the UK GDPR, the CCPA and other applicable privacy legislation, with the corresponding obligations around lawful basis, retention and data-subject rights. Technographic data about businesses should not be used to profile or target identifiable individuals without a proper legal basis.

Please also note the build status described at the top of this document: technology detection results are placeholders in this version and must not be presented, resold or relied upon as factual findings about any website.

For questions about data handling, or to request removal of data associated with your runs, contact scraperhubapi@gmail.com.