# Website Technology Scraper - Tech Stack, CMS & Analytics Lookup (`topapi/website-technology-scraper`) Actor Detect any website's technology stack for $1.90 per 1,000 sites: CMS, ecommerce, analytics, ads, tag managers, frameworks, CDN, hosting, payments and email services, with versions, confidence and evidence. 7,600+ fingerprints, real browser render, DNS and TLS signals, change monitor. - **URL**: https://apify.com/topapi/website-technology-scraper.md - **Developed by:** [Top Api](https://apify.com/topapi) (community) - **Categories:** Developer tools, Automation, SEO tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.90 / 1,000 website analyzeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ### What does Website Technology Scraper do? **Website Technology Scraper** detects the technology stack of any website. Paste a list of domains and get one row per website with every technology found: CMS, ecommerce platform, analytics, advertising pixels, tag managers, JavaScript frameworks, CDN, hosting, payment processors, live chat, cookie consent and email services, each with its category, confidence, version where the site exposes one, and the evidence that matched. Built for sales and marketing teams, agencies, investors and researchers who need a tech stack lookup in bulk without checking sites one by one. Tech stack lookup - website technology checker - BuiltWith alternative - Wappalyzer alternative - what CMS is this site using Website technology lookup: a homepage that Website Technology Scraper turns into a technology stack row - **7,600+ technology fingerprints:** the open Wappalyzer-format fingerprint set plus DNS-based checks, across 109 categories - **Live detection:** every website is fetched when you run the actor, so the stack is today's, not a cached crawl - **Versions, confidence and evidence:** each technology says how it was detected (header, script URL, page markup, cookie, global variable, DNS record type) and what matched - **DNS and TLS signals:** email, DNS and SaaS services that only show up in a domain's DNS records (Google Workspace, Microsoft 365, SendGrid, Zendesk, HubSpot, Salesforce...), plus the TLS certificate authority - **Headless Chrome when a page needs it:** pages built by JavaScript are rendered in a real browser to see the scripts and tags they load - **Change monitor:** schedule it and get only the websites whose stack changed, with the technologies added and removed and the versions that changed - **No login, cookies or API key:** you pass domains, nothing else - **Export anywhere:** JSON, CSV, Excel, XML; schedules, webhooks, Make, Zapier, n8n, Google Sheets; Apify API and MCP ### What data can you get from a website?
🧩 Technologies (`technologies`) 🏷️ Categories (`categories`) πŸ”’ Versions (`version`)
🎯 Confidence 0-100 (`confidence`) πŸ” Evidence per detection (`evidence`) ⭐ Primary stack (`primaryStack`)
πŸ“ CMS and site builder πŸ›’ Ecommerce platform πŸ“Š Analytics and tag managers
πŸ“£ Advertising and retargeting pixels βš›οΈ JavaScript and web frameworks ☁️ Hosting, PaaS and CDN
βœ‰οΈ Email and SaaS services from DNS (`dnsDetected`) πŸ”’ TLS issuer and expiry (`tls`) πŸ›‘οΈ Security headers (`securityHeaders`)
πŸ–₯️ Server and X-Powered-By 🌐 Third-party domains loaded πŸ” Added / removed since the last run
On a 100-domain test run of B2B company websites, the 89 websites that answered had **32 technologies on average** (median 34, range 11-46) across 22 categories; 19 of them on average came from DNS records. ### How to look up a website's technology stack with Website Technology Scraper 1. [Create a free Apify account](https://console.apify.com/sign-up) - it comes with monthly free credit. 2. Open [Website Technology Scraper](https://apify.com/topapi/website-technology-scraper) and click **Try for free**. 3. Paste your domains into **Websites**, one per line (`hubspot.com`, `www.example.com` or a full URL), or link a text file under **Website URLs (or a file)**. 4. Leave **Browser render** on Auto, or set it to Never for the fastest HTTP + DNS lookup. 5. Click **Start**. Rows appear in the dataset as websites finish. 6. Open **Storage** and **Export** as JSON, CSV or Excel - or schedule the actor and turn on Monitor mode. ### ⬇️ Input | Parameter | Type | Description | |---|---|---| | **πŸ” What to analyze** | | | | `domains` | List | Domains or URLs, e.g. `shopify.com`, `www.example.com`, `https://example.com/pricing`. A domain is analyzed at its homepage. An empty list (and no `startUrls`) runs the 3-site demo: shopify.com, wordpress.org, notion.so. A list that gives no website (a file link that fails, an empty file, no valid domain) ends the run as FAILED with nothing charged. | | `startUrls` | URL list | Same as `domains` in URL form, or a link to a text/CSV file with one domain per line. | | **🎯 Detection** | | | | `renderJavaScript` | Select | `auto` renders JavaScript-built pages in headless Chrome, `always` renders every page (slower), `never` is HTTP + DNS only. On long lists of heavy pages a cost limit can skip some renders; those rows say `renderedWith: http`. Default: `auto` | | `maxPagesPerDomain` | Integer | Homepage plus up to N-1 same-site pages (pricing, checkout, blog, careers). Contact pages are never picked, in English or other common languages (Kontakt, Contactez-nous, Contacto, Contatti...), and neither are imprint (Impressum, mentions lΓ©gales) or directions (Anfahrt) pages. Default: `1` | | `includeDns` | Boolean | DNS records (MX, TXT, NS, SOA, CNAME) as detection signals. Default: `true` | | `includeTls` | Boolean | TLS certificate issuer and validity. Default: `true` | | `renderWaitSecs` | Integer | Seconds the browser waits after load for tag managers. Default: `3` | | **βœ‚οΈ Refine results** | | | | `categories` | List | Keep only these categories, e.g. `CMS`, `Analytics`, `Ecommerce`, `Tag managers`. | | `onlyTechnologies` | List | "Does it use X?" mode: rows get `matches` and `matchedTechnologies`, e.g. `Shopify`, `HubSpot`. | | `onlyMatchingSites` | Boolean | With `onlyTechnologies`: return only websites that use one of them. Default: `false` | | `minConfidence` | Integer | Drop detections under this confidence (0-100). Default: `0` | | **βž• Extra data** | | | | `includeEvidence` | Boolean | The signals behind each detection. Default: `true` | | `includeSecurityHeaders` | Boolean | HSTS, CSP, X-Frame-Options and 6 more headers. Default: `true` | | `includeRawSignals` | Boolean | Script URLs, XHR hosts, response headers and cookie names. Store ids (such as a Shopify shop id) are removed. Default: `false` | | **πŸ” Monitoring** | | | | `monitorMode` | Boolean | Return only websites whose stack changed since the last run: technologies added or removed, or a version changed. `categories` and `minConfidence` only choose which changes a row shows: each run is compared on the whole stack, so changing or clearing a filter between runs is never a change. Default: `false` | | `monitorStoreName` | String | One name per list you monitor. Default: `website-technology-monitor` | | **πŸ“¦ Output** | | | | `maxResults` | Integer | Stop after this many websites; `0` = the whole list. Default: `0` | | `compact` | Boolean | Core fields only, sized for AI agents and MCP. Default: `false` | | `compareMode` | Boolean | Save a `COMPARISON` record: technologies common to all websites and unique to each. | | `includeFailedSites` | Boolean | Free rows for websites that could not be analyzed, so every valid, distinct website gets a row (invalid entries and duplicates are skipped; the log and `RUN_SUMMARY.inputsSkipped` count them). Default: `true` | | **βš™οΈ Advanced** | | | | `proxyConfiguration` | Proxy | Default: direct first, Apify proxy for hosts that refuse. | | `maxConcurrency` | Integer | Websites analyzed in parallel (1-50), capped at 8 per GB of run memory (16 at the default 2 GB). Default: `10` | | `requestTimeoutSecs` | Integer | Per-request time limit. Default: `20` | | `respectRobotsTxt` | Boolean | Follow the robots.txt rules for all user agents (`User-agent: *`). A website whose address, or the page it redirects to, is disallowed gets a free `robotsDisallowed` row. Extra pages that robots.txt disallows are never fetched or billed, and the next allowed page is taken instead; an extra page that redirects to a disallowed address is dropped and not billed. A missing or unreadable robots.txt allows every page. `RUN_SUMMARY` counts the skipped extra pages (`robotsSkippedPages`). Default: `false` | | `ignoreHttpsErrors` | Boolean | Analyze websites with invalid certificates. Default: `false` | The actor also accepts the input names other technology lookups use (`startDomains`, `urls`, `websites`, `url`, `maxItems`, `maxUrls`, `render_js`, `max_pages_per_site`, `deep_scan`, `categories_filter`, `verify_ssl`, `enableDns`, `enableTls`, `enableSecurityScore`, `include_details`, `include_scripts`, `respect_robots_txt`, `timeout_seconds`), so an existing integration keeps working. Such a name is used while the matching field above is left at its default; `render_js: true` means Always (`false` means Auto) and `deep_scan: true` means 5 pages. The run log and `RUN_SUMMARY.inputAliases` list the names used. #### Input JSON ```json { "domains": ["shopify.com", "wordpress.org", "notion.so"], "renderJavaScript": "auto", "maxPagesPerDomain": 1, "includeDns": true } ``` #### Pass domains, not deep links The homepage carries the site-wide tags (tag manager, analytics, consent, chat). **Do:** `hubspot.com` Β· **Don't:** `hubspot.com/products/crm/some-landing-page` unless that page is what you want to check. Use **Pages per website** to add pricing, checkout and blog pages automatically. #### Pick the render mode by list size **Auto** renders only pages that need JavaScript and is the best default. **Never** is the fastest for very large lists (HTTP, DNS and TLS still run). **Always** renders every homepage in Chrome: on a 10-site test it found 51 technologies per website instead of 32, at roughly 12-16 seconds per site. A render costs more compute than a plain lookup, so on long lists of heavy pages a cost limit can skip some renders; those rows say `renderedWith: http` (two runs of the 10-site test rendered 9 and 10 of the 10 websites). ### ⬆️ Output Each website becomes one dataset item. Here is the table view of a 100-domain run: Website Technology Scraper output: technology stack per website A row, shortened (real data from the test run): ```json { "input": "bigcommerce.com", "domain": "bigcommerce.com", "finalUrl": "https://www.bigcommerce.com/", "status": "ok", "title": "Commerce built for momentum. | BigCommerce", "renderedWith": "http", "techCount": 46, "technologyNames": ["BigCommerce", "Next.js", "Vercel", "Cloudflare", "Tailwind CSS", "HubSpot", "Drift", "..."], "primaryStack": { "ecommerce": "BigCommerce", "webFramework": "Next.js", "hosting": "Vercel", "cdn": "Cloudflare", "uiFramework": "Tailwind CSS", "marketingAutomation": "HubSpot" }, "technologies": [{ "name": "Next.js", "version": null, "confidence": 100, "categories": [{ "id": 12, "slug": "javascript-frameworks", "name": "JavaScript frameworks" }], "tag": "javascript-frameworks", "website": "https://nextjs.org", "detectedVia": ["headers"], "evidence": [{ "type": "headers", "key": "x-powered-by", "match": "x-powered-by: Next.js" }] }], "tls": { "issuer": "DigiCert Inc", "validTo": "2027-02-26T23:59:59.000Z" }, "analyzedAt": "2026-10-08T13:11:10.649Z" } ``` #### Fields | Field | Type | What it is | Fill on test run | |---|---|---|---| | `domain`, `finalUrl`, `statusCode` | string / int | Website, URL after redirects, HTTP status | 100% | | `technologies` | array | name, slug, version, confidence, categories, tag, website, link, description, icon, iconUrl, cpe, saas, oss, pricing, detectedVia, evidence | 100% | | `technologyNames`, `categories` | array | Flat lists of names | 100% | | `techs` | array | Same list in the `name, tag, categories, link` shape other lookups use | 100% | | `primaryStack` | object | Leading technology per group (cms, ecommerce, analytics, tagManager, hosting, cdn, crm, paymentProcessor...) | 100% (cms 60%, analytics 46%, hosting 93%) | | `byCategory` | object | Technology names grouped by category | 100% | | `dnsDetected` | array | Technologies found only through DNS | 100% | | `title`, `metaDescription` | string | Page title and meta description | 100%, 97% | | `metaGenerator` | string | The page's generator meta tag | 17% | | `server`, `poweredBy` | string | Server and X-Powered-By headers | 85%, 26% | | `tls` | object | issuer, issuerCn, protocol, validFrom, validTo | 100% | | `securityHeaders`, `securityHeadersPresent` | object, int | 9 security headers (null when absent) and how many are set | 100% (3.2 set on average) | | `thirdPartyDomains` | array | Other domains the homepage loads scripts, stylesheets, images, media, frames or XHR from (http and https addresses only) | 99% | | `pagesAnalyzed`, `pageCount` | array, int | Pages analyzed for the website. With **Compact output**, rows with more than one page keep `pageCount` and the page URLs | 100% | | `matches`, `matchedTechnologies` | boolean, array | With `onlyTechnologies` | when used | | `isNew`, `added`, `removed`, `versionChanged`, `previousTechCount`, `previousAnalyzedAt` | | Monitor mode (see the FAQ) | when used | | `version` (in `technologies`) | string | Version, when the site exposes it | 1% of detections over HTTP, 4% with the browser | ### How can I use website technology data? - 🎯 **Sales teams** - find companies on a given platform (Shopify, HubSpot, Salesforce, Zendesk) and target accounts by the tools they already pay for - 🧭 **Agencies** - check a prospect's CMS, analytics and ad pixels before a pitch or an audit - πŸ“ˆ **SaaS marketers** - size the install base of a competitor and track sites that add or drop it - πŸ’Ό **Investors and analysts** - map technology adoption across a portfolio or a market - πŸ”’ **Security and IT teams** - list TLS issuers, certificate expiry and security headers across many domains - πŸ€– **AI agents and data teams** - enrich account records with a structured stack through the API or MCP ### How much does it cost to look up website technologies? Website Technology Scraper is pay-per-result: **$1.90 per 1,000 websites** ($0.0019 per website), no start fee. Extra same-site pages (when **Pages per website** is above 1) are **$1.00 per 1,000 pages** ($0.001 each). Websites that could not be analyzed (blocked, unreachable, no technology found) come back as free rows. | Websites | Cost | |---:|---:| | 10 | $0.019 | | 100 | $0.19 | | 1,000 | $1.90 | | 10,000 | $19.00 | Apify platform usage is included. On the test runs, 100 B2B domains took 86-138 seconds and the 3-site demo about 5 seconds. ### Integrations, API, scheduling & MCP - **Schedules:** run it daily or weekly from **Schedules**; with Monitor mode on, each run returns only the websites whose stack changed. - **Webhooks:** call a URL when a run finishes to push rows to a CRM, a sheet or Slack. - **No-code:** Make, Zapier, n8n, Google Sheets, Airtable and Slack via [Apify integrations](https://docs.apify.com/integrations). - **API:** run it from Node.js or Python with `apify-client` - code on the [API tab](https://apify.com/topapi/website-technology-scraper/api/python) and in the [Apify API reference](https://docs.apify.com/api/v2). - **MCP / AI agents:** connect Claude, Cursor or any MCP client through the [MCP tab](https://apify.com/topapi/website-technology-scraper/api/mcp) and ask what a website is built with. ### More data tools by TopApi
πŸ“„ Website Content Crawler πŸ“ˆ Similarweb Scraper πŸ”Ž Bing Search Scraper
⭐ Trustpilot Reviews Scraper πŸ“ Google Maps Reviews Scraper πŸ—ΊοΈ Apple Maps Scraper
πŸ“’ LinkedIn Ad Library Scraper 🎬 TikTok Ads Library Scraper ℒ️ USPTO Trademark Scraper
🍽️ Yelp Reviews Scraper 🧭 All TopApi actors
### Limits & empty runs - **Bot protection:** some websites answer automated visits with a bot challenge (Cloudflare, Akamai, Imperva, DataDome). The row comes back free with `status: blocked` and names only what answered (the bot protection, CDN or host) as technologies; the challenge is not solved. On the 100-domain test, 9 websites did this from Apify's datacenter IPs and 2 did not answer - set **Proxy** to RESIDENTIAL for such lists. - **Single-page apps and consent walls:** technologies loaded only after a cookie consent click, a login or a user action are not seen. - **Versions:** most marketing sites hide library versions; a version appears only when the site exposes it. - **Speed:** HTTP + DNS lookups run 10 websites in parallel (about 1 website per second on 2 GB); a browser render takes about 12-16 seconds on a heavy homepage (a light one, such as notion.so, about 6). - **Free rows, failed runs and empty datasets:** a website that could not be analyzed comes back as a free row whose `status` (`blocked`, `unreachable`, `notFound`, `notHtml`, `error`, `noTechnologies`, `robotsDisallowed`) and `error` say why. If no website at all could be analyzed (every one blocked, unreachable, not found, not HTML or answering with an error), the run ends as FAILED after writing those free rows; **Monitor mode** and the filters (**Only websites that match**, **Only these categories**, **Minimum confidence**) never fail a run this way. If the input gives no website at all (a list file that cannot be read or holds no domain, or no valid domain or URL), the run ends as FAILED in every mode, without rows or charges, and the status message says why. The dataset is empty without a failure when a **Monitor mode** run finds no website changed (unchanged websites are not written), when **Only websites that match** finds no website using the technologies, or when the maximum charge is below the price of one website. With **Rows for failed websites** switched off, failed websites leave no row at all. The last line of the run log and the `RUN_SUMMARY` record count the websites not delivered per reason. ### FAQ #### How many websites can Website Technology Scraper check in one run? As many as you paste or link in a file. The 100-domain test took 86-138 seconds; Max websites (`maxResults`) stops a run early, and Apify's run timeout (one hour by default) can be raised for very large lists. #### Can I schedule Website Technology Scraper to track technology changes? Yes. Save your list as a task, add a schedule, and turn on **Monitor mode**. The first run returns every website; later runs return only websites whose stack changed, with `added`, `removed` and `versionChanged`. A version change alone counts as a change when the version saved last time is no longer detected and the new one was not detected last time (a page that loads two copies of a library shows either version, and that is not reported). Each run is compared only with the checks that can see a technology: - A technology is **removed** when every check that found it ran and missed it on 2 runs in a row (3 renders in a row for a technology only the browser sees, because a script can set its marker too late for one render). Until then it stays in the row, with a `previousRun` evidence entry. - A technology only the browser sees is **added** once 2 renders in a row find it. A technology that a check finds the first time that check runs for the website (for example the first browser render of a page checked over HTTP so far) is recorded, not reported. - When a run skips a check, the technologies that check found are kept from the last run that did it (`previousRun` evidence). This happens when the page as served is unchanged since the last render (the browser render is skipped and the row says `renderedWith: http`), when the cost limit skips a render, or when a DNS answer comes back incomplete. - In Auto mode, a website the browser rendered once is rendered again whenever its page changes, so every run compares the same checks. - **Only these categories** and **Minimum confidence** do not change what counts as a change. Each run is compared on the website's whole stack, and the filters only choose which changes the row shows (`added`, `removed`, `versionChanged` and `previousTechCount` cover the technologies the filters keep). So adding, changing or clearing a filter between runs reports nothing by itself. A change outside your filters is saved in the monitor store but not returned or billed. If the last technology your filters keep is removed, the row comes back with an empty technology list and that technology in `removed`. Unchanged websites are not written and not billed. #### Does Website Technology Scraper detect technologies loaded by JavaScript? Yes. In Auto mode, pages built by JavaScript are rendered in headless Chrome, which sees window globals, tag-manager tags and dynamically added scripts. Set **Browser render** to Always to render every homepage. On long lists of heavy pages a cost limit can skip some renders; those rows say `renderedWith: http`. #### How does Website Technology Scraper detect email services and other SaaS tools? From the domain's public DNS records: SPF includes and domain-verification records reveal services such as Google Workspace, Microsoft 365, SendGrid, Mailgun, Zendesk, HubSpot or Salesforce. They are listed as technologies with `detectedVia: ["dns"]`. #### Can I check whether websites use one specific technology? Yes. Put the names in **Check for technologies** (`onlyTechnologies`, e.g. `Shopify`, `HubSpot`). Each row gets `matches` and `matchedTechnologies`; turn on **Only websites that match** to keep just those. #### Can I use Website Technology Scraper with the Apify API? Yes. Start runs and read datasets with `apify-client` for [Node.js](https://apify.com/topapi/website-technology-scraper/api/javascript) or [Python](https://apify.com/topapi/website-technology-scraper/api/python); the API tab has ready-to-paste code. #### Can I use Website Technology Scraper through an MCP server? Yes. Open the [MCP tab](https://apify.com/topapi/website-technology-scraper/api/mcp), connect your MCP client and ask what a website runs on. Turn on **Compact output** for smaller rows. #### Do I need an API key or an account on another service? No. Website Technology Scraper reads what any visitor's browser and DNS lookup can see; you only pass domains. #### Is this the official BuiltWith or Wappalyzer product? No. It is an independent Apify actor that detects technologies live with the open Wappalyzer-format fingerprint set and its own DNS checks. The `techs` field uses the same `name, tag, categories, link` shape as BuiltWith-style lookups so existing integrations can switch easily. #### Why did a website come back as blocked or unreachable? `blocked` means the website answered with a bot challenge or refused the request; `unreachable` means the domain does not resolve or did not answer. Both rows are free. Try a RESIDENTIAL proxy for websites that block datacenter IPs. #### Is it legal to check which technologies a website uses? Website Technology Scraper only reads publicly visible information - response headers, page markup, scripts and public DNS records - the same as any browser visit. It does not log in or collect personal data. #### Your feedback Missing a technology or a field? Open an issue on the [Issues tab](https://apify.com/topapi/website-technology-scraper/issues/open) with the domain and what you expected. # Actor input Schema ## `domains` (type: `array`): Domains or URLs, one per line, e.g. "shopify.com" or "https://example.com/pricing". Empty (and no file) runs a 3-site demo. A file that fails or has no domain fails the run, unbilled. ## `startUrls` (type: `array`): Optional. The same as Websites, in URL form, or a link to a text/CSV file with one domain or URL per line. ## `renderJavaScript` (type: `string`): Headless Chrome adds what JavaScript loads (globals, tags, pixels). Auto: pages that need it. Always: every page, but a cost limit can skip renders on long heavy lists (renderedWith: http). ## `maxPagesPerDomain` (type: `integer`): Homepage plus up to N-1 same-site pages (pricing, checkout, blog, careers); contact, imprint and directions pages are never picked, in any common language. Each is charged as an extra page. ## `includeDns` (type: `boolean`): Read the domain's DNS (MX, TXT, NS, SOA, CNAME) to detect email, DNS and SaaS services such as Google Workspace, Microsoft 365, SendGrid or Zendesk. ## `includeTls` (type: `boolean`): Read the TLS certificate (issuer, validity) and detect the certificate authority. ## `renderWaitSecs` (type: `integer`): How long the browser waits after the page loads so tag managers can inject their tags (0-15). ## `categories` (type: `array`): Keep only technologies in these categories, e.g. "CMS", "Analytics", "Ecommerce", "Tag managers". Empty = all. Monitor mode shows only changes in these categories. ## `onlyTechnologies` (type: `array`): "Does it use X?" mode: each row gets matches (true/false) and matchedTechnologies, e.g. "Shopify", "HubSpot", "Salesforce". ## `onlyMatchingSites` (type: `boolean`): With Check for technologies: return only websites that use at least one of them. ## `minConfidence` (type: `integer`): Drop technologies detected with a lower confidence (0-100). 0 keeps everything. Monitor mode shows only changes that pass it. ## `includeEvidence` (type: `boolean`): For each technology, the signals that matched (header, script URL, global variable, DOM selector, DNS record type) and the matched text. ## `includeSecurityHeaders` (type: `boolean`): HSTS, CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy and cross-origin headers of the homepage. ## `includeRawSignals` (type: `boolean`): Add the script URLs, XHR hosts, response headers and cookie names seen on the homepage (for audits). Store ids such as a Shopify shop id are removed. ## `monitorMode` (type: `boolean`): Only websites whose stack changed since the last run (added, removed, versionChanged). A removal needs 2 runs that miss it (3 renders if browser-only). Changing a filter is not a change. ## `monitorStoreName` (type: `string`): Named key-value store that remembers each website's last stack. Use one name per list you monitor. ## `maxResults` (type: `integer`): Stop after this many delivered websites. 0 = analyze every website in the list. ## `compact` (type: `boolean`): Core fields only (names, versions, categories, primary stack), sized for AI agents and MCP. ## `compareMode` (type: `boolean`): Also save a COMPARISON record: technologies common to all websites, shared ones and those unique to each website. ## `includeFailedSites` (type: `boolean`): Add a free row (status blocked / unreachable / notFound ...) for websites that could not be analyzed, so every valid, distinct website gets a row. ## `proxyConfiguration` (type: `object`): Default: websites are fetched directly and a host that refuses is retried through the Apify proxy. Pick RESIDENTIAL for sites that block datacenter IPs. ## `maxConcurrency` (type: `integer`): How many websites are analyzed at the same time (1-50, capped at 8 per GB of memory: 16 at the default 2 GB). ## `requestTimeoutSecs` (type: `integer`): Time limit for each HTTP request (5-120). ## `respectRobotsTxt` (type: `boolean`): Follow robots.txt (User-agent: \*): a disallowed website (or redirect) gets a free robotsDisallowed row; disallowed extra pages are never fetched or billed. No readable robots.txt = all allowed. ## `ignoreHttpsErrors` (type: `boolean`): Analyze websites with invalid or expired certificates. ## Actor input object example ```json { "domains": [ "shopify.com", "wordpress.org", "notion.so" ], "renderJavaScript": "auto", "maxPagesPerDomain": 1, "includeDns": true, "includeTls": true, "renderWaitSecs": 3, "onlyMatchingSites": false, "minConfidence": 0, "includeEvidence": true, "includeSecurityHeaders": true, "includeRawSignals": false, "monitorMode": false, "monitorStoreName": "website-technology-monitor", "maxResults": 0, "compact": false, "compareMode": false, "includeFailedSites": true, "proxyConfiguration": { "useApifyProxy": true }, "maxConcurrency": 10, "requestTimeoutSecs": 20, "respectRobotsTxt": false, "ignoreHttpsErrors": false } ``` # Actor output Schema ## `websites` (type: `string`): The dataset: one item per website with its technologies, categories, versions and evidence. ## `summary` (type: `string`): Websites analyzed, delivered and not delivered by reason, renders, egress and charged events. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "domains": [ "shopify.com", "wordpress.org", "notion.so" ] }; // Run the Actor and wait for it to finish const run = await client.actor("topapi/website-technology-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "domains": [ "shopify.com", "wordpress.org", "notion.so", ] } # Run the Actor and wait for it to finish run = client.actor("topapi/website-technology-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "domains": [ "shopify.com", "wordpress.org", "notion.so" ] }' | apify call topapi/website-technology-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,topapi/website-technology-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/LSGJqUgOa7LvUM0kH/builds/gl8rraOdDTRuDauTw/openapi.json