Website Tech Detector

Scrape single-page in JavaScript template

A template for scraping data from a single web page in JavaScript (Node.js). The URL of the web page is passed in via input, which is defined by the input schema. The template uses the Axios client to get the HTML of the page and the Cheerio library to parse the data from it. The data are then stored in a dataset where you can easily access them.

The scraped data in this template are page headings but you can easily edit the code to scrape whatever you want from the page.

Included features

• Apify SDK - toolkit for building Actors
• Input schema - define and easily validate a schema for your Actor's input
• Dataset - store structured data where each object stored has the same attributes
• Axios client - promise-based HTTP Client for Node.js and the browser
• Cheerio - library for parsing and manipulating HTML and XML

How it works

1. Actor.getInput() gets the input where the page URL is defined

2. axios.get(url) fetches the page

3. cheerio.load(response.data) loads the page data and enables parsing the headings

4. This parses the headings from the page and here you can edit the code to parse whatever you need from the page

   $("h1, h2, h3, h4, h5, h6").each((_i, element) => {...});
   

5. Actor.pushData(headings) stores the headings in the dataset

Resources

• Web scraping in Node.js with Axios and Cheerio
• Web scraping with Cheerio in 2023
• Video tutorial on building a scraper using CheerioCrawler
• Written tutorial on building a scraper using CheerioCrawler
• Integration with Zapier, Make, Google Drive, and others
• Video guide on getting data using Apify API
• A short guide on how to build web scrapers using code templates:

Getting started

For complete information see this article. To run the Actor use the following command:

$apify run

Deploy to Apify

Connect Git repository to Apify

If you've created a Git repository for the project, you can easily connect to Apify:

1. Go to Actor creation page
2. Click on Link Git Repository button

Push project on your local machine to Apify

You can also deploy the project on your local machine to Apify without the need for the Git repository.

1. Log in to Apify. You will need to provide your Apify API Token to complete this action.

   $apify login
   

2. Deploy your Actor. This command will deploy and build the Actor on the Apify Platform. You can find your newly created Actor under Actors -> My Actors.

   $apify push
   

Documentation reference

To learn more about Apify and Actors, take a look at the following resources:

• Apify SDK for JavaScript documentation
• Apify SDK for Python documentation
• Apify Platform documentation
• Join our developer community on Discord

Input example

{
  "urls": [
    "https://github.com",
    "https://vercel.com"
  ]
}

All fields are described in the Input tab, and the same JSON works in the API and for AI agents.

Output example

Each result is one dataset item. A real item from a test run (long fields shortened):

{
  "url": "https://github.com",
  "finalUrl": "https://github.com/",
  "title": "GitHub · Change is constant. GitHub keeps you ahead. · GitHub · GitHubAmerican AirlinesDuolingoErnst and YoungFordInfoSysMercado LibreMercedes-BenzShopifyPhi...",
  "description": "Join the world's most widely adopted, AI-powered developer platform where millions of developers, businesses, and the largest open source community build sof...",
  "technologies": [
    "React",
    "Shopify",
    "Cloudflare",
    "AWS",
    "Bootstrap",
    "Server: github.com"
  ],
  "techCount": 6,
  "hasViewport": true,
  "statusCode": 200,
  "contentType": "text/html; charset=utf-8",
  "scrapedAt": "2026-10-21T10:01:50.807Z"
}

How much does Website Tech Detector cost?

This Actor uses pay-per-event pricing: you pay only for what it delivers, and platform usage is included.

Event	Price
Result (one dataset item)	$0.002 each ($2.00 per 1,000)

Set Maximum cost per run in the run options to cap your spend; the Actor stops when the limit is reached. Check the Pricing tab for current prices and plan discounts.

Use Website Tech Detector with AI agents (MCP)

Give Claude, ChatGPT, Cursor, VS Code or any MCP client this Actor as a tool through the hosted Apify MCP server. The agent sees the input fields with descriptions and example values, runs the Actor, and reads the results.

MCP server URL (this Actor only):

https://mcp.apify.com/?tools=technicaldost/website-tech-detector

Claude Code:

$claude mcp add --transport http apify "https://mcp.apify.com/?tools=technicaldost/website-tech-detector" --header "Authorization: Bearer YOUR_APIFY_TOKEN"

Cursor, VS Code, Claude Desktop and other clients (mcp.json or the client's MCP settings):

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=technicaldost/website-tech-detector",
      "headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }
    }
  }
}

Clients that support sign-in (such as Claude.ai connectors) can use the URL without a token and log in to Apify instead. Get your token in Apify Console → Settings → API & Integrations, or generate a ready config for your client at mcp.apify.com. In the agent the tool is named technicaldost--website-tech-detector; ask it, for example, to "run Website Tech Detector with {"urls": ["https://github.com", "https://vercel.com"]}".

AI agents without an Apify account can also run and pay for this Actor per request with x402 agentic payments.

Run Website Tech Detector via API

Run the Actor and get its results in one call (replace YOUR_APIFY_TOKEN):

curl -X POST "https://api.apify.com/v2/acts/technicaldost~website-tech-detector/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://github.com", "https://vercel.com"]}'

Python (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("technicaldost/website-tech-detector").call(run_input={"urls": ["https://github.com", "https://vercel.com"]})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

JavaScript / TypeScript (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('technicaldost/website-tech-detector').call({"urls": ["https://github.com", "https://vercel.com"]});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Long runs: start them with POST /v2/acts/technicaldost~website-tech-detector/runs and fetch the dataset when the run finishes, or receive a webhook.

Related Actors

• Website Change Detector — Monitor websites for content changes with visual comparison. Get alerts when pages update
• Website Brand & Favicon Extractor — Pull the brand assets a website declares about itself: favicons and apple-touch icons at every size, logo, Ope
• Website Contact & Email Extractor — Extract contact details from websites in bulk: emails, phone numbers and social profiles, plus the contact pag
• Website RSS & Atom Feed Finder — Find every RSS, Atom and JSON feed a website publishes. Checks declared link tags, on-page links and well-know

FAQ

Can I use Website Tech Detector with AI agents like Claude or ChatGPT?

Yes. Connect the Apify MCP server with ?tools=technicaldost/website-tech-detector (see above) and the agent can run the Actor and read its results on its own.

Can I call Website Tech Detector from my own code?

Yes. Use the REST API, or the official Python and JavaScript clients shown above. Results are available as JSON, CSV, Excel, XML or HTML.

How do I run it on a schedule?

Save your input as a task in Apify Console and add a schedule (for example, daily). Use webhooks or integrations to send each run's results onward.

Is it legal to use this Actor?

It works with publicly available data. Make sure your use complies with the source website's terms and with data protection laws such as GDPR, especially for personal data.

Something doesn't work or I need a feature. What should I do?

Open an issue in the Issues tab on this Actor's page with an example input, and we'll look into it.