# Website Classifier & Text Classifier - Industry & Sentiment (`tidytools/ai-page-classifier`) Actor

Classify URLs, domains or texts with AI: industry with B2B/B2C, page type, topic, IAB category, sentiment, content moderation or your own labels, with confidence and reason. $4/1,000.

- **URL**: https://apify.com/tidytools/ai-page-classifier.md
- **Developed by:** [Yukai Lin](https://apify.com/tidytools) (community)
- **Categories:** AI, Automation, SEO tools
- **Stats:** 1 total users, 1 monthly users, 99.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.20 / 1,000 classified pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Website Classifier & Text Classifier do?

Give it a list of URLs, bare domains or texts and it **reads each one and labels it with AI**: by page type, by topic, by **industry and business model**, by IAB category, by **sentiment**, for **content moderation**, or by **your own labels**. Every result comes with a confidence level, a one-sentence reason and the language, ready to filter in a spreadsheet.

**Input:** URLs, bare domains, your own texts or another Actor's dataset. **Output:** one row per item with the label, confidence, reason and language (industry also gives business model and a company summary). **Price:** $4.00 per 1,000 items at the standard depth ($3.00 Quick, $8.00 Deep), no start fee; unreadable pages and bad input lines are free.

- 🏢 **Industry + business model**: 30 industries (SaaS, e-commerce, financial services, healthcare, marketing agencies, local services…), plus `businessModel` (B2B, B2C, B2B2C, nonprofit, government) and a one-sentence `companySummary`. Built for lead lists.
- 🏷️ **Page type**: homepage, product, category, article, news, documentation, pricing, about, contact, careers, login, legal, forum, video, event, recipe, search, landing, profile, other
- 🗂️ **Topic** (23 topics) or **IAB content categories** (30 tier-1 categories used in ad tech), with several labels per page when they apply
- 💬 **Text presets**: **Sentiment** (positive, neutral, negative, mixed) for reviews, comments and support tickets, and **Content moderation** (safe, adult, violence, hate, spam, self-harm; several can apply)
- 🏠 **Almost-empty home pages handled**: when a home page or bare domain has less than 300 characters of text, the site's About page is read too (free), and `pagesRead` shows which pages were used
- ✍️ **Custom labels in any language**: e.g. `Competitor`, `Potential customer`, `Irrelevant`, each with an optional description to guide the AI
- 🔗 **Domains, URLs, texts or another Actor's dataset**: `stripe.com` works, and so do reviews, posts or company descriptions
- 💬 **Explainable**: a short reason for every label, so you can spot-check quickly
- 🌐 **Reads protected and JavaScript sites**: plain HTTP first, a second network if a site blocks data centers, a real browser when needed
- 🅿️ **Parked and placeholder sites flagged for free**: parked or for-sale domains, "coming soon" pages and default server pages (e.g. "Welcome to nginx") come back with `label: null` and a `siteStatus`, without an AI call and without a charge
- 🧹 **Duplicates removed, rows map back to your list**: `stripe.com` and `HTTPS://www.Stripe.com/` are one site, classified and charged once; every row has `input` (your line) and `inputs` (all lines merged into it)
- 💸 **$4 per 1,000 pages, no start fee**, and pages that cannot be read are free

### Who is it for?

- **Sales and lead generation**: enrich a list of company websites with industry, B2B/B2C and a summary, then keep only the leads that fit
- **SEO and content teams**: map a whole site by page type, find thin landing pages or misplaced content
- **Data and AI teams**: label URLs or texts before scraping, building datasets or routing pages to the right parser
- **Ad tech and brand safety**: classify publisher pages into IAB categories, including sensitive topics

### How much does it cost?

| Analysis depth | What the AI reads | Price |
|---|---|---|
| Quick | first 4,000 characters | **$3 / 1,000 pages** |
| Standard (default) | first 12,000 characters | **$4 / 1,000 pages** |
| Deep | up to 32,000 characters | **$8 / 1,000 pages** |

**Example:** classifying 1,000 company websites costs **$4.00** at the standard depth, $3.00 with Quick or $8.00 with Deep.

**No start fee.** Texts are charged like pages. Pages that cannot be read (blocked, errors) are reported with an `errorType` but **not charged**. Higher Apify plans get volume discounts (see the *Pricing* tab).

For comparison, the dedicated website-industry classifiers we found on Apify Store charge $4 per 1,000 sites, the same as our standard price (checked September 2026). This Actor adds page types, topics, IAB, custom labels, confidence and reasons at that price.

#### Control your cost

- **What is charged**: one event per page or text that got a label.
- **What is free**: lines that are not a URL, private or local addresses, pages robots.txt disallows, duplicates of a site already in the list, parked / placeholder / default server pages, domains that do not exist, blocked pages, pages that only show a bot check, an "enable JavaScript" notice, a cookie-consent notice or (except with page-type labels) a login wall, image-only URLs and pages without readable text, empty texts, errors and time-outs, and results reused from your dedupe store (`fromCache: true`). Every row has `charged: true/false`, and failed rows say "(not charged)".
- **Before the run starts**, the log and the status message show the plan: how many items, the worst-case cost and your max charge per run (also saved as `costPlan` in `SUMMARY`).
- **Max charge per run**: set *Maximum cost per run* in the run options (or `maxTotalChargeUsd` in the API). When it is reached, the run stops starting new items and finishes cleanly: `SUMMARY.status` is `LIMIT_REACHED`. `SUMMARY.notProcessed` remains `{count}` without input values; `SUMMARY.remainingInput` gives the private key-value store key `REMAINING_INPUT` and the total pending count. Pending items are not written to the results dataset.
- **Continue in a new run**: open this run's key-value store and copy `REMAINING_INPUT.urls` into the new input's `domains` or `urls`, and `REMAINING_INPUT.texts` into `texts`. Copy its `taxonomy` and `analysisDepth`, plus `customLabels`, `multiLabel` or `readAboutPage` when present. Re-enter any other options you need, including extra instructions; they are not copied into this record. The saved lists retain their input order and omit completed items, so a new run does not repeat their results or charges. The record includes `totalCount`, `savedCount` and `truncated`; it holds at most 1,000 pending items and 256 KiB of JSON. If `truncated` is `true` (or `settingsTruncated` is `true`), use the original input list to add the omitted tail (or custom settings) before continuing. The saved items form a prefix of the pending list. If the original input came from a dataset, the saved values are copied into `urls` and `texts` rather than retaining the dataset link.
- **If Apify restarts the run** (server migration or Resurrect), items already finished are skipped and not charged again (`SUMMARY.resumedSkipped`).

#### Large lists: how to run thousands of domains

1. **Paste the whole list in one run** (up to 50,000 URLs or domains). Duplicates of a site are merged and charged once.
2. **Passing failures are retried for you.** A page that timed out, lost its connection or answered HTTP 408, 429, 500, 502, 503, 504, 522 or 524 is read once more at the end of the run, with half as many pages in parallel. It still gets exactly one row: the result (charged once, `retried: true`) or, if it fails again, its free error row (`retried: true`). Domains that do not exist, blocked pages, 404s, robots.txt refusals and parked sites are not retried; they fail the same way again. `SUMMARY.retryPass` counts `attempted`, `recovered`, `stillFailed` and `skipped` (not retried because the run's timeout or your max charge was near).
3. **Interrupted runs continue without paying twice.** If Apify migrates or restarts the run, finished items are skipped, items waiting for their retry keep waiting, and nothing is charged again.
4. **Run the failures again in one click.** The key-value store record `FAILED_DOMAINS` lists every URL or domain that still failed, one per line exactly as you entered it (a one-column CSV without a header). Paste it into *URLs or domains* of a new run. `SUMMARY.failedByType` tells you why they failed (`network` with a domain that does not exist, `blocked`, `not_found`…), so you can drop the ones that will never work.
5. **Lists you run again and again: skip what you already have.** Set *Skip items classified in earlier runs* (`dedupeStoreName`, e.g. `"my-leads"`). Each charged result is also kept in a key-value store named `classifier-cache-my-leads` in your own Apify account. A later run with the same store name and the same settings (taxonomy, custom labels, several labels, extra instructions, depth, About page; for URLs also the page reading mode, route and proxy) returns the earlier result for a URL or text it already classified within `dedupeMaxAgeDays` (default 30), with `fromCache: true` and `cachedAt`, **not charged**. Change any of those settings and items are classified (and charged) again. Off by default; delete the store in *Storage* to start over.
6. **Check the counts.** `SUMMARY.byResult` has one count per result type: `classified` (charged), `fromCache`, `placeholderSites`, `failed`, `invalidInput` and `notProcessed` (stopped at your max charge; see `REMAINING_INPUT`).

```json
{ "domains": ["stripe.com", "notion.so", "…5,000 more…"], "taxonomy": "industry", "dedupeStoreName": "my-leads", "dedupeMaxAgeDays": 30 }
```

### How to use it

1. Paste **URLs or bare domains** (one per line), or add **texts**. A line that is not a URL becomes one free `invalid_input` row; the rest of the run continues.
2. Choose **Classify by**: *Page type*, *Topic*, *Industry + business model*, *IAB* or *Custom labels*.
3. For custom labels, write one label per line, optionally with a description after a colon.
4. Click **Start** and export the results as CSV, Excel or JSON. The **Summary** counts items per label.

#### Input example: industry of company websites

```json
{
    "domains": ["stripe.com", "allbirds.com", "redcross.org", "www.gov.uk", "hubspot.com"],
    "taxonomy": "industry"
}
```

#### Output example (real results)

```json
[
    { "url": "https://stripe.com", "label": "financial-services", "businessModel": "B2B", "confidence": "high",
      "companySummary": "Stripe is a financial services platform that helps businesses accept payments, build flexible billing models, and manage money movement." },
    { "url": "https://allbirds.com", "label": "consumer-products", "businessModel": "B2C", "confidence": "high",
      "companySummary": "Allbirds is a company that designs and sells comfortable and sustainable shoes made from natural materials." },
    { "url": "https://redcross.org", "label": "nonprofit", "businessModel": "nonprofit", "confidence": "high" },
    { "url": "https://www.gov.uk", "label": "government-public-sector", "businessModel": "government", "confidence": "high" },
    { "url": "https://hubspot.com", "label": "software-saas", "businessModel": "B2B", "confidence": "high",
      "companySummary": "HubSpot provides software and tools for businesses to manage marketing, sales, customer service, and CRM" }
]
```

A text works the same way: *"We are a family-run plumbing company serving Leeds since 1998…"* came back as `local-home-services`, `B2C`. Business models are the AI's reading of the page; for mixed cases (marketplaces, platforms) check them where it matters. Sites that refuse automated access (in our test, doordash.com returned HTTP 403 even in a browser) come back with `errorType: "blocked"` and are not charged.

#### Parked domains and placeholder pages

Lead lists often contain domains with no business behind them. When a short page matches a parked or for-sale domain, a "coming soon" / under-construction page, an example page or a default web server page, the row has `success: true`, `label: null`, `siteStatus` (`parked`, `placeholder` or `default_server_page`) and a `reason`, and it is **not charged** (no AI call). Real sites get `siteStatus: "active"`. Real result: `example.com` → `siteStatus: "placeholder"`, not charged. A domain that does not exist fails in a few seconds with `errorType: "network"` ("does not exist (DNS lookup failed)").

#### Lead-generation workflow: scraper → classifier → filter

1. Run a lead source, for example a Google Maps or company-directory scraper, that outputs each company's `website`.
2. Start this Actor with that run's dataset:

```json
{
    "datasetId": "<dataset ID of the scraper run>",
    "datasetField": "website",
    "taxonomy": "industry",
    "analysisDepth": "quick"
}
```

3. In the results, filter by `label` and `businessModel` (e.g. only `software-saas` + `B2B`), or use **Custom labels** such as `Good fit: B2B software company with 10+ employees` / `Poor fit: anything else`. *Quick* depth ($3 / 1,000) is enough for home pages.

With a dataset, values that look like URLs or domains are read as pages; other values (e.g. review texts) are classified as text. Leave *Dataset field* empty to use the first URL field found.

#### Input example: page types

```json
{
    "domains": [
        "https://stripe.com/pricing",
        "https://docs.python.org/3/tutorial/index.html",
        "https://www.bbc.com/news",
        "https://www.allbirds.com/products/mens-tree-runners"
    ],
    "taxonomy": "pageType"
}
```

Real results for this input: `pricing`, `documentation`, `news` and `product`, all with high confidence.

```json
{
    "url": "https://www.ikea.com/us/en/cat/billy-bookcases-58288/",
    "success": true,
    "label": "category",
    "confidence": "high",
    "reason": "The page lists several products, all variations of the BILLY bookcase, with prices and details.",
    "language": "en",
    "unmatchedLabel": null,
    "taxonomy": "pageType",
    "analysisDepth": "standard",
    "mode": "fast",
    "via": "backend"
}
```

#### IAB categories

With `"taxonomy": "iab"` (real results, quick depth): bbc.com/sport/football → `Sports` + `News and Politics`; allbirds.com → `Shopping` + `Style & Fashion`; stripe.com → `Business and Finance` + `Technology & Computing`.

#### Almost-empty home pages: the About page is read too

Some company home pages are a logo and a video. When a home page (or bare domain) gives less than 300 characters of text, the Actor follows the site's About / Company link (or tries `/about` and `/about-us`) and classifies both pages together, at no extra cost. Real result for `palantir.com` read over plain HTTP (the home page gave 14 characters of text):

```json
{
    "url": "https://palantir.com",
    "label": "software-saas",
    "businessModel": "B2B",
    "confidence": "high",
    "companySummary": "Palantir builds software that empowers organizations to integrate their data, decisions, and operations.",
    "pagesRead": ["https://www.palantir.com/", "https://www.palantir.com/about"]
}
```

This is used with every taxonomy except *Page type* (where the home page itself is what you want labelled). Turn it off with `"readAboutPage": false`.

#### Text presets: sentiment and content moderation

```json
{
    "texts": [
        "Arrived two days early and the sound quality is amazing. Best headphones I have owned.",
        "Great screen, but the keyboard feels cheap and the fan is loud."
    ],
    "taxonomy": "sentiment"
}
```

Real results: the first text → `positive`, the second → `mixed` ("The text mentions both a positive and a negative aspect of the product."); a complaint about a dead battery and unanswered emails → `negative`; "The package contains one charger and a USB-C cable." → `neutral`. All four with `confidence: high`.

With `"taxonomy": "contentModeration"`, five test texts came back as: a thank-you comment on a recipe → `safe`; a news sentence about a city budget → `safe`; "CLICK HERE to win a FREE iPhone!!! ... enter your card details" → `spam`; an insult telling people to leave the country → `hate`; a message about planning to end one's life → `self-harm`. The `labels` field lists every category that applies when there is more than one. Moderation labels are an aid for review queues, not a replacement for human moderators.

#### Input example: your own labels

```json
{
    "domains": ["https://asana.com/", "https://www.atlassian.com/software/jira", "https://www.allbirds.com/"],
    "taxonomy": "custom",
    "customLabels": [
        "Competitor: sells project management or team collaboration software",
        "Potential customer: software or technology company that may need a project management tool",
        "Irrelevant: anything else"
    ]
}
```

Labels can be in any language; results use your labels exactly as written. With **Topic** and **IAB**, the `labels` field lists every label that applies.

#### Output fields

| Field | Meaning |
|---|---|
| `input`, `inputs`, `inputIndex` | The line you entered, every line merged into this site (duplicates), and its position in your list |
| `label` | The best label (`null` for parked or placeholder sites) |
| `siteStatus` | `active`, or `parked`, `placeholder`, `default_server_page` (not charged) |
| `charged` | Whether this row was charged |
| `datasetIndex` | With *Dataset ID*: position of the value in the source dataset |
| `labels` | All labels that apply (only with several labels per page) |
| `pagesRead` | Only when the About page was read too: the home page and the About page used |
| `businessModel`, `companySummary` | Only with *Industry*: B2B / B2C / B2B2C / nonprofit / government, and one sentence about the company |
| `confidence` | `high`, `medium` or `low`: the AI's own estimate of how clearly the page fits |
| `reason` | One sentence explaining the choice |
| `language` | Main language of the page (ISO 639-1 code) |
| `unmatchedLabel` | Filled only in the rare case the AI answered with a label that is not in your list |
| `via` | How the page was fetched: `backend`, `direct` (Apify's network) or `browser` |
| `source`, `textPreview` | For texts: `text-0`, `dataset-12`… and the start of the text |
| `error`, `errorType` | For items that failed: `invalid_input`, `robots_disallowed`, `blocked`, `unreachable`, `network`, `not_found`, `timeout`, `no_text`… (not charged) |
| `retried` | `true` when the row comes from the end-of-run retry of a passing failure |
| `fromCache`, `cachedAt` | Only with *Skip items classified in earlier runs*: `true` when the result was reused from your dedupe store (not charged), and when it was saved there |

The key-value store record `SUMMARY` has `status` (`SUCCESS`, `PARTIAL_RESULTS`, `FAILED`, `NO_RESULTS`, `LIMIT_REACHED`), `byResult` (one count per result type), `failedByType` (failures per `errorType`), the count per label and, when robots.txt excluded pages, `skippedByRobotsTxt`. When pages were retried it has `retryPass`; when some still failed, `failedDomains: {key, count}` points to the `FAILED_DOMAINS` record; with a dedupe store, `dedupe: {store, maxAgeDays, fromCache}`. A charge-limited run also has `notProcessed: {count}` and `remainingInput: {key, count}`; the private `REMAINING_INPUT` record contains the pending input values.

### Use with AI agents (MCP)

Connect Apify's MCP server (https://mcp.apify.com?tools=tidytools/ai-page-classifier) to Claude, Cursor or any MCP client, then ask e.g. "What industry are these 20 companies in, and which of them sell to businesses?".

```json
{ "domains": ["stripe.com", "notion.so"], "taxonomy": "industry" }
```

Failed items are not charged and carry an `errorType`, so an agent can tell a blocked site from a real answer.

### Use it from code and integrations

Run it from your own code with the Apify API. This call waits for the run and returns the results as JSON (replace `YOUR_TOKEN` with your Apify API token):

```bash
curl -X POST "https://api.apify.com/v2/acts/tidytools~ai-page-classifier/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["https://stripe.com/pricing","https://docs.python.org/3/tutorial/index.html","https://www.bbc.com/news","https://www.allbirds.com/products/mens-tree-runners"]}'
```

The synchronous endpoint waits up to 5 minutes. For bigger runs, start the run with `POST https://api.apify.com/v2/acts/tidytools~ai-page-classifier/runs` and read the dataset when it finishes, or use the `apify-client` package for JavaScript or Python.

**Schedules and integrations:** run it daily or weekly with Apify Schedules, get a webhook when a run finishes, or send the results to Zapier, Make, n8n, Google Sheets, Slack and other apps with Apify integrations. Results can be exported as JSON, CSV, Excel or XML.

### Tips

- Descriptions make custom labels much more accurate: say what belongs in each label, not just its name.
- Use **Extra instructions** (up to 1,000 characters) for edge cases, e.g. "Our own website is never a competitor."
- For company classification, use home pages or bare domains; a single blog post may be about a different topic than the company.
- `confidence` is the model's self-assessment. Spot-check `low` and `medium` results, and a sample of `high` ones for important decisions.
- If a site blocks our servers, the Actor retries from Apify's network automatically. For stubborn sites, set *Plain HTTP requests from* to Apify's network and add a proxy under *Advanced settings*.
- To extract specific fields (prices, names, dates) instead of a label, use **AI Web Data Extractor** (by TidyTools).

### How accurate is it?

We labelled a sample ourselves and ran the Actor on it (October 3, 2026, Standard depth, default settings). Pages the Actor could not read (blocked sites, errors) are free and are not counted below.

| Classify by | Items sent | Read and labelled | Matches our label | Accuracy | Not readable (free) |
|---|---|---|---|---|---|
| Industry + business model (company home pages) | 82 | 63 | 61 | **97%** | 19 (mostly bot-blocked sites) |
| Page type (one page of each kind) | 22 | 21 | 20 | **95%** | 1 |
| Sentiment (short reviews and notes) | 12 | 12 | 12 | **100%** | 0 |
| **All** | **116** | **96** | **93** | **97%** | 20 |

The three mismatches are arguable: cisco.com came back as `software-saas` (we had said telecommunications), tyson.com as `consumer-products` (we had said agriculture-food-production), and the GPL 3.0 license text as `documentation` (we had said `legal`).

Read this as a rough guide, not a benchmark: one person wrote the labels, the sample is small, mostly well-known sites and clear-cut cases, and the closest label is a judgment call for companies that fit two industries. Smaller or unusual sites, short pages and mixed-purpose pages will score lower. 1 in 5 of the company sites in our industry sample could not be read at all, which is why a free `blocked` row is not an error in the AI. Spot-check with the one-sentence `reason` on every row.

### Limitations

- The AI reads the first part of each page (4,000, 12,000 or 32,000 characters depending on the depth); details further down a long page are not considered.
- Some sites block automated access; those pages are reported with an error and not charged. The Actor identifies itself (`TidyToolsFetch`) and does not get around bot protection.
- **robots.txt is respected** for every page it reads (including the About page and the page a redirect lands on): a page the site's robots.txt disallows is not read and comes back as a free `robots_disallowed` row. Texts you paste are never checked.
- Private, local and reserved addresses (`localhost`, `127.0.0.1`, `192.168.x.x`, `169.254.x.x`, `*.local`…) are not public web pages: each becomes a free `invalid_input` row and is never requested. A website that cannot be reached or has an invalid SSL certificate (Cloudflare 525/526/527/530) comes back as a free `unreachable` row.
- Input mistakes never fail the run: with nothing usable (no valid URL, text or dataset, or fewer than 2 custom labels), the run ends normally with free `invalid_input` rows (`charged: false`) and `SUMMARY.status` `NO_RESULTS`. An empty entry in *Texts to classify* gets its own free row.
- *Time limit per item* (Advanced settings, default 180 seconds) stops a page that takes too long; it comes back as a free `timeout` row.
- As with any AI model, a small share of labels can be wrong, especially for pages that mix several purposes.
- Runs on a paid Apify plan use a higher-accuracy AI model for the labels; runs on the free plan use a standard model, so their labels can be somewhat less accurate. The price per item is the same.

### FAQ

**Does the Actor collect usage data?** The Actor sends one anonymous usage counter per run (input type, item count range, outcome) to improve the tool; no URLs, content or account data.

**Can this website classifier tell me each company's industry?** Yes. Set **Classify by** to Industry + business model to get one of 30 industries, plus `businessModel` (B2B, B2C, B2B2C, nonprofit, government) and a one-sentence `companySummary`. Bare domains like `stripe.com` work, and when a home page has less than 300 characters of text, the site's About page is read too at no extra cost.

**Can I use it as a text classifier for sentiment analysis or content moderation?** Yes. Add texts in **Texts to classify (optional)** and choose Sentiment (positive, neutral, negative, mixed) or Content moderation (safe, adult, violence, hate, spam, self-harm; several can apply). Moderation labels are an aid for review queues, not a replacement for human moderators.

**Does it support IAB categories and my own labels?** Yes. IAB uses the 30 tier-1 content categories used in ad tech, with several labels per page when they apply. With **Custom labels**, write one label per line in any language, optionally with a description after a colon; results use your labels exactly as written.

**What is charged and what is free?** One event per page or text that got a label: $4 / 1,000 pages at the Standard **Analysis depth**, $3 with Quick or $8 with Deep, with no start fee. Lines that are not a URL, private or local addresses, pages robots.txt disallows, duplicates of a site already in the list, parked / placeholder / default server pages, domains that do not exist, blocked pages, image-only URLs and pages without readable text, empty texts, errors and time-outs are free.

**Does it respect robots.txt?** Yes. Before reading a page the Actor checks the site's robots.txt (as `TidyToolsFetch`); a disallowed page is not read, comes back as a free `robots_disallowed` row and is counted in `SUMMARY.skippedByRobotsTxt`.

**What happens if my input is wrong?** The run ends normally. An unusable URL or private address such as `localhost:3000` and each empty text get free `invalid_input` rows (`input: null`, `charged: false`) with a fixed reason and, where available, the source field and entry number; usable items continue. An unknown *Classify by* value or fewer than 2 usable custom labels instead stops processing with one free `invalid_input` row and `SUMMARY.status` `NO_RESULTS`.

### Related Actors

More tools by TidyTools that work well with this one:

- [Web Page Summarizer](https://apify.com/tidytools/web-page-summarizer): summarize pages or texts with key points and checked quotes
- [AI Web Data Extractor](https://apify.com/tidytools/ai-web-data-extractor): turn any page into structured JSON with the fields you choose
- [Company Website Enrichment](https://apify.com/tidytools/company-website-enrichment): add company details to a list of domains
- [Website Tech Stack Detector](https://apify.com/tidytools/website-tech-stack-detector): find the CMS, shop platform and tools a site uses

### Support

Open an issue in the **Issues** tab with an example URL. Issues are checked regularly.

# Actor input Schema

## `domains` (type: `array`):

Main input (fill this, or `startUrls` / `texts` / `datasetId`). Websites or pages to classify, one per line: bare domains (stripe.com) or full URLs. Duplicates of the same site (stripe.com, https://www.Stripe.com/) are classified and charged once, with every original line listed in "inputs". A line that is not a URL becomes one free error row; the rest of the run continues.

## `startUrls` (type: `array`):

Same as "URLs or domains", in Apify's request list format (also accepts a link to a text file with URLs). Kept for existing integrations; new integrations should use "domains".

## `taxonomy` (type: `string`):

pageType: 20 page types (homepage, product, category, article, news, documentation, pricing, careers, legal, other...). topic: 23 topics such as technology, finance, health (several can apply). industry: the company's industry (30) plus business model and a one-line summary; best with domains. iab: the 30 tier-1 IAB categories. sentiment: positive, neutral, negative or mixed. contentModeration: safe, adult, violence, hate, spam, self-harm (several can apply). custom: your labels below.

## `customLabels` (type: `array`):

Used when "Classify by" is "Custom labels". One label per line, optionally with a description after a colon, e.g. "Competitor: sells project management software". 2 to 100 labels, any language.

## `multiLabel` (type: `boolean`):

Return all labels that clearly apply (in "labels") besides the main one. On by default for topics and IAB, off for page types, industries and custom labels.

## `instructions` (type: `string`):

Optional hints for the AI, up to 1,000 characters, e.g. "Pages about our own product are not competitors."

## `texts` (type: `array`):

Your own texts instead of (or besides) URLs: reviews, posts, company descriptions, support tickets. Each text is one item.

## `datasetId` (type: `string`):

Classify the output of another Actor run, e.g. a Google Maps or lead scraper. Values that look like URLs or domains are read as web pages; other values are classified as text.

## `datasetField` (type: `string`):

Field of the dataset items to use, e.g. "website" or "text" (dot paths such as "company.url" work). Empty = the first URL field found (url, website, link...), else the first long text field.

## `analysisDepth` (type: `string`):

How much of each page the AI reads. Quick: the first 4,000 characters (cheapest, fine for home pages). Standard: 12,000. Deep: 32,000 (long or complex pages). Each depth has its own price.

## `readAboutPage` (type: `boolean`):

For home pages or bare domains with less than 300 characters of text (JavaScript shells, splash pages), also read the site's About page (an About / Company link, or /about) and classify both together. No extra charge. Not used with the Page type taxonomy.

## `mode` (type: `string`):

"Auto" reads pages over plain HTTP and uses a real browser only when a site blocks that or needs JavaScript. Values: auto = Auto (recommended); fast = Plain HTTP only; browser = Browser only.

## `maxConcurrency` (type: `integer`):

How many pages are classified at the same time.

## `httpVia` (type: `string`):

Some sites block requests from data centers. Auto retries from a second network before falling back to a real browser. If both are refused, Auto also tries our second server (Oracle, different IP) before a real browser.

## `itemTimeoutSecs` (type: `integer`):

A page or text that takes longer (reading + AI) is stopped and returned as a free error row with errorType "timeout". 0 = no limit.

## `dedupeStoreName` (type: `string`):

For lists you run again and again. Give a name (e.g. "my-leads"): each charged result is also kept in a key-value store named classifier-cache-<name> in your Apify account, and a later run with the same name and the same settings (taxonomy, labels, instructions, depth; for URLs also the reading mode, route and proxy) returns the earlier result for that URL or text instead of classifying it again, marked fromCache and not charged. Empty = off.

## `dedupeMaxAgeDays` (type: `integer`):

Only with the store name above: results older than this are classified again (and charged as usual).

## `proxyConfiguration` (type: `object`):

Only used for requests sent from Apify's network. Apify proxy usage is billed to your Apify account.

## Actor input object example

```json
{
  "domains": [
    "https://stripe.com/pricing",
    "https://docs.python.org/3/tutorial/index.html",
    "https://www.bbc.com/news",
    "https://www.allbirds.com/products/mens-tree-runners"
  ],
  "taxonomy": "pageType",
  "analysisDepth": "standard",
  "readAboutPage": true,
  "mode": "auto",
  "maxConcurrency": 10,
  "httpVia": "auto",
  "itemTimeoutSecs": 180,
  "dedupeMaxAgeDays": 30
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `full` (type: `string`):

No description

## `details` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "https://stripe.com/pricing",
        "https://docs.python.org/3/tutorial/index.html",
        "https://www.bbc.com/news",
        "https://www.allbirds.com/products/mens-tree-runners"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("tidytools/ai-page-classifier").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "https://stripe.com/pricing",
        "https://docs.python.org/3/tutorial/index.html",
        "https://www.bbc.com/news",
        "https://www.allbirds.com/products/mens-tree-runners",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("tidytools/ai-page-classifier").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "https://stripe.com/pricing",
    "https://docs.python.org/3/tutorial/index.html",
    "https://www.bbc.com/news",
    "https://www.allbirds.com/products/mens-tree-runners"
  ]
}' |
apify call tidytools/ai-page-classifier --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tidytools/ai-page-classifier"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/6XEMjr8sYaAARYqyi/builds/tkilX8krL7pg8z6RO/openapi.json