SEO Meta Tags Extractor

Extract HTML title, description, robots and language metadata from up to five supplied pages.

What it does

Reads each supplied page and extracts its HTML <title>, <meta name="description">, <meta name="robots">, root HTML lang, and declared <meta charset>. Reports title and description character counts. It does not score rankings, crawl linked pages, or parse social cards.

Who it is for

SEO analysts, content editors, and website owners checking title and description coverage on a few public pages.

Input example (prefilled in Apify Console)

{
  "urls": ["https://example.com/"]
}

One to five URLs. Paste them into the Page URLs field. Only HTTP/HTTPS pages on standard ports 80/443 with public domain names are accepted.

Output example (illustrative, not a claim about example.com)

Each input URL produces exactly one default-dataset item in input order, even if fetching fails. Field names and types are stable; missing values are null (lengths are zero when tags are absent on a successfully parsed page).

{
  "url": "https://example.com/",
  "finalUrl": "https://example.com/",
  "httpStatus": 200,
  "title": "Example page",
  "description": "A short example description.",
  "titleLength": 12,
  "descriptionLength": 28,
  "metaRobots": "index,follow",
  "language": "en",
  "charset": "utf-8",
  "status": "ok",
  "error": null
}

On failure, status is "error" and error contains a plain-language reason. Fields unavailable due to failure remain null. httpStatus is null for failures before a successful page response.

Limits and safety

• Only supplied URLs plus their /robots.txt files and necessary HTTP redirects are fetched; no discovery, login, or unrestricted crawling. Known search engine result-page URL patterns are explicitly rejected; the tool is not for SERP extraction.
• Robots rules are evaluated using Python's urllib.robotparser. Missing robots files (HTTP 404/410) are treated as no rules; robots fetch failures, HTTP restrictions, or explicit disallow rules fail closed. Some crawlers use different robots interpretation details.
• At most 5 pages; 4 redirects per page, 10-second socket timeout per request, 1,000,000 bytes per HTML response, 256,000 bytes per robots response. Redirect destinations get their own robots check.
• Public-host DNS and IP checks are applied before each connection; DNS-selected IP is pinned for the connection (prevents DNS rebinding within the connection). Direct IP URLs, private targets, credential-bearing URLs and nonstandard ports are rejected.
• Static HTML only: client-side JavaScript is not executed. No support for pages requiring cookies, authorization, or deliberately encoded compressed responses.
• Sites can block bot traffic, and redirects across sites are possible; the Actor does not bypass blocks.
• Parsed characters are counted as Unicode code points, not rendered pixel widths. Only the first occurrence of each supported meta tag is used.

Storage and local run

The Actor uses Python standard library only; requirements.txt contains no dependencies. On Apify it reads INPUT from the default key-value store via the API and appends results to the default dataset via the API (no SDK). For a local run, create storage/key_value_stores/default/INPUT.json with the input shown above and execute:

$python src/main.py

Local dataset JSON files appear under storage/datasets/default/. You can instead set APIFY_LOCAL_STORAGE_DIR to a custom storage folder. Running locally again will replace numbered items in that folder; clean the folder between test runs to avoid stale extras.

Suggested Apify Store categorization

SEO tools, Marketing. These are publisher-suggested categories, not fields in actor.json. Apify's 2026 Store classification rollout automatically assigns Store categories; the Actor manifest has no documented categories setting.

Suggested pricing

Pay per event, per run: start at US$0.01/run for up to five URLs, as an initial hypothesis subject to actual cost and demand testing. Configure the synthetic apify-actor-start event under Apify Console → Publishing → Monetization; disable any additional priced dataset-item event if you want one charge per run. No custom billing API logic is included. Pricing is a recommendation and is not activated by this source bundle.

Deployment

From this folder, use Apify CLI to deploy (apify push), then configure Store publication and pricing in Apify Console. Cloud validation and billing configuration require deployment; this package does not assert that it has been tested on Apify infrastructure.