Feed Autodiscovery, Self-Link, and Serving-Header Publish Gate

This Actor is a stateless gate for feed publishers. Run it before and after a site deploy. It checks the path from the HTML page to the feed, not only the feed body.

What it checks

For each site, the Actor runs these checks. You can select a subset with checks.

Check	What it does	Reason codes
discovery	Reads <link rel="alternate"> in the page. Feed types: RSS, Atom, JSON.	SITE_UNREACHABLE, SITE_HTTP_ERROR, NO_FEED_DISCOVERED, FEED_NOT_ADVERTISED, BLOCKED_TARGET
feed_status	Fetches each feed. Records the HTTP status and the redirect chain.	FEED_UNREACHABLE, FEED_HTTP_ERROR, TOO_MANY_REDIRECTS, FEED_TOO_LARGE, BLOCKED_TARGET
content_type	Compares the served Content-Type with the detected format.	CONTENT_TYPE_MISSING, CONTENT_TYPE_MISMATCH
format	Detects RSS 2.0, Atom 1.0, or JSON Feed 1.0/1.1.	FORMAT_UNKNOWN
self_link	Compares atom:link rel="self" (or feed_url) with the final fetched URL.	SELF_LINK_MISSING, SELF_LINK_MISMATCH
home_link	Compares the channel/home link host with the site host. www. is ignored.	HOME_LINK_MISSING, HOME_HOST_MISMATCH
item_backlinks	Fetches up to 5 item pages (HTML head only). Checks that each page advertises the feed.	ITEM_UNREACHABLE, ITEM_NO_BACKLINK

Each failed check has one reason code for each feed. A check that did not run has the state skipped.

Accepted Content-Type values: RSS: application/rss+xml, application/xml, text/xml. Atom: application/atom+xml, application/xml, text/xml. JSON Feed: application/feed+json, application/json.

Input

Field	Default	Notes
sites	2 public sites	1 to 100 public http(s) URLs. The Actor removes duplicates.
feedUrls	{}	Map of site URL to at most 2 explicit feed URLs.
checks	all	Subset of the check names above.
maxConcurrency	5	1 to 10.
requestTimeoutSecs	15	3 to 60.

An empty input {} runs with the defaults and succeeds.

Output

One dataset row for each site. Main fields: site_url, final_site_url, site_status, discovered_feeds, feeds, checks, failures, passed. Each entry in feeds has the feed URL, HTTP status, redirect chain, content_type, format, self-link data, home-link data, and sampled_items. Each entry in failures has check, code, and feed_url.

A failed gate is a normal result. The run still ends as SUCCEEDED. Read passed in the dataset to decide whether your CI (continuous integration) job passes. The status message shows the totals. The run fails only for a real malfunction of the Actor.

Limits and safety

• 1 MB per HTML page. 2 feeds per site. 2 MB per feed. 5 item pages per feed. The Actor reads at most 256 KB of each item page.
• At most 5 redirects for each request.
• The Actor uses HTTP only. It does not run a browser.
• The Actor rejects private, loopback, link-local, and reserved addresses. It checks the address of every redirect hop. Only ports 80 and 443 are allowed.
• The Actor refuses feeds that contain a DTD (document type definition) or entity declarations. These feeds get FORMAT_UNKNOWN.
• Known limit: the address check and the connection use two separate DNS lookups. A hostile DNS server could change its answer between them.

Pricing (pay-per-event)

The Actor calls Actor.charge for one event: site-checked. It charges once for each site checked. One event covers up to 2 feeds and 10 sampled item pages. The Actor does not charge for a site that has an invalid URL, a blocked private target, or an internal error. It charges for a site whose target is down, because it did check that site. The Actor stops when the charge limit is reached. The status message says so.

The pricing proposal is $0.004 for each site-checked event. Set the price in the Apify Console. This repository does not change pricing.

Development

uv run pytest
uv run ruff check .

Local run: apify run with .actor/default_input.json.