Website Freshness Detector

How recently was this site actually touched? One HTTP probe, one verdict.

Point this actor at any website URL and it returns a structured freshness snapshot: when the sitemap was last updated, what the Last-Modified header says, the footer copyright year, the newest post in the RSS/Atom feed, and a stale_flag verdict that rolls all four signals into a single label.

Pure Go HTTP with a real Chrome 124 TLS fingerprint. No browser, no JavaScript rendering. Fast enough to scan a lead list of thousands in minutes.

What it does

For each URL it probes up to four independent freshness signals:

1. sitemap.xml lastmod. Fetches /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, plus any Sitemap: directive from robots.txt. Parses XML and returns the max <lastmod> across all URL entries. Handles sitemap-index files by recursing one level into the first 3 child sitemaps.
2. Homepage Last-Modified header. Reads the Last-Modified HTTP response header on the homepage.
3. Footer copyright year. Scans the last 4000 bytes of the homepage HTML for a copyright anchor and picks the highest 4-digit year nearby. Handles ©, ©, (c), Copyright 2024, and ranges like 2018 - 2024.
4. RSS/Atom feed pubDate. Discovers feeds from <link rel="alternate"> tags plus well-known paths (/feed, /rss, /rss.xml, /feed.xml, /atom.xml, /index.xml) and extracts the newest <pubDate> (RSS) or <updated> (Atom).

The verdict is computed against the most-recent signal across all four.

Use cases

• Web-design agency lead prioritization. Filter your Google Maps or lead-list export for stale_flag = "stale" and pitch a rebuild. The concrete "your site's sitemap says it was last updated in 2019" line lands harder than a generic "your site looks old" cold email.
• Competitive analysis. Snapshot competitor sites monthly to see who is actively publishing and who has gone dormant.
• Content marketing audits. Identify blogs that stopped publishing so you can pitch a content-refresh or ghostwriting engagement.
• Portfolio monitoring. Nightly snapshot of a domain portfolio to catch abandoned properties before they get delisted from search.
• M&A due diligence. Combine with the Compliance Scanner and Tech Stack Detector for a fast website-health snapshot on target companies.

When to use it

Pair with Broken Link Checker and SEO Metadata Extractor to build a complete "website health" dossier per lead. Freshness answers "when was it last touched?", broken links answer "is it maintained?", SEO metadata answers "is it findable?".

Input

Parameter	Type	Default	Description
urls	array	required	Website homepage URLs to scan. Bare domains accepted.
companies	array	[]	Alternative paired input: [{"company_id":"...","website_url":"..."}] for CRM join-back.
probeTimeoutMs	integer	5000	HTTP timeout per fetch. Cap 15000.
respectRobotsTxt	boolean	true	Honor the site's robots.txt on sitemap/feed probes.
concurrency	integer	5	Number of URLs scanned in parallel.
proxyConfiguration	object	Apify datacenter	Optional proxy. Datacenter is fine for freshness probes.

Output shape

One row per input URL:

{
  "url": "https://example.com/",
  "final_url": "https://www.example.com/",
  "status_code": 200,
  "sitemap_url": "https://www.example.com/sitemap.xml",
  "sitemap_lastmod": "2024-11-14",
  "homepage_last_modified": "2024-11-14",
  "footer_copyright_year": 2024,
  "latest_feed_pubdate": "2024-11-01",
  "most_recent_signal": "2024-11-14",
  "most_recent_source": "sitemap",
  "stale_flag": "active",
  "months_since_last_signal": 3.2,
  "signals_found": 4,
  "signal_note": null,
  "probed_at": "2026-07-21T09:00:00Z",
  "error": null
}

Missing signals serialize as null. When the homepage fetch fails:

{
  "url": "https://broken.example/",
  "status_code": 0,
  "sitemap_lastmod": null,
  "homepage_last_modified": null,
  "footer_copyright_year": null,
  "latest_feed_pubdate": null,
  "most_recent_signal": null,
  "stale_flag": "unknown",
  "signals_found": 0,
  "probed_at": "2026-07-21T09:00:00Z",
  "error": "request timed out"
}

Stale flag threshold table

flag	Meaning	Age of most recent signal
active	Recently updated	Within 6 months
stale	Rebuild candidate	6 to 18 months
abandoned	Long-dormant	Over 18 months
unknown	No usable signal, or only a footer copyright year was available	See signal_note for the reason

Signal precedence

Four signals are ranked from strongest to weakest: sitemap > homepage_header > feed > footer_copyright_year. Whichever signal has the most-recent date wins most_recent_signal and drives stale_flag.

footer_copyright_year is the weakest signal on purpose. Footer years are frequently forgotten (a live 2024 site can still say © 2019), and they only carry year granularity (normalised to YYYY-01-01). If a probe finds only a footer year and nothing else, stale_flag is set to unknown and signal_note explains why — better to skip the row than to mislabel a fresh site as abandoned because someone forgot to bump the footer.

The most_recent_source field tells you which signal drove the verdict so you can weigh it in your outreach. A stale_flag = "stale" driven by sitemap or feed is high-conviction — those dates almost always reflect real publishing activity.

Pricing

• $0.001 per run start
• $0.003 per URL probed

Failed probes are billed too. They still consumed a run slot and returned a stale_flag = "unknown" row so you can retry the URL later.

1000 URLs = $3.00.

Related Actors

• Broken Link Checker. Crawl a site for 404s, redirects, SSL errors, and dead outbound links.
• Compliance Scanner. Detect Impressum, privacy policy, and cookie consent for GDPR/Abmahnung risk scoring.
• SEO Metadata Extractor. Pull titles, meta tags, Open Graph, Twitter Cards, and JSON-LD schema.
• Website Tech Stack Detector. Identify the CMS, frameworks, analytics, and hosting behind any site.
• SSL Certificate Inspector. Inspect TLS certificates and TLS versions.

Issues & Feedback

Found a signal we should detect, or a site type where the verdict feels wrong? Please open an issue on the actor page.