Versions follow MAJOR.MINOR.PATCH (src/version.py); Apify shows MAJOR.MINOR from .actor/actor.json.
Every run logs its version and records it in the RUN_STATS key-value record.
- A run that returns nothing (every input bad, private, blocked or unreachable, or an empty input) now ends
SUCCEEDED with an empty dataset and the reason as its status message and in the log, instead of FAILED. Only an
unexpected error still fails the run.
RUN_STATS keeps the reason as before.
- No change to output fields or price.
- A run no longer fails when the store it remembers earlier runs in can't be opened (403: the account already has
a store of that name that this actor didn't create). It falls back to
humble-echidna-<name>, and if that is
refused too it runs without memory for that run. Users' existing memory is untouched: the usual name is tried first.
- Waits out longer
Retry-After answers: when a site or API asks it to wait before retrying, it now waits up
to 90 seconds (was 30), once per site, with at most 3 minutes of such long waits per run; a longer wait, or a
site that asks again after it was waited out, is reported with the reason instead of stalling the run.
A check still has 60 seconds in all, so a site asking for longer than fits is reported as rate-limited at once.
- No change to input, output or price.
urls is no longer marked required, so AI agents (Apify's MCP server) see that a dataset alone (datasetId) is enough. Nothing else changes: a run with neither still stops with a clear message.
- 2 ready-to-run examples published on Apify Store and linked from the README.
- Cheaper on paid Apify plans: $0.90 per 1,000 URLs checked on Starter, $0.80 on Scale and $0.70 on Business ($1.00 on the Free plan).
Check the websites in another actor's results, e.g. dead websites in Google Maps leads.
- New inputs Or: websites from a dataset (
datasetId, a dataset picker with read access) and Field with the
website (datasetUrlField). With a dataset, each item's website becomes one line and urls is ignored; the
mode applies as usual (in sitemap and links mode every row found through a website carries its source
fields). The field is found automatically when left empty: the first of website, url, domain, site,
homepage with a website in the first 100 items, then the same names one level down (contact.website); a dotted
path names a nested one.
- The same website twice (by its canonical address, tracking parameters aside) is checked and charged once, with the
first item's source fields.
- Items without a website, values that aren't a public web address (a phone number,
N/A, an email) and Google
Maps links (the place's own url in a Maps scraper's output) are skipped, never fetched or charged, and counted in
the new RUN_STATS.dataset record (itemsRead, websites, withoutWebsite, notAWebsite, googleMapsLinks,
duplicates, the field used and whether it was found automatically). Tracking parameters (utm_*, gclid,
fbclid, ...) are dropped from the address. A dataset with no website at all, or one that can't be read, fails
the run with the reason.
- New nullable fields on every result:
sourceTitle (the item's title, else name), sourcePlaceId (its
placeId) and sourceIndex (its position in the dataset, 0 = the first), to join results back to the leads;
null for URLs typed in. A new "Leads (from a dataset)" dataset view shows them next to the key columns.
- Read with the run's own token (the user's access, read-only), only the fields needed (whole items only
for the first 100 when the field is looked for one level down); at most the first 20,000
items and 10,000 distinct websites per run. Max results / maximum cost per run apply as before. Outside Apify, a
dataset in local storage with that name is read.
- Pricing unchanged: no new events; a website from a dataset costs the same as one typed in.
- Internal: the reader is shared (
mms_common.dataset_input), as are its test stand-ins (mms_testing).
- Input field descriptions rewritten for AI agents (Apify's MCP server shows agents only the description, not the
form): each now states its default, its allowed range, the value of each option and how it combines with other
fields. No change to field names, types, defaults or behaviour.
- Links mode: links to different sites are checked in turns, at most 2 at a time per site, and a worker never waits
on a busy site while another site's link is ready, so one slow or rate-limiting site no longer holds up every
check. A site that answers HTTP 429 (after the retry) is left alone for the rest of the run: its other links
aren't requested, aren't charged, and are listed in
NOT_CHECKED with outcome rate-limited; the site is named
in linkHostsThrottled in RUN_STATS and in the status message. (The same scheduling SEO Audit Crawler uses,
now shared in common/mms_linkcheck.) The URLs and sitemap modes are unchanged.
- The URL checking and link parsing code moved to the shared
common/mms_linkcheck (SEO Audit Crawler uses it
too). No behaviour change.
First release.
- Checks URLs and reports the status code, every redirect hop (address, status, target, time), final URL,
canonical URL (HTML
<link rel=canonical> or the HTTP Link header), response time, content type and size,
with a clear outcome: ok, broken, refused, robots-blocked, not-public, invalid-url, dns-error or error.
- Three modes: a list of URLs; every URL in a site's sitemaps (the same discovery as Sitemap URL Extractor:
robots.txt
Sitemap: lines, /sitemap.xml, indexes, .xml.gz); or every link on given pages, following same-site
pages up to a crawl depth and page cap, with the pages each link was found on.
- HEAD first, GET when the HEAD answer is an error or the page's canonical URL is wanted; HEAD-only and GET-only
options.
- "Only report URLs that changed since the last run", remembered per check in a key-value store in the user's
account; only URLs actually written to the results update the memory.
- robots.txt respected on every hop (a disallowed URL is reported, not requested); public addresses on ports 80
and 443 only; at most 2 requests at a time per site; Crawl-delay honoured.
- URLs that are never requested (disallowed by robots.txt, a private address or non-standard port, an invalid
address, a site asking for more than 30 s between requests) aren't results and aren't charged: they're listed
in the
NOT_CHECKED key-value record, counted in RUN_STATS and logged. They don't use up the max charge
either. A URL that was requested and then redirected into one of those is a result, with its redirect chain.
- A HEAD request that fails outright (dropped connection, timeout) is retried with GET before the URL is reported
as an error. HEAD answers carrying
Content-Encoding: gzip with no body (as CDNs send them) are read correctly.
- Charged per URL in the results, through Apify's standard
apify-default-dataset-item event. "Max results per
run" and the maximum cost per run are honoured before anything is requested.
- Failure isolation: a failing URL, page or sitemap only affects itself.
