The reliability release. Extraction no longer depends on the Threads UI, maxItems
is delivered in full, and batch mode actually works.
Added
Hybrid extraction engine. Data is now read from Threads' own JSON payloads —
embedded bootstrap blobs first, intercepted GraphQL responses second — with the
original DOM parser kept as a third-tier fallback. Every record carries
scrapeMethod (embedded | graphql | dom) so degradation is visible in the
dataset instead of silent.
Exact engagement counts.likes, replies and reposts come from the payload
as integers. The DOM parser reported Threads' abbreviated labels, so 1.5K became
1500 and a profile with 5,687,751 followers was recorded as 5,600,000.
Pay-per-event pricing with three events: actor-start, post-output,
profile-output. Items are charged only after they are stored, and maxItems is
capped up front to what the run's budget covers. Reaching the charge limit ends the
run as a success with a status message, never a failure.
Batch mode is now on the input form — keywords, usernames, tags,
postUrls and concurrency were implemented but invisible on the Console.
maxReplies for the post action, and flat requestDelayMs / maxRetries fields.
The rateLimitConfig object form still works and is overridden per key.
RUN_SUMMARY key-value record per run: items per source, the extraction engine
mix, charged events, and why the run stopped. Linked from the Output tab.
Profiles view in the dataset schema, plus type, source, parentId,
profile, scrapeMethod, stats.shares and stats.quotes as described fields.
Link-preview thumbnails and GIF attachments are now collected into images.
Anti-blocking: session pool with early session retirement, fingerprints pinned to
desktop Chrome with en-US, and automatic session rotation on a login wall or rate
limit. When login cookies are supplied, fingerprint rotation is disabled and the
pool is pinned to one session — rotating devices under a single real identity is an
account-ban signal.
CI on every push and pull request, plus a weekly live smoke test that opens an issue
when Threads changes its payload.
Fixed
maxItems under-delivery. Three separate counters disagreed, and posts that
failed validation still consumed the quota, so runs routinely returned fewer items
than requested. A single budget now counts only stored items.
Batch mode silently scraped nothing after the first target. Every crawler shared
one request queue, so later targets hit the per-crawl request cap immediately and
finished without crawling — while still reporting success.
Batch mode returned mislabelled data at concurrency > 1. On the shared queue a
crawler could pick up another crawler's request and run the wrong handler on it,
producing profile records scraped from a search page. Each crawler now owns a
private queue.
Empty search results were reported as blocked pages: the error detector matched the
bare words blocked, unavailable and Try again anywhere in the page, which
ordinary Threads chrome contains.
Batch runs dropped rateLimitConfig and includePosts, and skipped input validation.
Profile records now report followingCount and postsCount in missingFields
rather than leaving them silently absent. Threads does not expose either field to
anonymous requests.
The main post of a post run had no source, leaving an empty column in the
Output tab.
Changed
README restructured around what the Actor produces, with pricing, a quick start and
an expanded FAQ. The limitations section is unchanged in substance but no longer
opens the page.
Live smoke tests assert shape and presence instead of pinned values, so they fail on
schema changes rather than on an account being renamed.
Known limitations
followingCount and postsCount are not available without login — Threads omits
them from anonymous responses entirely.
Anonymous runs are capped by Threads itself, roughly 10 posts for search and 20 for
a profile. Login cookies raise this, at some risk to the account used.