Instagram Comment Scraper with Sentiment
Pricing
from $2.20 / 1,000 comments
Scrape Instagram comments from post URLs or an account's recent posts, with a sentiment label and score on each. $1.75 per 1,000 comments: 32% below the Free plan price of apify/instagram-comment-scraper, one flat rate for every buyer.
Notable changes to this Actor, newest first. Version numbers follow the
platform's two-part MAJOR.MINOR format.
A run that collected nothing because of its input now ends SUCCEEDED (2026-09-30), not FAILED, with its free error rows and a status message saying why:
invalid_input or nothing_to_collect, one
row with an empty inputUrl;input_refused);collection_failed still fails the run. Logged out, a mistyped handle and a
refused exit look the same, so it cannot be called the buyer's mistake.
A docs-versus-code audit of all three listings (2026-09-23), and four bugs it turned up:
profile_private or no_posts. Before, it
was a green run with no rows, the same answer as a filter that matched
nothing. details and the profile listing still return the profile row.resultsLimit over 100 failed the
word ("limit must be between 1 and 100"). They are now paged like a
profile's.latestComments on a post row carried sentimentLabel/sentimentScore
on the listings that do not sell sentiment. It now follows the listing./reels/audio/<id>/ is refused as an audio page. Before, it was
collected as the post "audio" and failed as a post nobody could find.The documentation fixes:
firstComment/latestComments appear only after a comment walk.addParentData adds ownerId/ownerFullName to post rows only.directUrls.latestPosts is [], not absent.public_email/public_phone_number are not offered logged out.engagementRatePct formula is stated as computed.commentsPostLimit; the 30-minute per-target deadline; error
rows are free; and duplicates and inputs that matched nothing write no row.Every post row names its own author (2026-09-23). ownerUsername was
written only with addParentData on, and then it was the profile being
walked: a reels run of @nasa with addParentData: false returned twelve
rows naming nobody, and with it on would have credited all twelve to nasa -
ten were collab reels published by @ravens, @astro_jessica, @steelers and
others. Each post now carries owner_username, read off the post itself
(the grid and the keyword feed select user.username, a post's own page
user), at no extra request, and the row writes it whatever
addParentData says. ownerId and ownerFullName stay behind
addParentData and appear only when the walked profile is the author. A
snapshot collected before the field falls back to the old behaviour.
View counts are no longer promised on profile reels (2026-09-23).
Measured on the same twelve reels: logged out, neither the profile grid nor
a post's own page reports views, so videoViewCount was absent on every
one. It is present on videos found by a hashtag search, whose feed carries
play_count. The videoViewCount and avgVideoViews descriptions, the
enrichPosts description and both READMEs now say so.
Hashtag fixes after the first review (2026-09-23). An empty keyword now
says why it is empty instead of always blaming Instagram:
keyword_filtered_out when the word answered and the run's own
onlyPostsNewerThan, resultsType: reels or comment walk emptied it,
keyword_not_held on a data source that cannot collect, and
collection_timed_out - a failure, not an answer - when the job was still
running with nothing held. keyword_not_catalogued is left for a collection
that finished with nothing. A search word that repeats an
/explore/tags/<word>/ URL in the same run is collected and billed once.
HASHTAG_TARGETS_ENABLED is derived from whether a listing's form shows
searchLimit, so Instagram Profile Scraper refuses a tag URL too; only the
URL is gated, and the search field sent over the API runs on every listing
as it did before 2026-09-19. A tag URL's rows carry the tag page as url,
/explore/tags/ with no word is refused as a missing hashtag, and
/collect-post given a tag names it as one.
resultsType: reels counts videos (2026-09-23). "12 reels" returned the
videos among the 12 newest posts - 3 rows for @nasa, reported by a user and
measured live. The collection job carries a new videos_only field (both
brokers, in the in-flight fingerprint): the worker lists past the photos,
bounded at VIDEO_WALK_FACTOR (5) times the limit, and reads post pages for
the videos only. The same input now returns 12 rows in 35s, 27 posts listed
and 12 filled. A hashtag run with reels counts searchLimit in videos the
same way. A worker older than the field ignores it and walks N posts as
before. The form's resultsType description now says the limit counts
videos.
Found by the live three-listing simulation (2026-09-23,
tools/bench/actor_listings_sim.py). A run with search and no
directUrls - the input instagram-scraper's README sells - failed with
"Nothing to collect"; it runs now, and the empty-input message mentions
search on the listings whose form has it. A word the worker failed as
empty (its only way of reporting a closed word) reached the dataset as
internal_error and a failed target; it is now the unbilled
keyword_not_catalogued row on a green run, written without reading the
word again (the read collected on miss, a second request for a word already
refused). Any other failed or rejected collection is collection_failed
with the collector's own reason, no longer internal_error.
Hashtags are collected, and the 3 September verdict was our own bug
(2026-09-19). /explore/tags/<tag>/ is a target on Instagram Scraper:
planner._hashtag_keyword reads the word out of the URL and runs it as a
keyword, so directUrls and the search field reach the same surface. The
reason this was thought closed is that the logged-out probe sent debug as a
JSON boolean, and Instagram answered noncoercible_variable_value — a
complaint about our variables, not a refusal. It wants the string "false".
Three refusals had been read as one: only Unauthorized logged out query. is
a perimeter. Measured from Italian residential exits: Instagram's own
logged-out tag page preloads PolarisLoggedOutPopularSearchPageQuery with the
bare word, a median 94% of the posts it returns carry #word in the caption
and every one of them mentions it, and about 60 posts is the entire
logged-out catalogue for a word — media_count is honoured to 50, and from
100 up the feed returns ~60 and closes. Instagram opens this per word: on the
day of measurement travel served the feed and food served a login wall, so
a word that returns nothing writes one keyword_not_catalogued row explaining
that, and the row is not charged. Two collector defects came out of the same
tests: a leading # returned zero posts silently (the feed wants the bare
word) and the page size was pinned at 24, so sixty posts cost three requests.
#travel at --limit 60 now returns 60 posts in one request, 11s, 57 KiB,
where it previously returned none. The whole shipped run, with the per-post
completion the keyword feed makes necessary (SEARCH_CANNOT_CARRY: no date, no
counts, no dimensions, no tags), measured 61 requests, 8m 29s and 5.1 MiB with
all 61 successful - one request for the tag and one per post. Hashtag URLs are refused by name on
Instagram Profile Scraper (resultsType: details returns one account's own
figures, and a tag has no account) and on Instagram Comment Scraper — not a
perimeter there but that listing's form, which copies
apify/instagram-comment-scraper's three fields so a migrating input runs
unchanged and therefore has no searchLimit to bound a sixty-post comment
walk. The switch is HASHTAG_TARGETS_ENABLED, written into actor.json by
the generator like SENTIMENT_ENABLED, and the refusal names the Actor that
does have the ceiling.
Sentiment is the comment Actor's alone (2026-09-16). Instagram Scraper
and Instagram Profile Scraper drop "with Sentiment" from their titles, pages,
schemas and tabs, and their runs no longer write sentimentPositive/
Neutral/Negative or sentimentLabel/Score even when the snapshot holds
labels. The switch is SENTIMENT_ENABLED, written into every Store
listing's actor.json by the generator; the collector still scores comments,
so a snapshot shared between listings is labelled for the one that sells it.
Store prices are pushed by tools/apify_pricing.py (2026-09-16).
pay_per_event.json never reached the platform, so all three listings showed
"Pay per usage". Event titles are bare nouns now so the badge reads
"$1.95 / 1,000 profiles".
One price, $1.75 per 1,000 results, on every Actor (2026-09-17, extended
to the base MCP-server manifest 2026-09-18). unit_prices_usd is gone and
price_usd in tools/actor_manifests/listings.py is the whole pricing
model: the base manifest used to charge $0.00195 / $0.002025 / $0.001725 by
row shape, so one MCP call's invoice depended on what its answer happened to
contain. The listing pages name no plan tiers of ours at all - there are
none - and no longer reprint the counterparts' discount ladders; they state
the flat rate, the percentage below the counterpart's Free-plan price, and
one clause saying that rate of theirs moves with the buyer's subscription
while ours does not. It replaces both the old per-listing prices
($2.025 / $1.95 / $1.725) and a four-tier ladder drafted on the 17th and
reverted the same day, which never reached the platform. $1.75 is 35% below
what apify/instagram-scraper charges its Free-plan buyers and 32% below the
profile and comment counterparts', and it does not move with anything — so the
figure in a description holds for whoever reads it. It is not a saving against
every rate they charge: their tiers dip under $1.75 for their larger
subscribers (from Gold for the general and profile counterparts, from Platinum
for the comment one). That is recorded internally — CROSSES_AT in
tests/test_actor_variants.py fails the suite if a counterpart discounts
further and a claim goes stale — and stated on the listing pages as one clause
rather than a reprint of their ladder. apify 4.0.2 will not bill an event whose
run pricing arrives as a tier table, and the biller still logs an error the
first time it sees one: dead code at a flat price, kept for the next time tiers
are considered.
apify push does not update an Actor's description (2026-09-18).
Three successful pushes of version 1.5 carrying the new wording left the
platform still serving June's "$2.025 per 1,000 results: 25% below ...".
Title and description are account-level metadata, not part of a build.
tools/apify_listing_meta.py (check / --apply) PUTs them, the same shape
as tools/apify_pricing.py and for the same reason: a file in the build that
looks authoritative and is not. The Store README does travel with the build.
Prices are live: $1.75 / 1,000 on all three listings (2026-09-19). Payout
billing info was filled in, which unblocked the 2026-09-16 HTTP 400. Two
things had to be fixed first. The Console records set by hand priced Apify's
synthetic apify-default-dataset-item, which bills every row written to the
default dataset - error rows included, against what all three READMEs
promise - and on two listings also apify-actor-start, charging a run before
it returns anything. Both are gone: the four custom events the code charges
are now the price list, so an error row is free again. And pricingInfos is
an append-only history rather than the current price: PUTting one record
reads as deleting the rest and the API refuses with
cannot-remove-pricing-info, so the pusher now appends and the newest
startedAt is what bills.
A run never writes rows its spending limit will not pay for (2026-09-17).
Rows were pushed in chunks of 100 and charged afterwards, so the chunk that
crossed ACTOR_MAX_TOTAL_CHARGE_USD landed whole and was billed in part: up
to 99 rows free per run. Each chunk is now trimmed first to what the SDK's
charging manager says the budget covers, the same count
Actor.push_data(charged_event_name=...) would push. The price push also
sets minimalMaxTotalChargeUsd to one primary result, so no user can start a
run whose limit cannot pay for a single row.
The four Apify manifests are generated now, not copied (2026-09-15).
actor.json, input_schema.json, dataset_schema.json,
output_schema.json and pay_per_event.json existed as four hand-maintained
near-copies under .actor/ and actors/*/ — around 8.000 lines, of which
the dataset schema alone was ~2.050 lines of byte-identical paste encoding
three real decisions. tools/actor_manifests/ replaces them with three
tables: columns.py (every dataset column once, its prose carried across
character for character), form.py (every input field, with the wordings
each listing gives it) and listings.py (which of them each Actor shows, in
what order, at what price). python -m tools.actor_manifests --check is the
drift gate and is a test. Editing the JSON by hand now survives exactly until
the next regeneration, which is why the generator, not the output, is where a
change goes.
Each listing declares only the columns its own runs can produce
(2026-09-15). The comment Actor declared all 78 columns — latestPosts,
avgLikes, relatedProfiles, every profile figure — on a listing whose runs
can only ever emit the 14 comment keys and the 5 error keys, and titled its
Output tab "Instagram profiles, posts and comments". A buyer reading that
page was being shown 59 columns their run would never fill. It now declares
19, is titled "Instagram comments with sentiment", and its second Console tab
shows what it has rather than fifteen mostly-blank profile columns. The base
manifest and instagram-scraper still declare all 78, because they really do
sell all four row shapes; instagram-profile-scraper declares its 30. A
column we have measured as unreachable logged out stays ABSENT — never null,
never 0, because both of those are answers and we do not have one.
"Profile Scraper with Sentiment" can emit a sentiment figure
(2026-09-15). It could not, on any path. The batch runner upgraded a post
summary to the detail record that carries the sentiment tallies only when
includeMediaUrls was set, so a listing sold on sentiment returned none
unless the buyer happened to also want expiring CDN URLs — two unrelated
things behind one flag. The upgrade is one dictionary lookup over a record
already in hand, so it now always happens, including for the posts nested in
latestPosts; includeMediaUrls governs the media URLs alone, which is what
its name says.
With the columns present, the first end-to-end run showed the other half of
the same bug: 0 / 0 / 0 on every post whose comments had not been walked
— nasa's three newest, carrying 4.309, 975 and 4.722 comments on Instagram.
services._distribute seeds its dict with all three labels at zero and
returns it whether or not anything was scored, so the mapper's if counts:
was always true. The REST surface answers this with comments_unlabelled
beside the tallies; a dataset row has no such column, so the three keys are
now written only when something was actually scored. Same rule as everywhere
else here: not measured means absent, not 0.
Eight MCP tools that could never be called are gone (2026-09-15).
build_mcp_server registered ten tools, decorated them, wrapped them in the
biller and then called remove_tool on eight at the end of every startup —
get_profile, list_posts, get_post, get_post_comments,
get_sentiment_summary, search_posts, request_collection and
get_collection_job. An MCP client now sees the two aggregate operations,
which submit one collection and do every page of pagination internally. The
REST routes behind the granular reads are untouched: a caller who wants one
piece of a profile still has one.
The HTTP JSONL data source is gone, with BackendTimeoutError, the five
backend_* settings and their two fields on the base form. Nothing deployed
it. DataSourceKind is now Literal["mock", "redis"] rather than leaving
"http" accepted: build_data_source ends in return MockDataSource(...),
so an operator who set it would have got a server that started, answered, and
served the bundled sample fixture.
isNewestComments is off both Console forms. The field is still accepted,
so an input copied from apify/instagram-comment-scraper runs unchanged, but
a tickbox whose own description reads "has no effect" is a control offered to
a buyer who then has to be told it was never a control.
Ten manifest defects fixed in passing, each of which had survived because
four copies had to agree and nothing read all four: dataType declared the
value "comment" on every listing while comment rows have never carried the
key; the base form defaulted addParentData to true against the model's
false; .actor/'s timestamp example still said +00:00 where the code
has emitted .000Z since 2026-09-04; pay_per_event.json shipped one
paragraph twice with no separating space on two of four copies; the scraper's
overview tab selected postUrl, a comment-only column, while having no
comments tab at all; two profile-form fields carried no editor key; the
comment listing's output_schema.json told buyers to set a resultsType its
form does not have; and all three public forms opened "What to collect" by
warning that a Standby run would ignore the fields below, on three Actors
whose own actor.json sets usesStandbyMode: false — that sentence now
lives in standbySectionDescription and the generator refuses to write the
word onto a listing that has no such mode. Two more were stale prose pointing
at things that are not there: the base form sent operators to "the
request_collection tool", one of the eight deleted above, and
comment_row's docstring said a caller needing Instagram's media id would
find it on post_row as postId, a key no builder has emitted since
2026-09-10 (it is id). The version is now single — pyproject.toml is read
by the generator — after /health spent some time reporting 1.0.0 for an
Actor published as 1.4.
views[*].display.properties is an object again (2026-09-15). The first
generator emitted it as an array of column names, which threw away the
{label, format} map on every tab of all four manifests — timestamp shown
as "Published" on a post tab and "Written" on a comment one, commentsCount
as "Declared" beside commentsCollected as "Held". The suite did not catch
it, because the only rule anything checked was "displayed is a subset of
selected" and a list of names satisfies that. Apify's build did, rejecting
the comment listing's push with
/views/comments/display/properties ... must be objectDisplay record on the View (the label belongs to the tab, not the column),
and test_a_views_display_properties_is_an_object_not_a_list asserts the type
the platform enforces — there is no offline validator for it, so the failed
build was the first thing that could have said so.
ENABLE_PPE is documented as what it charges. Its config.py description
and the README's configuration table both still said "one tool-call event
per MCP tool call", which stopped being how the charge worked when the
per-unit events landed: it is one event per RESULT delivered, at the rate for
profiles, posts or comments respectively. The Console form already said so.
A profile row carries the bio again (2026-09-14). biography,
profilePicUrl, externalUrl, externalUrls and highlightReelCount were
excluded by an allowlist at ingestion, deliberately and in writing, since
2026-08-10. Measured against the live build of
apify/instagram-profile-scraper on 2026-09-14, that left our profile row
sharing 13 of its 37 columns, and the two most visible losses — the bio and
the avatar — are in that Actor's own Console overview view, which is the
first thing a buyer sees. The data controller's decision, recorded as a dated
row in loggedout/COMPLIANCE.md §3 with the §4 legitimate-interest
assessment redone on the new basis, is output parity with the reference and
not one field further: public_email and public_phone_number arrive in
the same payload, the reference does not publish them, and they stay closed.
The change is at the allowlist (PROFILE_FIELDS), so all four profile
projections carry it or none do, and selftest.test_minimisation is what
says which.
Eight more columns on a post row (2026-09-14): alt, productType,
locationName/locationId/locationLat/locationLng, childPosts for a
carousel's slides, originalWidth/originalHeight, and
firstComment/latestComments. The last two cost nothing: the post response
already carried the first ~15 comments and the walk already parsed them, so
they were being dropped on the floor. What is still missing from the
reference row is now seven columns rather than twelve — isPinned,
isCommentsDisabled, videoDuration, videoPlayCount, musicInfo,
coauthorProducers, paidPartnership — and musicInfo and
coauthorProducers will not be declared until they have been probed against
a reel with attached audio and a co-authored post respectively. Guessing is
how a column ships permanently null.
A failed target used to write nothing at all (2026-09-14). A run over 100
URLs where three were unreachable produced 97 rows and no way to tell which
three, because the failure handler counted the target and moved on. All three
reference Actors emit an error row, so 100 inputs is always 100 rows out.
mapper.error_row now writes inputUrl, url, error, errorDescription
and requestErrorMessages through _prune_keeping, so the five columns are
present as explicit nulls on a run where nothing failed — an export builds its
columns from the keys it sees, and a consumer filtering on error must find
the column either way. Error rows are not charged. Refused targets get one
too; a refused input is the commonest failure there is.
relatedProfiles, off by default (2026-09-14). The accounts Instagram
suggests beside a profile, on a surface measured open on 2026-09-03 and left
unused since because COMPLIANCE.md §3 held an open question against it: it
hands out third parties' avatars for accounts nobody named. Resolved
affirmatively, and the cost is why it is opt-in rather than always on — one
extra request per profile, which loggedout/economics.py now reports as its
own named line rather than folding into the profile's cost. The reference
publishes the column unconditionally; includeRelatedProfiles is ours and is
declared as an addition.
Keyword search, and what it actually is (2026-09-15). search,
searchType and searchLimit were three entries in _FOREIGN_FIELDS —
refused by name with an explanation — and are now real fields. The surface
behind them is Instagram's popular-search feed, which ranks posts for a
word: coffee returns what Instagram ranks for coffee, not every post
carrying #coffee, and the field description says so rather than implying a
hashtag page. searchType: profile and user stay refused (account search is
behind the login curtain) and so does place (measured 2026-09-03: its cursor
returns the first page forever, so a two-page run would hand back the same
posts twice). The refusal moved from the FIELD to the VALUE, and only fires
when a term is actually present — which is what lets the reference Actor's own
example input, searchType set and search empty, run here unchanged.
Keyword results got their own Redis namespace rather than being filed under
the author's profile. A keyword's posts have many authors, and a one-post,
null-follower record filed under profile_key(author) would outrank a real
snapshot of that account; filed under post_key, "what did this word return"
becomes unanswerable. Keyword posts are always completed from their own pages,
because the feed node carries none of ten fields a post row declares, and a
column that is null for a reason invisible from the row is worse than a slower
run. Post rows from a keyword carry searchTerm, so a run mixing
directUrls and search stays attributable in a flat dataset. searchSource
was considered and not added: it would have been a column nobody measured.
Input parity, checked against the schemas instead of the screenshots (2026-09-15). Fetched from
GET https://api.apify.com/v2/acts/apify~<name>/builds/defaultapify/instagram-profile-scraper
publishes two input fields, apify/instagram-comment-scraper three,
apify/instagram-scraper eight — and every one of the thirteen is on the
matching form here. All three of their published example inputs run unchanged
and are pinned in tests/test_actor_variants.py. The divergences that remain
are deliberate and each has a measurement behind it: resultsType drops
mentions and stories, includeAboutSection: true is refused, and
searchType values other than hashtag are refused when a term is present.
skipPinnedPosts and addProfileStatistics appear in no published schema —
they came from a README FAQ, which is exactly why they are refused by name.
The price comparison in the Store blurbs was true on two of six plan tiers
(2026-09-15). All three actor.json descriptions read "25% below
apify/", flat and unqualified. Ours is one rate for everybody; theirs
runs down a tier ladder from Free to Diamond, so the claim reversed at Silver,
Gold or Platinum depending on the listing — up to +305% against
apify/instagram-scraper on Diamond. The descriptions now name the plan the
claim holds on, each README carries the full six-tier table, and
test_a_listing_description_names_the_plan_its_claim_is_true_on fails any
description that names a competitor without naming a tier.
Re-measuring also corrected a figure this repo had been asserting against:
apify/instagram-comment-scraper charges $2.60 on Free, not the $2.30
recorded on 2026-09-04 from its Store page in prose. $2.30 is their Bronze
rate. Our $1.725 is unchanged and is now 33% below their Free price rather
than 25%, and the two tests that guard the price were split accordingly: the
ceiling tracks the live market, the floor tracks the figures as they stood
when the price was signed. A competitor raising their price is not us
subsidising, and should not fail a test suite.
A configured Actor still ran as mock, because the build had frozen the
old environment (2026-09-11). Setting DATA_SOURCE and BROKER_URL in the
Console left three pushed Actors failing with "This Actor is not configured
to collect", while the Console showed both variables present and correct.
Apify bakes a version's environment variables into the image that version's
build produces, so the variables were real and the running image was the
one built before they existed. deploy/apify-env.sh now rebuilds every
version it writes to (useCache=true, so it costs seconds) and waits for
SUCCEEDED; -B opts out. deploy/deploy.sh down rebuilds for the
mirror-image reason and that one is a security fix: deleting BROKER_URL
from the version left the published image still carrying the Redis
credential, so an Actor reported as disconnected was still connected.
A 100-comment request answered in 73,8s and now answers in 17-24s
(2026-09-11). The collector, the proxies, Redis and the server were all fine:
the walk itself is ~8s and everything else was the sentiment gateway, which
streams ~3,3 output tokens/s per request, so BATCH=20/CONCURRENCY=3
made the labelling almost serial. Measured on 100 real comments, BATCH=5
with CONCURRENCY=20 is the knee (68,7s -> 16,7s; BATCH=3 buys no more
speed for half again as many input tokens, since the system prompt is re-sent
per batch). A process-wide MAX_IN_FLIGHT=40 semaphore was added at the same
time, because CONCURRENCY is per call and the worker runs comment walks up
to 24 wide — 24 x 20 would have been 480 requests at one host. Anything still
unlabelled after the concurrent pass is re-asked once over just the gaps: 20
runs of 100 comments, 7 extra requests in total, 0 rows shipped unlabelled.
The model is the remaining lever and is now reachable by environment alone
(SENTIMENT_MODEL, SENTIMENT_REASONING_EFFORT); the default is unchanged,
because which model labels a customer's comments is not a performance
decision. See loggedout/README.md for the tables.
Comment rows were rejected by the platform's dataset validation
(2026-09-10, three failed runs): the schema said repliesCount is a
number, replies an array, the sentiment pair a string/number, and
the row said null - always, for the first two, because Instagram renders no
replies to a logged-out reader; sometimes, for the sentiment pair, when the
gateway misses a batch. Apify validates every pushed item with AJV and
discards the whole request on the first bad one, so a 100-comment walk
ended in InvalidRequestError: Schema validation failed. Those five
columns (and ownerProfilePicUrl) are now ["<type>", "null"] in every
dataset schema, and tests/test_dataset_rows_validate.py runs every row
kind through every listing's schema with the same draft-07 validator.
The comment listing takes apify/instagram-comment-scraper's input
verbatim (2026-09-10): directUrls, resultsLimit,
includeNestedComments — three keys, no resultsType. The form IS those
three fields and nothing else, so the Console's JSON tab shows the same
object the reference shows; includeNestedComments is declared on it (off is
the reference's default and a normal run; on is still refused, with the
measurement behind the refusal). What makes the run a comment run moved
from the form's default into the manifest: DEFAULT_RESULTS_TYPE=comments
in actor.json environmentVariables, read by Settings and handed to
BatchInput as validation context — so the same input means the same run
on the platform, under apify run, and in tools/try_actor_rows.py, where
it used to come back as 500 post rows.
REST and MCP comment objects are the reference scraper's (2026-09-10):
id, text, timestamp (.000Z), ownerUsername, ownerProfilePicUrl,
postUrl, commentUrl, likesCount, then sentiment_label and
sentiment_score unchanged. The dataset row is the reference's eleven keys
plus sentimentLabel/sentimentScore, and nothing else.
There is a way to deploy this that is not a laptop (2026-09-10). Every
script that takes the code somewhere now lives in deploy/, behind one
entry point: ./deploy/deploy.sh up | down | status. apify-deploy-actor.sh
moved there from the repository root.
The point of the folder is that this deploys to TWO PLACES and they have to
agree. This Actor is one of them: apify push is the facade's deployment,
and it runs in Apify's cloud. The collector is the other: it runs on a
server, next to the Redis 8 the Actor reaches it through. A request travels
customer to Actor to Redis to worker to Instagram, and the processed record
comes back the same way. The Actor carries no collector code and the server
serves no HTTP, so the server has exactly one port open.
remote-deploy.sh builds the worker as a real image - it had never been one,
it was python:3.14-slim with the repo bind-mounted and three packages
installed from PyPI on every container start - and carries it to the server
with docker save | ssh docker load. No registry, so no account anywhere, no
credential on the server and no copy of the collector in a third party's
storage. Redis travels the same way, so the server needs no route to Docker
Hub at all.
The password the Actor authenticates with is minted at deploy time from
urandom, not typed by anyone, and written to deploy/secrets/<host>.env
(600, gitignored). One run installs it on the server and writes it onto each
pushed Actor version as a secret BROKER_URL. The two halves cannot
disagree, and nothing prints a credential - the reports show a hash
fingerprint, #b7aa2470 (64 chars), which tells you whether two hosts hold
the same secret and gives away none of it.
What changes for the Actors on the Store: nothing, and this adds no input.
What changes underneath is that DATA_SOURCE becomes redis rather than the
bundled fixtures, which is the setting that makes a published Actor collect
rather than replay. down removes it again, deliberately leaving
DATA_SOURCE=redis in place so a disconnected Actor FAILS on its next run
rather than falling back to mock and serving fixture data to a paying
customer who cannot tell.
Sentiment is now actually produced (2026-09-09). Both comment Actors are
named "with Sentiment" and, measured on 2026-09-09, neither wrote a label:
every comment row carried sentimentLabel: null. Two switches were off. The
worker ran with --no-sentiment because the local classifier needed ~1.1 GB
of torch weights its image never installed, and the facade never set
sentiment on the jobs it queued, so even a worker that could label would
not have been asked to.
The label now comes from a hosted chat model (DigitalOcean's inference
gateway, deepseek-4-flash) instead of a local model, the facade asks for it
on every job that asks for comments, and the worker probes the gateway once
at startup and warns if it does not answer. Only the comment text is sent -
no usernames, ids or posts - in batches of 20; loggedout/COMPLIANCE.md
records the change of character, since text now leaves the machine. The
profile Actor is untouched: it never walks comments, so it never scores them.
What a row carries is unchanged: sentimentLabel in positive, neutral,
negative and sentimentScore in [0, 1]. The score is the model's stated
confidence rather than a softmax, so rank on it rather than threshold it.
A comment the gateway could not read stays null, never a guessed label.
Post rows carry the fields the profile grid never had (2026-09-08). A post
listed from an account came back with timestamp, likesCount,
commentsCount, videoViewCount, dimensions and taggedUsers all null or
absent, and nothing said why. The cause is the surface: Instagram's logged-out
profile query draws thumbnails, so it selects a caption, a shortcode, a media
id and one image, and nothing else. taggedUsers was the visible half of it -
p/DZfnXTHHfUa has @gucci tagged INSIDE the picture and a caption that is one
emoji, so a run listing 100 of @kimkardashian's posts reported the brand
nowhere, while the same post's own page names it.
Each listed post is now read from its own page, which has all of it. The new
enrichPosts input turns this off; it is on by default, because a row missing
half its columns is not a cheaper row, it is a wrong one. The price is one
request per post against roughly one per twelve to list them - 100 posts is
~9 requests listed and ~109 filled in - so a run that wants shortcodes and
captions fast should turn it off and expect those fields null.
Only the holes are filled: a count that came from a surface that measured it is never overwritten by a second, later reading, and a post that refuses keeps its nulls rather than gaining zeroes.
"There are no more comments" is now checked before it is believed
(2026-09-07). The worst of the four causes of a short answer, and the last one
found, because it does not look like a failure from the inside: Instagram
answers has_next_page: false while the connection still has thousands of
comments behind it. Four runs against p/Dc6zTucHTv8 within an hour ended at
140, 541, 1.000 and 3.379 comments; a single pinned session walking the same
post reached 3.379 over 256 pages. The 140 went out in a record saying
"Instagram's logged-out connection ended after 140 of 4314" - a statement
about Instagram that was not true, and that no consumer could check.
The walk now re-asks the cursor that announced the end, twice, before accepting it, and resumes if either answer disagrees. The same guard is on the posts timeline, which runs the same connection pattern. Re-asked pages are deduplicated by comment id and shortcode, so confirming an end can cost requests but never rows. Measured after the change: three consecutive runs asking for 1.000 comments on that post returned 1.000, 1.000 and 1.000.
Two consequences worth stating plainly. A capped ask - "give me N" - is now
exact whenever the post holds N. An UNCAPPED ask - "give me everything" - is a
lower bound and cannot be anything else: edges: [] with no next page is the
only end-of-data signal the surface has, and it was measured arriving at 512,
932 and 1.005 comments on one post on three different days. Numbers this
project published as "all a logged-out reader can see" were lower bounds
misread as ceilings, and have been corrected.
Ask for N results and get N (2026-09-07). Three things stood between the input schema and that sentence, and all three are fixed:
resultsLimit: 200 reached list_posts(limit=200) and came back
"limit must be between 1 and 50, got 200"; 200 comments hit the same wall at
100. The read API pages - it is the same call the MCP tools make - so the
run now pages through it, and the ceilings the caller was never shown stay
where they belong. A run that asks for more than exists still returns
everything that exists: 1.000 comments requested against a post holding 145
is 145 rows and no error.fetch() gives every request four attempts
from four addresses, and the fifth failure used to end the walk with
whatever had arrived - so "1.000 comments" could come back 512 because one
page in the middle was refused, which reads exactly like a post that only
has 512. A cursor now gets three further rounds, each after a cooldown that
starts at half a minute and grows, before the walk is declared over. It
costs nothing when nothing is wrong (four attempts have to fail first) and
it lowers the request rate on an address that just refused one, which is
what the pacing governor does with the same evidence.PostsPage carries a truncation_note for the timeline half of the same
question, fed by the collector's _posts_truncated.The login curtain is remembered for ten minutes per country (2026-09-07),
which is a collector change with a facade-visible effect: a run of forty
handles used to spend eight refusals and about four minutes per handle
rediscovering that web_profile_info and user_feed answer require_login
to logged-out callers. Measured on three handles in one run: 8 refusals for
the first, 0 for the other two, 199s against the ~330s the same run cost
before. The memory is short on purpose - the day the curtain lifts, posts
start carrying like and comment counts again with nobody editing anything -
and an ordinary refusal is not remembered at all, only Instagram's own
require_login.
Every rate is now exactly 25% below the Actor it competes with
(2026-09-04), raised from 10%, and each listing is priced against its own
counterpart rather than all three against one. The reference figures are the
vendors' undiscounted Free-plan prices, read that day from the Store pages —
/v2/acts/apify~<name>/builds/default returns eventPriceUsd: null to an
anonymous caller, so the page is the only source:
instagram-scraper $2.025 / 1.000 results, against $2.70.instagram-profile-scraper $1.95 / 1.000 profiles, against $2.60.instagram-comment-scraper $1.725 / 1.000 comments, against $2.30.Each Store listing writes one shape of row per run, so each carries its own
pay_per_event.json with all four events at that single rate — a run's price
can be quoted before it starts. The table in .actor/, which is the MCP and
REST listing and answers profiles, posts and comments in one call, keeps a
rate per event instead: $0.00195, $0.002025 and $0.001725 respectively, each
25% below that row's own counterpart.
A third Store listing: instagram-profile-scraper (2026-09-04), mirroring
apify/instagram-profile-scraper rather than the general scraper. Its form
asks for usernames and nothing else; a run answers one row per account with
its recent posts nested in a new latestPosts column, and no separate post
rows. usernames implies resultsType: details — an explicit resultsType
still wins — and resultsLimit says how many posts to nest. The definition
that used to sit in actors/instagram-profile-scraper/ was the general
scraper under a misleading name and is now actors/instagram-scraper/.
includeAboutSection, the reference's paid extra, is refused with its reason:
date joined, country of registration and broadcast channel are not on the
logged-out surfaces, so accepting it would bill for an empty panel.
The dataset rows were checked field for field against the two reference
Actors, and five names or shapes were changed to match (2026-09-04). The
comparison was made against apify/instagram-scraper and
apify/instagram-comment-scraper's own machine-readable dataset schemas and
their published output samples, read on 2026-09-04 from
api.apify.com/v2/acts/apify~<name>/builds/default — not from prose, which
in both cases is out of date. What moved:
images is a list of URL strings, not of objects. That is the shape
apify/instagram-scraper writes into a column of that name, and the near
miss was the dangerous kind: same name, same type at the top level, and
row.images[0] silently a dict instead of a URL. The per-rendition
measurements are not lost — they move intact to mediaAssets
(url, width, height, isVideo), a name no reference scraper uses.isPrivate is now private and categoryName is now
businessCategoryName, the spellings the reference profile row uses.
This row already spelt verified their way rather than isVerified, so
two halves of one convention had drifted apart.commentUrl: the post permalink, /c/, then the
comment id, with no trailing slash — the shape the reference Actor emits
(its own schema example shows a trailing slash; its output does not).
Derived from two columns already on the row, so nothing is collected for
it. Absent when the snapshot carries no comment id.postId, Instagram's media id for the post the
comment sits under, as a string. The current reference comment scraper has
no such column; a postId it published until 2025 held a shortcode, and
ours does not — the shortcode is in postShortCode next to it.collectedAt on a post row is …T18:01:36.000Z, like every other
timestamp this Actor writes. It was passed through unnormalised, so one run
put +00:00 on post rows and .000Z on profile and comment rows of the
same dataset.Two more of the reference scraper's inputs are refused by name instead of
ignored (2026-09-04): skipPinnedPosts and addProfileStatistics. Both are
documented in apify/instagram-scraper's README FAQ and in neither of its
input schemas, which is exactly what made them dangerous here - a caller
copies the key out of that FAQ, extra="ignore" deletes it, and the run
reports success. skipPinnedPosts is the sharper of the two: this Actor does
not collect whether a post is pinned, so the pinned posts come back either way
and only the flag says otherwise.
The deploy contract for the two Store definitions is written down and
tested (2026-09-04). apify push reads .actor/actor.json and nothing
else, so publishing either Actor under actors/ means putting that
directory's contents at facade/.actor/ first. The ../Dockerfile and
../CHANGELOG.md in those manifests are written for that destination and
resolve nowhere from where the files sit in the repo - correcting them in
place would break the push. Neither definition carries pay_per_event.json,
which lives only in .actor/ and has to be brought along.
The type column's declared values in .actor/dataset_schema.json were
wrong (2026-09-04) — image/video/carousel, this repo's internal
vocabulary, where the rows have always carried the Store's Image/Video/
Sidecar. The two Actor definitions under actors/ were already right. The
schemas under actors/ had no test over them at all until today; three now
check that both declare every column the mapper can write, that their views
select only declared fields, and that a column's type and enum do not depend
on which Actor page you read it from.
Comment rows carry id (2026-09-04): Instagram's identifier for the
comment itself, as a string, matching the reference comment scraper. It is
the comment's id, not the commenter's account id, which stays uncollected.
Absent on snapshots collected before this date. The other fields the node
carries (repliesCount, parent id, restriction flags) were measured null
or constant on every logged-out node and are deliberately not shipped: a
column that can only ever say 0 would read as "no replies", which the
logged-out surface never measures.
A comment list Instagram cut short says so (2026-09-04). A post's
commentsCount counts replies and comments the logged-out view does not
serve: measured on a post declaring 5,156, the connection served 512 and
then answered has_next_page: false. That is not a truncation — nothing was
capped and nothing failed — so the record now carries _comments_partial
in the collector's words and the comments page repeats it in
truncation_note, instead of leaving "512 of 5,156" to be read as a
collector that quit early.
postsCount, followersCount and followsCount are absent when not
measured (2026-09-04). postsCount used to fall back to the number of posts
the run held, which published postsCount: 1 for an account with thousands of
posts whenever the logged-out header omitted the total, and the two audience
counts were written as 0 on a post URL's owner. A count the surface did not
carry is now null in the REST/MCP profile and an absent column in the
dataset; postsInSnapshot is the number of posts held and always present.
Single post and reel URLs are inputs. directUrls used to refuse them by name,
because the query behind one post was reachable only to a signed-in visitor;
re-measured on 3 September 2026 it answers a logged-out one, so a post URL is a
target rather than a refusal.
https://www.instagram.com/p/DCMUrLltVlM/ and its /reel/, /reels/,
/tv/ and /<username>/p/<code>/ spellings all name one post, and one list
may mix post URLs with profile URLs and bare handles. The same post named
twice is collected once, and deduplication is on the shortcode rather than on
the URL, so three spellings of one post are one collection and one invoice.resultsType over a post URL: posts returns that one post,
reels returns it only if it is a video (a photo yields zero rows and a line
in the log saying the filter matched nothing, which is not a failure),
comments returns its comments with resultsLimit as the number taken.
commentsPostLimit is inert for a post URL — there is one post to read.resultsType: details with a post URL is refused with a reason, per URL,
and the rest of the list still runs. details asks for an account's own
figures — followers, follows, total posts, engagement averages — and the
single-post response carries the owner's identity and nothing else about
them. A profile row built from it would report a missing follower count,
which a consumer reads as "none". The refusal names the account URL that
would answer instead.includeNestedComments is refused when true, rather than ignored. There
is no logged-out query that returns comment replies — measured across
Instagram's own JavaScript bundles on 3 September 2026 — and accepting the
flag would make a run look like it had walked the threads and found them
empty. isNewestComments is accepted and inert: comments arrive in the
order Instagram's connection serves them and are not reordered here. It said
so on the form until 1.5, which removed the control and kept the field.hashtags,
mentions, taggedUsers, dimensionsWidth and dimensionsHeight.
hashtags and mentions are cut out of the caption already on the row —
Unicode, so #città is a hashtag; in the order written; case-insensitive
duplicates removed keeping the first spelling, because #Coffee and
#coffee open one page on Instagram and counting them twice skews every
group-by. An @ inside a word is an address and not a mention, so
hello@roasters.coffee publishes nobody. Both are always present, empty list
included: the caption is on the row, so "no hashtags" is something this row
knows.taggedUsers is the accounts tagged INSIDE the picture, and it is
minimised. Each entry is {username, id, isVerified, imageIndex, position}
— imageIndex is which frame of a carousel the tag sits on, 0-based, and
position is Instagram's normalised [x, y]. The tagged account's display
name and profile picture are not collected, though the payload carries
them and the other Instagram scrapers publish them: a tag says which account
an image references, which the handle and the id say completely, and a name
and a face are personal data about somebody who is not even the profile that
was asked for. Recorded as a data-protection decision by the controller, with
its reasoning, in the collector's own COMPLIANCE.md §3 (row 2026-09-03).taggedUsers, dimensionsWidth and dimensionsHeight are absent rather
than empty when the surface carried none. Posts read from an account's
logged-out grid have no tags and no frame size in the response at all —
checked on the selection set of Instagram's own query, not inferred from an
empty answer — so those columns are left out. [] would read as "this
picture tags nobody" and 0 as a zero-pixel image; neither was measured. A
post collected by its own URL carries all three.Compatibility with the Instagram scrapers already on the Store, at the value
level rather than only in the field names. Measured against
apify/instagram-scraper and apify/instagram-comment-scraper on 2 September
2026: 14 of the post fields were already spelled identically, and two of the
values underneath them were not.
type is now Image / Video / Sidecar, was image / video /
carousel. The words differ, not just the case: a consumer switching on
"Sidecar" fell through every carousel silently. The HTTP and MCP surfaces
keep the domain's spelling — the translation is in mapper.py, at the edge
that faces the dataset, and nowhere else.Z: 2026-08-16T18:01:36.000Z, was
2026-08-16T18:01:36+00:00. The same instant, and every ISO 8601 parser
reads both, but the shape a string-slicing consumer was written against is
the Store's. Applies to timestamp and collectedAt. A non-UTC offset is
converted rather than relabelled; an unparseable value survives unchanged
rather than disappearing.addParentData now defaults to false, was true.
apify/instagram-scraper defaults it to false, and one input object must not
mean two output shapes depending on which Actor reads it. A run that was
relying on ownerUsername, ownerId and ownerFullName arriving unasked
must now set addParentData: true explicitly.search, searchType and
searchLimit say that targets are named in directUrls here, and
resultsType: "stories" or "mentions" names the perimeter rather than
reading as a typo. Validation messages no longer carry pydantic's
Value error, prefix and now name the field they are about.Not changed: comment rows still carry no id. The collector discards the
comment pk at projection (loggedout/endpoints.py, COMMENT_FIELDS), and
COMPLIANCE.md §3 records that tuple's contents as a data-protection decision
rather than a code change. Adding it needs a row in that file's decision table.
Store readiness. The Actor was a server on a marketplace of scrapers: a normal run printed usage and exited, which is what "Try for free" started, and there was no output schema — which the platform requires before an Actor can be published at all.
directUrls, resultsType
(details/posts/reels/comments), resultsLimit, commentsPostLimit,
onlyPostsNewerThan, includeMediaUrls and addParentData drive a batch
collection that writes one flat row per result to the default dataset.
Standby mode is unchanged and still serves MCP and REST over the same data.shortCode, likesCount, ownerUsername, ids as strings, a hidden like
count as -1), so an input object written for one of those can be pasted in..actor/output_schema.json and .actor/dataset_schema.json: 45 documented
fields and three views — an overview, posts, and comments with their
sentiment./tagged/ page each
come back with an explanation. One bad profile in a list no longer costs the
rest of the run.tool-call event for everything, so a comment — a step down a paced,
paginated thread — billed at the same price as a post, and a collection
bundle carrying 2.400 comments billed as a single result. Profiles, posts and
comments are now separate events; collect_posts_with_comments charges for
both units it returns; asking for a collection and polling it are free.First public release.
POST /mcp and a REST API under
/profiles/..., both thin facades over the same ProfileService: profile
metadata, posts, one post in depth, comments, sentiment, caption search,
requesting a collection and following one.request_collection waits for the job it queues (default 90 s, tunable via
brokerWaitMs) and usually returns the finished snapshot in a single call;
a walk too long to wait for comes back as a job id to poll instead.mock (bundled fixture, the default), http (a JSONL
endpoint) and redis (the collector behind a broker, which can trigger a
fresh collection on a cache miss).tool-call event per successful MCP tool call, refusing further calls
once the run's spending limit is reached.