# Changelog of Instagram Comment Scraper with Sentiment (`omaraw/instagram-comment-scraper`) Actor - **URL**: https://apify.com/omaraw/instagram-comment-scraper/changelog.md - **Full Actor documentation**: https://apify.com/omaraw/instagram-comment-scraper.md ## Changelog Notable changes to this Actor, newest first. Version numbers follow the platform's two-part `MAJOR.MINOR` format. ### 1.5 — 2026-09-15 - **Sentiment is the comment Actor's alone** (2026-09-16). Instagram Scraper and Instagram Profile Scraper drop "with Sentiment" from their titles, pages, schemas and tabs, and their runs no longer write `sentimentPositive`/ `Neutral`/`Negative` or `sentimentLabel`/`Score` even when the snapshot holds labels. The switch is `SENTIMENT_ENABLED`, written into every Store listing's `actor.json` by the generator; the collector still scores comments, so a snapshot shared between listings is labelled for the one that sells it. - **Store prices are pushed by `tools/apify_pricing.py`** (2026-09-16). `pay_per_event.json` never reached the platform, so all three listings showed "Pay per usage". Event titles are bare nouns now so the badge reads "$1.95 / 1,000 profiles". - **One price, $1.75 per 1,000 results, on every Actor** (2026-09-17, extended to the base MCP-server manifest 2026-09-18). `unit_prices_usd` is gone and `price_usd` in `tools/actor_manifests/listings.py` is the whole pricing model: the base manifest used to charge $0.00195 / $0.002025 / $0.001725 by row shape, so one MCP call's invoice depended on what its answer happened to contain. The listing pages name no plan tiers of ours at all - there are none - and no longer reprint the counterparts' discount ladders; they state the flat rate, the percentage below the counterpart's Free-plan price, and one clause saying that rate of theirs moves with the buyer's subscription while ours does not. It replaces both the old per-listing prices ($2.025 / $1.95 / $1.725) and a four-tier ladder drafted on the 17th and reverted the same day, which never reached the platform. $1.75 is 35% below what `apify/instagram-scraper` charges its Free-plan buyers and 32% below the profile and comment counterparts', and it does not move with anything — so the figure in a description holds for whoever reads it. It is not a saving against every rate they charge: their tiers dip under $1.75 for their larger subscribers (from Gold for the general and profile counterparts, from Platinum for the comment one). That is recorded internally — `CROSSES_AT` in `tests/test_actor_variants.py` fails the suite if a counterpart discounts further and a claim goes stale — and stated on the listing pages as one clause rather than a reprint of their ladder. apify 4.0.2 will not bill an event whose run pricing arrives as a tier table, and the biller still logs an error the first time it sees one: dead code at a flat price, kept for the next time tiers are considered. - **A run never writes rows its spending limit will not pay for** (2026-09-17). Rows were pushed in chunks of 100 and charged afterwards, so the chunk that crossed `ACTOR_MAX_TOTAL_CHARGE_USD` landed whole and was billed in part: up to 99 rows free per run. Each chunk is now trimmed first to what the SDK's charging manager says the budget covers, the same count `Actor.push_data(charged_event_name=...)` would push. The price push also sets `minimalMaxTotalChargeUsd` to one primary result, so no user can start a run whose limit cannot pay for a single row. - **The four Apify manifests are generated now, not copied** (2026-09-15). `actor.json`, `input_schema.json`, `dataset_schema.json`, `output_schema.json` and `pay_per_event.json` existed as four hand-maintained near-copies under `.actor/` and `actors/*/` — around 8.000 lines, of which the dataset schema alone was ~2.050 lines of byte-identical paste encoding three real decisions. `tools/actor_manifests/` replaces them with three tables: `columns.py` (every dataset column once, its prose carried across character for character), `form.py` (every input field, with the wordings each listing gives it) and `listings.py` (which of them each Actor shows, in what order, at what price). `python -m tools.actor_manifests --check` is the drift gate and is a test. Editing the JSON by hand now survives exactly until the next regeneration, which is why the generator, not the output, is where a change goes. - **Each listing declares only the columns its own runs can produce** (2026-09-15). The comment Actor declared all 78 columns — `latestPosts`, `avgLikes`, `relatedProfiles`, every profile figure — on a listing whose runs can only ever emit the 14 comment keys and the 5 error keys, and titled its Output tab "Instagram profiles, posts and comments". A buyer reading that page was being shown 59 columns their run would never fill. It now declares 19, is titled "Instagram comments with sentiment", and its second Console tab shows what it has rather than fifteen mostly-blank profile columns. The base manifest and `instagram-scraper` still declare all 78, because they really do sell all four row shapes; `instagram-profile-scraper` declares its 30. A column we have measured as unreachable logged out stays ABSENT — never null, never 0, because both of those are answers and we do not have one. - **"Profile Scraper with Sentiment" can emit a sentiment figure** (2026-09-15). It could not, on any path. The batch runner upgraded a post summary to the detail record that carries the sentiment tallies only when `includeMediaUrls` was set, so a listing sold on sentiment returned none unless the buyer happened to also want expiring CDN URLs — two unrelated things behind one flag. The upgrade is one dictionary lookup over a record already in hand, so it now always happens, including for the posts nested in `latestPosts`; `includeMediaUrls` governs the media URLs alone, which is what its name says. With the columns present, the first end-to-end run showed the other half of the same bug: **`0 / 0 / 0` on every post whose comments had not been walked** — nasa's three newest, carrying 4.309, 975 and 4.722 comments on Instagram. `services._distribute` seeds its dict with all three labels at zero and returns it whether or not anything was scored, so the mapper's `if counts:` was always true. The REST surface answers this with `comments_unlabelled` beside the tallies; a dataset row has no such column, so the three keys are now written only when something was actually scored. Same rule as everywhere else here: not measured means absent, not `0`. - **Eight MCP tools that could never be called are gone** (2026-09-15). `build_mcp_server` registered ten tools, decorated them, wrapped them in the biller and then called `remove_tool` on eight at the end of every startup — `get_profile`, `list_posts`, `get_post`, `get_post_comments`, `get_sentiment_summary`, `search_posts`, `request_collection` and `get_collection_job`. An MCP client now sees the two aggregate operations, which submit one collection and do every page of pagination internally. **The REST routes behind the granular reads are untouched**: a caller who wants one piece of a profile still has one. - **The HTTP JSONL data source is gone**, with `BackendTimeoutError`, the five `backend_*` settings and their two fields on the base form. Nothing deployed it. `DataSourceKind` is now `Literal["mock", "redis"]` rather than leaving `"http"` accepted: `build_data_source` ends in `return MockDataSource(...)`, so an operator who set it would have got a server that started, answered, and served the bundled sample fixture. - **`isNewestComments` is off both Console forms.** The field is still accepted, so an input copied from `apify/instagram-comment-scraper` runs unchanged, but a tickbox whose own description reads "has no effect" is a control offered to a buyer who then has to be told it was never a control. - **Ten manifest defects fixed in passing**, each of which had survived because four copies had to agree and nothing read all four: `dataType` declared the value `"comment"` on every listing while comment rows have never carried the key; the base form defaulted `addParentData` to `true` against the model's `false`; `.actor/`'s `timestamp` example still said `+00:00` where the code has emitted `.000Z` since 2026-09-04; `pay_per_event.json` shipped one paragraph twice with no separating space on two of four copies; the scraper's overview tab selected `postUrl`, a comment-only column, while having no comments tab at all; two profile-form fields carried no `editor` key; the comment listing's `output_schema.json` told buyers to set a `resultsType` its form does not have; and **all three public forms opened "What to collect" by warning that a Standby run would ignore the fields below, on three Actors whose own `actor.json` sets `usesStandbyMode: false`** — that sentence now lives in `standbySectionDescription` and the generator refuses to write the word onto a listing that has no such mode. Two more were stale prose pointing at things that are not there: the base form sent operators to "the `request_collection` tool", one of the eight deleted above, and `comment_row`'s docstring said a caller needing Instagram's media id would find it on `post_row` as `postId`, a key no builder has emitted since 2026-09-10 (it is `id`). The version is now single — `pyproject.toml` is read by the generator — after `/health` spent some time reporting 1.0.0 for an Actor published as 1.4. - **`views[*].display.properties` is an object again** (2026-09-15). The first generator emitted it as an array of column names, which threw away the `{label, format}` map on every tab of all four manifests — `timestamp` shown as "Published" on a post tab and "Written" on a comment one, `commentsCount` as "Declared" beside `commentsCollected` as "Held". The suite did not catch it, because the only rule anything checked was "displayed is a subset of selected" and a list of names satisfies that. **Apify's build did**, rejecting the comment listing's push with `/views/comments/display/properties ... must be object`. The labels are restored byte for byte, the map now lives in a `Display` record on the `View` (the label belongs to the tab, not the column), and `test_a_views_display_properties_is_an_object_not_a_list` asserts the type the platform enforces — there is no offline validator for it, so the failed build was the first thing that could have said so. - **`ENABLE_PPE` is documented as what it charges.** Its `config.py` description and the README's configuration table both still said "one `tool-call` event per MCP tool call", which stopped being how the charge worked when the per-unit events landed: it is one event per RESULT delivered, at the rate for profiles, posts or comments respectively. The Console form already said so. ### 1.4 — 2026-09-14 - **A profile row carries the bio again** (2026-09-14). `biography`, `profilePicUrl`, `externalUrl`, `externalUrls` and `highlightReelCount` were excluded by an allowlist at ingestion, deliberately and in writing, since 2026-08-10. Measured against the live build of `apify/instagram-profile-scraper` on 2026-09-14, that left our profile row sharing 13 of its 37 columns, and the two most visible losses — the bio and the avatar — are in that Actor's own Console *overview* view, which is the first thing a buyer sees. The data controller's decision, recorded as a dated row in `loggedout/COMPLIANCE.md` §3 with the §4 legitimate-interest assessment redone on the new basis, is **output parity with the reference and not one field further**: `public_email` and `public_phone_number` arrive in the same payload, the reference does not publish them, and they stay closed. The change is at the allowlist (`PROFILE_FIELDS`), so all four profile projections carry it or none do, and `selftest.test_minimisation` is what says which. - **Eight more columns on a post row** (2026-09-14): `alt`, `productType`, `locationName`/`locationId`/`locationLat`/`locationLng`, `childPosts` for a carousel's slides, `originalWidth`/`originalHeight`, and `firstComment`/`latestComments`. The last two cost nothing: the post response already carried the first ~15 comments and the walk already parsed them, so they were being dropped on the floor. What is still missing from the reference row is now seven columns rather than twelve — `isPinned`, `isCommentsDisabled`, `videoDuration`, `videoPlayCount`, `musicInfo`, `coauthorProducers`, `paidPartnership` — and `musicInfo` and `coauthorProducers` will not be declared until they have been probed against a reel with attached audio and a co-authored post respectively. Guessing is how a column ships permanently null. - **A failed target used to write nothing at all** (2026-09-14). A run over 100 URLs where three were unreachable produced 97 rows and no way to tell which three, because the failure handler counted the target and moved on. All three reference Actors emit an error row, so 100 inputs is always 100 rows out. `mapper.error_row` now writes `inputUrl`, `url`, `error`, `errorDescription` and `requestErrorMessages` through `_prune_keeping`, so the five columns are present as explicit nulls on a run where nothing failed — an export builds its columns from the keys it sees, and a consumer filtering on `error` must find the column either way. Error rows are **not charged**. Refused targets get one too; a refused input is the commonest failure there is. - **`relatedProfiles`, off by default** (2026-09-14). The accounts Instagram suggests beside a profile, on a surface measured open on 2026-09-03 and left unused since because `COMPLIANCE.md` §3 held an open question against it: it hands out third parties' avatars for accounts nobody named. Resolved affirmatively, and the cost is why it is opt-in rather than always on — one extra request per profile, which `loggedout/economics.py` now reports as its own named line rather than folding into the profile's cost. The reference publishes the column unconditionally; `includeRelatedProfiles` is ours and is declared as an addition. - **Keyword search, and what it actually is** (2026-09-15). `search`, `searchType` and `searchLimit` were three entries in `_FOREIGN_FIELDS` — refused by name with an explanation — and are now real fields. The surface behind them is Instagram's popular-search feed, which **ranks posts for a word**: `coffee` returns what Instagram ranks for coffee, not every post carrying `#coffee`, and the field description says so rather than implying a hashtag page. `searchType: profile` and `user` stay refused (account search is behind the login curtain) and so does `place` (measured 2026-09-03: its cursor returns the first page forever, so a two-page run would hand back the same posts twice). The refusal moved from the FIELD to the VALUE, and only fires when a term is actually present — which is what lets the reference Actor's own example input, `searchType` set and `search` empty, run here unchanged. Keyword results got their own Redis namespace rather than being filed under the author's profile. A keyword's posts have many authors, and a one-post, null-follower record filed under `profile_key(author)` would outrank a real snapshot of that account; filed under `post_key`, "what did this word return" becomes unanswerable. Keyword posts are always completed from their own pages, because the feed node carries none of ten fields a post row declares, and a column that is null for a reason invisible from the row is worse than a slower run. Post rows from a keyword carry **`searchTerm`**, so a run mixing `directUrls` and `search` stays attributable in a flat dataset. `searchSource` was considered and not added: it would have been a column nobody measured. - **Input parity, checked against the schemas instead of the screenshots** (2026-09-15). Fetched from `GET https://api.apify.com/v2/acts/apify~/builds/default`, which answers anonymous callers for public Actors. `apify/instagram-profile-scraper` publishes **two** input fields, `apify/instagram-comment-scraper` **three**, `apify/instagram-scraper` **eight** — and every one of the thirteen is on the matching form here. All three of their published example inputs run unchanged and are pinned in `tests/test_actor_variants.py`. The divergences that remain are deliberate and each has a measurement behind it: `resultsType` drops `mentions` and `stories`, `includeAboutSection: true` is refused, and `searchType` values other than `hashtag` are refused when a term is present. `skipPinnedPosts` and `addProfileStatistics` appear in no published schema — they came from a README FAQ, which is exactly why they are refused by name. - **The price comparison in the Store blurbs was true on two of six plan tiers** (2026-09-15). All three `actor.json` descriptions read "25% below apify/", flat and unqualified. Ours is one rate for everybody; theirs runs down a tier ladder from Free to Diamond, so the claim reversed at Silver, Gold or Platinum depending on the listing — up to **+305%** against `apify/instagram-scraper` on Diamond. The descriptions now name the plan the claim holds on, each README carries the full six-tier table, and `test_a_listing_description_names_the_plan_its_claim_is_true_on` fails any description that names a competitor without naming a tier. Re-measuring also corrected a figure this repo had been asserting against: `apify/instagram-comment-scraper` charges **$2.60** on Free, not the $2.30 recorded on 2026-09-04 from its Store page in prose. $2.30 is their *Bronze* rate. Our $1.725 is unchanged and is now 33% below their Free price rather than 25%, and the two tests that guard the price were split accordingly: the ceiling tracks the live market, the floor tracks the figures as they stood when the price was signed. A competitor raising their price is not us subsidising, and should not fail a test suite. ### 1.3 — 2026-09-03 - **A configured Actor still ran as `mock`, because the build had frozen the old environment** (2026-09-11). Setting `DATA_SOURCE` and `BROKER_URL` in the Console left three pushed Actors failing with "This Actor is not configured to collect", while the Console showed both variables present and correct. Apify bakes a version's environment variables into the image that version's build produces, so the variables were real and the *running image* was the one built before they existed. `deploy/apify-env.sh` now rebuilds every version it writes to (`useCache=true`, so it costs seconds) and waits for `SUCCEEDED`; `-B` opts out. `deploy/deploy.sh down` rebuilds for the mirror-image reason and that one is a security fix: deleting `BROKER_URL` from the version left the published image still carrying the Redis credential, so an Actor reported as disconnected was still connected. - **A 100-comment request answered in 73,8s and now answers in 17-24s** (2026-09-11). The collector, the proxies, Redis and the server were all fine: the walk itself is ~8s and everything else was the sentiment gateway, which streams ~3,3 output tokens/s **per request**, so `BATCH=20`/`CONCURRENCY=3` made the labelling almost serial. Measured on 100 real comments, `BATCH=5` with `CONCURRENCY=20` is the knee (68,7s -> 16,7s; `BATCH=3` buys no more speed for half again as many input tokens, since the system prompt is re-sent per batch). A process-wide `MAX_IN_FLIGHT=40` semaphore was added at the same time, because `CONCURRENCY` is per call and the worker runs comment walks up to 24 wide — 24 x 20 would have been 480 requests at one host. Anything still unlabelled after the concurrent pass is re-asked once over just the gaps: 20 runs of 100 comments, 7 extra requests in total, 0 rows shipped unlabelled. The model is the remaining lever and is now reachable by environment alone (`SENTIMENT_MODEL`, `SENTIMENT_REASONING_EFFORT`); the default is unchanged, because which model labels a customer's comments is not a performance decision. See `loggedout/README.md` for the tables. - **Comment rows were rejected by the platform's dataset validation** (2026-09-10, three failed runs): the schema said `repliesCount` is a `number`, `replies` an `array`, the sentiment pair a `string`/`number`, and the row said null - always, for the first two, because Instagram renders no replies to a logged-out reader; sometimes, for the sentiment pair, when the gateway misses a batch. Apify validates every pushed item with AJV and discards the whole request on the first bad one, so a 100-comment walk ended in `InvalidRequestError: Schema validation failed`. Those five columns (and `ownerProfilePicUrl`) are now `["", "null"]` in every dataset schema, and `tests/test_dataset_rows_validate.py` runs every row kind through every listing's schema with the same draft-07 validator. - **The comment listing takes `apify/instagram-comment-scraper`'s input verbatim** (2026-09-10): `directUrls`, `resultsLimit`, `includeNestedComments` — three keys, no `resultsType`. The form IS those three fields and nothing else, so the Console's JSON tab shows the same object the reference shows; `includeNestedComments` is declared on it (off is the reference's default and a normal run; on is still refused, with the measurement behind the refusal). What makes the run a comment run moved from the form's `default` into the manifest: `DEFAULT_RESULTS_TYPE=comments` in `actor.json` `environmentVariables`, read by `Settings` and handed to `BatchInput` as validation context — so the same input means the same run on the platform, under `apify run`, and in `tools/try_actor_rows.py`, where it used to come back as 500 post rows. - **REST and MCP comment objects are the reference scraper's** (2026-09-10): `id`, `text`, `timestamp` (`.000Z`), `ownerUsername`, `ownerProfilePicUrl`, `postUrl`, `commentUrl`, `likesCount`, then `sentiment_label` and `sentiment_score` unchanged. The dataset row is the reference's eleven keys plus `sentimentLabel`/`sentimentScore`, and nothing else. - **There is a way to deploy this that is not a laptop** (2026-09-10). Every script that takes the code somewhere now lives in `deploy/`, behind one entry point: `./deploy/deploy.sh up | down | status`. `apify-deploy-actor.sh` moved there from the repository root. The point of the folder is that this deploys to TWO PLACES and they have to agree. This Actor is one of them: `apify push` is the facade's deployment, and it runs in Apify's cloud. The collector is the other: it runs on a server, next to the Redis 8 the Actor reaches it through. A request travels customer to Actor to Redis to worker to Instagram, and the processed record comes back the same way. The Actor carries no collector code and the server serves no HTTP, so the server has exactly one port open. `remote-deploy.sh` builds the worker as a real image - it had never been one, it was `python:3.14-slim` with the repo bind-mounted and three packages installed from PyPI on every container start - and carries it to the server with `docker save | ssh docker load`. No registry, so no account anywhere, no credential on the server and no copy of the collector in a third party's storage. Redis travels the same way, so the server needs no route to Docker Hub at all. The password the Actor authenticates with is minted at deploy time from `urandom`, not typed by anyone, and written to `deploy/secrets/.env` (600, gitignored). One run installs it on the server and writes it onto each pushed Actor version as a secret `BROKER_URL`. The two halves cannot disagree, and nothing prints a credential - the reports show a hash fingerprint, `#b7aa2470 (64 chars)`, which tells you whether two hosts hold the same secret and gives away none of it. What changes for the Actors on the Store: nothing, and this adds no input. What changes underneath is that `DATA_SOURCE` becomes `redis` rather than the bundled fixtures, which is the setting that makes a published Actor collect rather than replay. `down` removes it again, deliberately leaving `DATA_SOURCE=redis` in place so a disconnected Actor FAILS on its next run rather than falling back to `mock` and serving fixture data to a paying customer who cannot tell. - **Sentiment is now actually produced** (2026-09-09). Both comment Actors are named "with Sentiment" and, measured on 2026-09-09, neither wrote a label: every comment row carried `sentimentLabel: null`. Two switches were off. The worker ran with `--no-sentiment` because the local classifier needed ~1.1 GB of torch weights its image never installed, and the facade never set `sentiment` on the jobs it queued, so even a worker that could label would not have been asked to. The label now comes from a hosted chat model (DigitalOcean's inference gateway, `deepseek-4-flash`) instead of a local model, the facade asks for it on every job that asks for comments, and the worker probes the gateway once at startup and warns if it does not answer. Only the comment text is sent - no usernames, ids or posts - in batches of 20; `loggedout/COMPLIANCE.md` records the change of character, since text now leaves the machine. The profile Actor is untouched: it never walks comments, so it never scores them. What a row carries is unchanged: `sentimentLabel` in `positive`, `neutral`, `negative` and `sentimentScore` in \[0, 1]. The score is the model's stated confidence rather than a softmax, so rank on it rather than threshold it. A comment the gateway could not read stays `null`, never a guessed label. - **Post rows carry the fields the profile grid never had** (2026-09-08). A post listed from an account came back with `timestamp`, `likesCount`, `commentsCount`, `videoViewCount`, `dimensions` and `taggedUsers` all null or absent, and nothing said why. The cause is the surface: Instagram's logged-out profile query draws thumbnails, so it selects a caption, a shortcode, a media id and one image, and nothing else. `taggedUsers` was the visible half of it - `p/DZfnXTHHfUa` has @gucci tagged INSIDE the picture and a caption that is one emoji, so a run listing 100 of @kimkardashian's posts reported the brand nowhere, while the same post's own page names it. Each listed post is now read from its own page, which has all of it. The new `enrichPosts` input turns this off; it is on by default, because a row missing half its columns is not a cheaper row, it is a wrong one. The price is one request per post against roughly one per twelve to list them - 100 posts is \~9 requests listed and ~109 filled in - so a run that wants shortcodes and captions fast should turn it off and expect those fields null. Only the holes are filled: a count that came from a surface that measured it is never overwritten by a second, later reading, and a post that refuses keeps its nulls rather than gaining zeroes. - **"There are no more comments" is now checked before it is believed** (2026-09-07). The worst of the four causes of a short answer, and the last one found, because it does not look like a failure from the inside: Instagram answers `has_next_page: false` while the connection still has thousands of comments behind it. Four runs against `p/Dc6zTucHTv8` within an hour ended at 140, 541, 1.000 and 3.379 comments; a single pinned session walking the same post reached 3.379 over 256 pages. The 140 went out in a record saying "Instagram's logged-out connection ended after 140 of 4314" - a statement about Instagram that was not true, and that no consumer could check. The walk now re-asks the cursor that announced the end, twice, before accepting it, and resumes if either answer disagrees. The same guard is on the posts timeline, which runs the same connection pattern. Re-asked pages are deduplicated by comment id and shortcode, so confirming an end can cost requests but never rows. Measured after the change: three consecutive runs asking for 1.000 comments on that post returned 1.000, 1.000 and 1.000. Two consequences worth stating plainly. A capped ask - "give me N" - is now exact whenever the post holds N. An UNCAPPED ask - "give me everything" - is a lower bound and cannot be anything else: `edges: []` with no next page is the only end-of-data signal the surface has, and it was measured arriving at 512, 932 and 1.005 comments on one post on three different days. Numbers this project published as "all a logged-out reader can see" were lower bounds misread as ceilings, and have been corrected. - **Ask for N results and get N** (2026-09-07). Three things stood between the input schema and that sentence, and all three are fixed: - **A request bigger than one page of the internal read API failed the whole target.** `resultsLimit: 200` reached `list_posts(limit=200)` and came back "limit must be between 1 and 50, got 200"; 200 comments hit the same wall at 100\. The read API pages - it is the same call the MCP tools make - so the run now pages through it, and the ceilings the caller was never shown stay where they belong. A run that asks for more than exists still returns everything that exists: 1.000 comments requested against a post holding 145 is 145 rows and no error. - **A refused page ended a walk.** `fetch()` gives every request four attempts from four addresses, and the fifth failure used to end the walk with whatever had arrived - so "1.000 comments" could come back 512 because one page in the middle was refused, which reads exactly like a post that only has 512. A cursor now gets three further rounds, each after a cooldown that starts at half a minute and grows, before the walk is declared over. It costs nothing when nothing is wrong (four attempts have to fail first) and it lowers the request rate on an address that just refused one, which is what the pacing governor does with the same evidence. - **The post ceiling was a guess.** 500 became **1.000**, measured rather than assumed: @nasa, 1.000 asked for and 1.000 returned, all distinct, 84 pages of the logged-out grid, 660s and 954 KiB through Italian residential exits. Where a short answer is Instagram's doing, the run says so per post - requested, returned, and Instagram's own count next to each other in the log - and `PostsPage` carries a `truncation_note` for the timeline half of the same question, fed by the collector's `_posts_truncated`. - **The login curtain is remembered for ten minutes per country** (2026-09-07), which is a collector change with a facade-visible effect: a run of forty handles used to spend eight refusals and about four minutes per handle rediscovering that `web_profile_info` and `user_feed` answer `require_login` to logged-out callers. Measured on three handles in one run: 8 refusals for the first, 0 for the other two, 199s against the ~330s the same run cost before. The memory is short on purpose - the day the curtain lifts, posts start carrying like and comment counts again with nobody editing anything - and an ordinary refusal is not remembered at all, only Instagram's own `require_login`. - **Every rate is now exactly 25% below the Actor it competes with** (2026-09-04), raised from 10%, and each listing is priced against its own counterpart rather than all three against one. The reference figures are the vendors' undiscounted Free-plan prices, read that day from the Store pages — `/v2/acts/apify~/builds/default` returns `eventPriceUsd: null` to an anonymous caller, so the page is the only source: - `instagram-scraper` **$2.025 / 1.000 results**, against $2.70. - `instagram-profile-scraper` **$1.95 / 1.000 profiles**, against $2.60. - `instagram-comment-scraper` **$1.725 / 1.000 comments**, against $2.30. Each Store listing writes one shape of row per run, so each carries its own `pay_per_event.json` with all four events at that single rate — a run's price can be quoted before it starts. The table in `.actor/`, which is the MCP and REST listing and answers profiles, posts and comments in one call, keeps a rate per event instead: $0.00195, $0.002025 and $0.001725 respectively, each 25% below that row's own counterpart. - **A third Store listing: `instagram-profile-scraper`** (2026-09-04), mirroring `apify/instagram-profile-scraper` rather than the general scraper. Its form asks for `usernames` and nothing else; a run answers one row per account with its recent posts nested in a new `latestPosts` column, and no separate post rows. `usernames` implies `resultsType: details` — an explicit `resultsType` still wins — and `resultsLimit` says how many posts to nest. The definition that used to sit in `actors/instagram-profile-scraper/` was the general scraper under a misleading name and is now `actors/instagram-scraper/`. `includeAboutSection`, the reference's paid extra, is refused with its reason: date joined, country of registration and broadcast channel are not on the logged-out surfaces, so accepting it would bill for an empty panel. - **The dataset rows were checked field for field against the two reference Actors, and five names or shapes were changed to match** (2026-09-04). The comparison was made against `apify/instagram-scraper` and `apify/instagram-comment-scraper`'s own machine-readable dataset schemas and their published output samples, read on 2026-09-04 from `api.apify.com/v2/acts/apify~/builds/default` — not from prose, which in both cases is out of date. What moved: - **`images` is a list of URL strings**, not of objects. That is the shape `apify/instagram-scraper` writes into a column of that name, and the near miss was the dangerous kind: same name, same type at the top level, and `row.images[0]` silently a dict instead of a URL. The per-rendition measurements are not lost — they move intact to **`mediaAssets`** (`url`, `width`, `height`, `isVideo`), a name no reference scraper uses. - **`isPrivate` is now `private`** and **`categoryName` is now `businessCategoryName`**, the spellings the reference profile row uses. This row already spelt `verified` their way rather than `isVerified`, so two halves of one convention had drifted apart. - **Comment rows carry `commentUrl`**: the post permalink, `/c/`, then the comment id, with no trailing slash — the shape the reference Actor emits (its own schema example shows a trailing slash; its output does not). Derived from two columns already on the row, so nothing is collected for it. Absent when the snapshot carries no comment id. - **Comment rows carry `postId`**, Instagram's media id for the post the comment sits under, as a string. The current reference comment scraper has no such column; a `postId` it published until 2025 held a *shortcode*, and ours does not — the shortcode is in `postShortCode` next to it. - **`collectedAt` on a post row is `…T18:01:36.000Z`**, like every other timestamp this Actor writes. It was passed through unnormalised, so one run put `+00:00` on post rows and `.000Z` on profile and comment rows of the same dataset. - **Two more of the reference scraper's inputs are refused by name instead of ignored** (2026-09-04): `skipPinnedPosts` and `addProfileStatistics`. Both are documented in `apify/instagram-scraper`'s README FAQ and in neither of its input schemas, which is exactly what made them dangerous here - a caller copies the key out of that FAQ, `extra="ignore"` deletes it, and the run reports success. `skipPinnedPosts` is the sharper of the two: this Actor does not collect whether a post is pinned, so the pinned posts come back either way and only the flag says otherwise. - **The deploy contract for the two Store definitions is written down and tested** (2026-09-04). `apify push` reads `.actor/actor.json` and nothing else, so publishing either Actor under `actors/` means putting that directory's contents at `facade/.actor/` first. The `../Dockerfile` and `../CHANGELOG.md` in those manifests are written for that destination and resolve nowhere from where the files sit in the repo - correcting them in place would break the push. Neither definition carries `pay_per_event.json`, which lives only in `.actor/` and has to be brought along. - **The `type` column's declared values in `.actor/dataset_schema.json` were wrong** (2026-09-04) — `image`/`video`/`carousel`, this repo's internal vocabulary, where the rows have always carried the Store's `Image`/`Video`/ `Sidecar`. The two Actor definitions under `actors/` were already right. The schemas under `actors/` had no test over them at all until today; three now check that both declare every column the mapper can write, that their views select only declared fields, and that a column's type and enum do not depend on which Actor page you read it from. - **Comment rows carry `id`** (2026-09-04): Instagram's identifier for the comment itself, as a string, matching the reference comment scraper. It is the comment's id, not the commenter's account id, which stays uncollected. Absent on snapshots collected before this date. The other fields the node carries (`repliesCount`, parent id, restriction flags) were measured null or constant on every logged-out node and are deliberately not shipped: a column that can only ever say 0 would read as "no replies", which the logged-out surface never measures. - **A comment list Instagram cut short says so** (2026-09-04). A post's `commentsCount` counts replies and comments the logged-out view does not serve: measured on a post declaring 5,156, the connection served 512 and then answered `has_next_page: false`. That is not a truncation — nothing was capped and nothing failed — so the record now carries `_comments_partial` in the collector's words and the comments page repeats it in `truncation_note`, instead of leaving "512 of 5,156" to be read as a collector that quit early. - **`postsCount`, `followersCount` and `followsCount` are absent when not measured** (2026-09-04). `postsCount` used to fall back to the number of posts the run held, which published `postsCount: 1` for an account with thousands of posts whenever the logged-out header omitted the total, and the two audience counts were written as `0` on a post URL's owner. A count the surface did not carry is now `null` in the REST/MCP profile and an absent column in the dataset; `postsInSnapshot` is the number of posts held and always present. Single post and reel URLs are inputs. `directUrls` used to refuse them by name, because the query behind one post was reachable only to a signed-in visitor; re-measured on 3 September 2026 it answers a logged-out one, so a post URL is a target rather than a refusal. - **`https://www.instagram.com/p/DCMUrLltVlM/` and its `/reel/`, `/reels/`, `/tv/` and `//p//` spellings all name one post**, and one list may mix post URLs with profile URLs and bare handles. The same post named twice is collected once, and deduplication is on the shortcode rather than on the URL, so three spellings of one post are one collection and one invoice. - **`resultsType` over a post URL**: `posts` returns that one post, `reels` returns it only if it is a video (a photo yields zero rows and a line in the log saying the filter matched nothing, which is not a failure), `comments` returns its comments with `resultsLimit` as the number taken. `commentsPostLimit` is inert for a post URL — there is one post to read. - **`resultsType: details` with a post URL is refused with a reason**, per URL, and the rest of the list still runs. `details` asks for an account's own figures — followers, follows, total posts, engagement averages — and the single-post response carries the owner's identity and nothing else about them. A profile row built from it would report a missing follower count, which a consumer reads as "none". The refusal names the account URL that would answer instead. - **A post is stored under its own key**, never the owner's. The record a post URL produces is profile-shaped with three honest holes, and filed under the owner it would have overwritten a full snapshot with a partial one on the "freshest wins" rule. Follower counts are not invented to fill them: on post and comment rows the repeated parent block carries the owner's id and name, which is what the payload actually has. - **`includeNestedComments` is refused when true**, rather than ignored. There is no logged-out query that returns comment replies — measured across Instagram's own JavaScript bundles on 3 September 2026 — and accepting the flag would make a run look like it had walked the threads and found them empty. **`isNewestComments` is accepted and inert**: comments arrive in the order Instagram's connection serves them and are not reordered here. It said so on the form until 1.5, which removed the control and kept the field. - **Five new post columns, in the reference scrapers' spelling**: `hashtags`, `mentions`, `taggedUsers`, `dimensionsWidth` and `dimensionsHeight`. `hashtags` and `mentions` are cut out of the caption already on the row — Unicode, so `#città` is a hashtag; in the order written; case-insensitive duplicates removed keeping the first spelling, because `#Coffee` and `#coffee` open one page on Instagram and counting them twice skews every group-by. An `@` inside a word is an address and not a mention, so `hello@roasters.coffee` publishes nobody. Both are always present, empty list included: the caption is on the row, so "no hashtags" is something this row knows. - **`taggedUsers` is the accounts tagged INSIDE the picture, and it is minimised.** Each entry is `{username, id, isVerified, imageIndex, position}` — `imageIndex` is which frame of a carousel the tag sits on, 0-based, and `position` is Instagram's normalised `[x, y]`. The tagged account's **display name and profile picture are not collected**, though the payload carries them and the other Instagram scrapers publish them: a tag says which account an image references, which the handle and the id say completely, and a name and a face are personal data about somebody who is not even the profile that was asked for. Recorded as a data-protection decision by the controller, with its reasoning, in the collector's own `COMPLIANCE.md` §3 (row 2026-09-03). - **`taggedUsers`, `dimensionsWidth` and `dimensionsHeight` are absent rather than empty when the surface carried none.** Posts read from an account's logged-out grid have no tags and no frame size in the response at all — checked on the selection set of Instagram's own query, not inferred from an empty answer — so those columns are left out. `[]` would read as "this picture tags nobody" and `0` as a zero-pixel image; neither was measured. A post collected by its own URL carries all three. - **Requires a collector built on or after 2026-09-03.** An older worker drops the post target on the way in and discards the job without reporting it, so the facade now asks the worker registry before queueing and refuses out loud when nothing running can collect a post. Profile collections are unaffected. ### 1.2 — 2026-09-03 Compatibility with the Instagram scrapers already on the Store, at the value level rather than only in the field names. Measured against `apify/instagram-scraper` and `apify/instagram-comment-scraper` on 2 September 2026: 14 of the post fields were already spelled identically, and two of the values underneath them were not. - **`type` is now `Image` / `Video` / `Sidecar`**, was `image` / `video` / `carousel`. The words differ, not just the case: a consumer switching on `"Sidecar"` fell through every carousel silently. The HTTP and MCP surfaces keep the domain's spelling — the translation is in `mapper.py`, at the edge that faces the dataset, and nowhere else. - **Timestamps end in `Z`**: `2026-08-16T18:01:36.000Z`, was `2026-08-16T18:01:36+00:00`. The same instant, and every ISO 8601 parser reads both, but the shape a string-slicing consumer was written against is the Store's. Applies to `timestamp` and `collectedAt`. A non-UTC offset is converted rather than relabelled; an unparseable value survives unchanged rather than disappearing. - **BREAKING — `addParentData` now defaults to `false`**, was `true`. `apify/instagram-scraper` defaults it to false, and one input object must not mean two output shapes depending on which Actor reads it. A run that was relying on `ownerUsername`, `ownerId` and `ownerFullName` arriving unasked must now set `addParentData: true` explicitly. - Inputs written for the reference scraper that this Actor cannot honour are **refused by name** instead of being dropped: `search`, `searchType` and `searchLimit` say that targets are named in `directUrls` here, and `resultsType: "stories"` or `"mentions"` names the perimeter rather than reading as a typo. Validation messages no longer carry pydantic's `Value error,` prefix and now name the field they are about. Not changed: comment rows still carry no `id`. The collector discards the comment `pk` at projection (`loggedout/endpoints.py`, `COMMENT_FIELDS`), and COMPLIANCE.md §3 records that tuple's contents as a data-protection decision rather than a code change. Adding it needs a row in that file's decision table. ### 1.1 — 2026-08-31 Store readiness. The Actor was a server on a marketplace of scrapers: a normal run printed usage and exited, which is what "Try for free" started, and there was no output schema — which the platform requires before an Actor can be published at all. - A normal run now scrapes. `directUrls`, `resultsType` (`details`/`posts`/`reels`/`comments`), `resultsLimit`, `commentsPostLimit`, `onlyPostsNewerThan`, `includeMediaUrls` and `addParentData` drive a batch collection that writes one flat row per result to the default dataset. Standby mode is unchanged and still serves MCP and REST over the same data. - Field names on those rows follow the Instagram scrapers already on the Store (`shortCode`, `likesCount`, `ownerUsername`, ids as strings, a hidden like count as `-1`), so an input object written for one of those can be pasted in. - `.actor/output_schema.json` and `.actor/dataset_schema.json`: 45 documented fields and three views — an overview, posts, and comments with their sentiment. - Unsupported inputs are refused by name rather than collected as something else: a single post, a hashtag, a place, a story or a `/tagged/` page each come back with an explanation. One bad profile in a list no longer costs the rest of the run. - Pay-per-event now charges the rate for what was actually delivered. There was one `tool-call` event for everything, so a comment — a step down a paced, paginated thread — billed at the same price as a post, and a collection bundle carrying 2.400 comments billed as a single result. Profiles, posts and comments are now separate events; `collect_posts_with_comments` charges for both units it returns; asking for a collection and polling it are free. - Retitled to describe what it collects rather than how it is served. ### 1.0 — 2026-08-25 First public release. - MCP server (Streamable HTTP) at `POST /mcp` and a REST API under `/profiles/...`, both thin facades over the same `ProfileService`: profile metadata, posts, one post in depth, comments, sentiment, caption search, requesting a collection and following one. - `request_collection` waits for the job it queues (default 90 s, tunable via `brokerWaitMs`) and usually returns the finished snapshot in a single call; a walk too long to wait for comes back as a job id to poll instead. - Three data sources: `mock` (bundled fixture, the default), `http` (a JSONL endpoint) and `redis` (the collector behind a broker, which can trigger a fresh collection on a cache miss). - Standby mode with readiness-probe support; optional pay-per-event billing of one `tool-call` event per successful MCP tool call, refusing further calls once the run's spending limit is reached.