# Changelog of Facebook Posts Search Scraper (`socialdataapi/facebook-posts-search-scraper`) Actor - **URL**: https://apify.com/socialdataapi/facebook-posts-search-scraper/changelog.md - **Full Actor documentation**: https://apify.com/socialdataapi/facebook-posts-search-scraper.md ## Changelog All notable changes to this actor are documented here. ### \[0.4] - 2026-10-10 Gate-2 decision: keep this actor's cookie-free Google-SERP discovery design (Option A). Facebook's own authenticated internal-search approach evaluated against the `Code\Facebook Post Search Scraper\` reference was NOT adopted — it would have required every end user to supply their own Facebook session cookies and would have dropped the contact-lookup, child-comment, and geo/language-targeting features this actor already ships. See `_reports/facebook-post-search-scraper.md` for the full analysis. #### Fixed — `likes`/`comments`/`shares` fabricated a real `0` when the count genuinely wasn't found (behavior change) `_extract_post_from_fb_html` initialized `likes`/`comments` to `0` and, at the end of extraction, forced `shares` to `0` whenever no regex pattern matched on the page. All three counters therefore reported a real zero when the true answer was "not found" on this page layout. Fixed: all three now stay `null` when genuinely not found, matching this project's no-fake-data convention. **This changes some rows' shipped output** from `0` to `null` for `likes`/`comments`/`shares` versus the 0.3 behavior — flagged explicitly since it is a real behavior change, not just an internal refactor. #### Added — pay-per-event spend-limit check `Actor.push_data(..., charged_event_name="row_result")`'s result was checked for a failed push, but never for `event_charge_limit_reached`, so a user's configured spend cap was never actually respected in code. The push site now also stops the whole run cleanly (no more charged rows) when the cap is hit. #### Added — free-plan usage limit (`src/free_limits.py`) New module implementing the project's free-plan limits: up to 20 posts per `searchQueries` entry per run, and 200 posts per month in total across all runs, for non-paying users (unlimited for local runs, Console "Test" runs, and paying users). The counter lives in a named key-value store (`free-quota`) in the running user's own Apify account. Never crashes the run on a limit — logs a notice, sets a run status message, and exits cleanly. #### Changed — Actor lifecycle modernized Replaced the legacy `await Actor.init()` / `await Actor.exit()` pattern with the current `async with Actor:` context manager. #### Changed — logs now use a shared, consistent style (`src/log_style.py`) Added a startup banner, an input echo, a plan line, a free-limit notice, and an end-of-run summary (duration, posts pushed, comments pushed, failed searches). Per-post progress lines and warnings now use a single, consistent emoji set across every log line instead of ad-hoc emoji. #### Fixed — input schema text (no structural change) `input_schema.json`'s titles, descriptions, and section captions used multiple emoji per field and decorative `━━━` separator lines, and the proxy field had no `sectionCaption` of its own. Rewrote every `title` / `description` / `sectionCaption` / `sectionDescription` to use exactly one emoji at the start of each title/section caption and no emoji inside descriptions, and gave the proxy field its own section. No input key, type, `editor`, `enum` value, `default`, `prefill`, `minimum`/`maximum`, or `required` entry was changed — only display text. #### Added — `dataset_schema.json` `fields` block The dataset schema had `views` but no per-item `fields` (JSON-Schema) description at all. Added one covering every key ever pushed, with `likes`/`comments`/`shares` typed as nullable integers to match the bug fix above. #### Changed — Dockerfile migrated to the non-root convention Pinned `apify/actor-python:3.13` (was `apify/actor-python:latest`), `WORKDIR /home/myuser` (was `/app`), `COPY --chown=myuser:myuser`, and added `ENV PYTHONUNBUFFERED=1` immediately after `FROM` so logs reach the Apify Console without being block-buffered. #### Changed — `requirements.txt` Raised the `apify` floor from `>=1.0.0` to `>=2.3.0` (needed for `charged_event_name` support) and added `apify-client` (needed by the new free-plan limit module). Verified a clean install and `src.main` import against the versions this floor actually resolves to (`apify` 4.1.0, `apify-client` 3.3.0) — no `browserforge` pin is needed at this floor. #### Investigated, not changed Reviewed for (a) a literal internal GraphQL query ID / doc_id leaking into a log line, and (b) a silent "logged out" downgrade when nested comment replies are requested. Neither was found in the current code: no log line in `src/main.py` or `src/comments.py` prints a `doc_id`/query identifier (they only ever appear in code comments, which is already within the project's secret-process rule), and there is no `getReplies`/reply-fetching code path at all in this actor today — `comments.py` only ever fetches top-level comments, by design, and the actor has no login-gated behavior of any kind since it is fully cookie-free. Documented here rather than inventing a fix for code that doesn't exist. ### \[0.3] - 2026-08-30 #### Fixed — `text` field returned generic page boilerplate instead of the real post caption **Root cause (confirmed live, via temporary debug instrumentation deployed and run on the actual actor):** `_FB_MESSAGE_TEXT_PATTERN` (`"(?:message|text)"\s*:\s*"..."`) matches the *first* bare `"text":"..."` (or `"message":"..."`) JSON key it finds anywhere in the entire page bootstrap payload — which contains many unrelated UI-microcopy strings keyed `"text"` before the actual post content ever appears. Captured, on real live posts, the full ordered list of candidates this pattern matches; the first one satisfying the old filter (length > 10, doesn't start with "facebook") was consistently one of: a `"text":"See more about "` UI prompt, a follower/following count (`"2.8M followers"`), the Page's own bio/about text (identical to its `og:description`), or a `"Page · "` label — never the post's caption. Example observed live: for a real post whose actual caption began "JOÃO PESSOA CONHECEU OS CAMPEÕES! 🏆🏀 …", the actor's `text` field contained only `"See more about Basquete Brasil - CBB"`. The real caption is present on the same page in Facebook's structured message object, `"message":{"text":""}`, which the old pattern's "stop at the first match" loop never reached. A second, independent bug in the same code path: the manual unescaping (`.replace("\\n","\n").replace('\\"', '"')`) does not decode `\uXXXX` escapes or UTF-16 surrogate pairs, so any caption containing an emoji (extremely common) would have come through with literal garbled `\ud83c\udfc6`-style sequences even once pointed at the right field. **Fix (`src/main.py`):** added `_FB_MESSAGE_OBJECT_TEXT_PATTERN` (matches the structured `"message":{"text":"..."}` shape) and tries it first; the old loose pattern is now only a fallback for the rare page layout where that structured object isn't present, and is additionally filtered through a new `_GENERIC_TEXT_BOILERPLATE` check so it can no longer surface a "See more about", follower/following count, or "Page · category" string as if it were the caption. Added `_decode_json_string_fragment()`, which properly JSON-decodes the captured string (handling `\uXXXX`/surrogate-pair emoji correctly) instead of the old two-`.replace()` approach, and switched both the new and fallback extraction to use it. **Verification:** live-ran the fix (build with temporary debug fields first, to capture real evidence of which JSON shape holds the caption; then the actual fix, redeployed and re-run). Before: `"text": "See more about Basquete Brasil - CBB"`. After the fix, re-verified live with a fresh minimal run — `text` now contains the real, full post caption with correctly-decoded emoji and line breaks (see the live verification note in the project report for the exact before/after row). ### \[0.2] - 2026-08-30 #### Fixed — 0 posts discovered/saved on every run **Root cause (confirmed live):** the actor discovers candidate Facebook posts by scraping Google's `site:facebook.com` search results through Apify's `GOOGLE_SERP` proxy group, then parsing the returned SERP HTML for outbound links. That parsing (`_parse_facebook_urls_from_serp_html` in `src/main.py`) matched only the old Google redirect format — `` — via `_SERP_REDIRECT_PATTERN` (`/url\?q=`) and a generic `` scan. Google no longer emits that format. Live-tested by fetching a real SERP page through the account's actual `GOOGLE_SERP` proxy (`football site:facebook.com`): every organic result link is now wrapped behind an opaque, undecodable redirect — `href="/goto?url=CAESXAHrOzAV...">` — where the `url=` value is an encrypted protobuf-style token, not a URL. Confirmed there is no longer a single `/url?q=` occurrence, and no plain `` to any external site, anywhere in the returned page. Also confirmed the token cannot be resolved another way: requesting Google's `/goto` (or `/url`) endpoint directly through the `GOOGLE_SERP` proxy returns HTTP 400 `"Unsupported Google service or resource"` — that proxy group only forwards `/search`. Because of this, `_parse_facebook_urls_from_serp_html` matched zero URLs on every single query, `_fetch_posts` never had a URL to fetch, and the actor produced an empty dataset on every run regardless of `searchQueries`, `maxPosts`, or proxy settings. **Fix (`src/main.py`):** the real destination URL turns out to still be present as plain, unwrapped text elsewhere on the same SERP page — Google emits an adjacent embedded JS data array (used for its own client-side hover-preview-card hydration) shaped like `...,"/goto?url=CAES...",...],["https://www.facebook.com/page/","Title", "Snippet",...]`. `_parse_facebook_urls_from_serp_html` was rewritten to extract the URL from that structure (via the already-present-but-unused `_FACEBOOK_URL_PATTERN`) instead of from `` markup. The dead `_SERP_REDIRECT_PATTERN`, `_HREF_PATTERN`, and `_extract_url_from_serp_href` were removed along with the now-unused `unquote` import. **Verification:** fetched real, multi-page SERP HTML for `football site:facebook.com` through the account's actual `GOOGLE_SERP` proxy session and ran it through the new parser: 12 distinct, real `facebook.com` page/post/video URLs extracted from page 1 (`facebook.com/NFL/`, `facebook.com/PumaFootball/`, `facebook.com/SkySportsFootball/`, etc.), 8 more distinct URLs from page 2 (`start=10`) — pagination continues to work. No `/login`, `/help`, or `/policy` junk URLs in the output. #### Hardened — Facebook login-wall/block responses silently produced near-empty rows While live-testing the fix above, fetching a discovered post/Page URL (phase 2, via a shared Apify datacenter proxy session) was observed to be inconsistent: the identical URL with identical headers returned real page content on one attempt and, on the very next attempt, either a redirect to `/login` or Facebook's small generic "Sorry, something went wrong" error page — ordinary per-request IP-reputation noise on a shared proxy pool, not a permanent block on the URL itself. The existing code treated any HTTP-200 response (including a login-wall/error page) as success, so a blocked fetch would still get pushed to the dataset as a charged row with almost every field null/empty. **Fix (`src/main.py`):** added `_looks_blocked_or_login()` — a real post/Page document is always tens of KB and references `fbcdn.net`; Facebook's login-wall/error responses are a few KB of boilerplate with neither. Added `_fetch_post_html()`, which retries a blocked fetch on a freshly-rotated Apify proxy session (up to 3 attempts total) before giving up. `_fetch_posts` now uses this helper and skips (logs a warning, does not push or charge) a URL that is still blocked after retries, instead of silently saving a near-empty row. Proxy rotation is skipped (matching prior behavior) when the user explicitly disabled `proxyConfiguration.useApifyProxy`. ### \[0.1] - Initial version - Search Facebook by keyword via Google `site:facebook.com` discovery (`GOOGLE_SERP` proxy group) and extract post text, engagement counts, media, and page identity. - `postTimeRange` recency filter enforced before a row is saved/charged. - Optional per-post top-level comment fetch to an uncharged child dataset (`maxCommentsPerPost`). - Optional page contact/lead enrichment (`enableContactLookup`): email/phone/website/ address from each post's own Page, never fabricated. - `targetCountry` / `targetLanguage` geo/language biasing of the discovery search.