Facebook Posts Search Scraper
Pricing
from $1.49 / 1,000 results
Facebook Posts Search Scraper — Search and extract Facebook posts by keywords, including post text, authors, page details, timestamps, reactions, comments, shares, media URLs, and post links. Useful for content research, competitor analysis, audience insights, and social media monitoring.
All notable changes to this actor are documented here.
Gate-2 decision: keep this actor's cookie-free Google-SERP discovery design
(Option A). Facebook's own authenticated internal-search approach evaluated
against the Code\Facebook Post Search Scraper\ reference was NOT adopted —
it would have required every end user to supply their own Facebook session
cookies and would have dropped the contact-lookup, child-comment, and
geo/language-targeting features this actor already ships. See
_reports/facebook-post-search-scraper.md for the full analysis.
likes/comments/shares fabricated a real 0 when the count genuinely wasn't found (behavior change)_extract_post_from_fb_html initialized likes/comments to 0 and, at
the end of extraction, forced shares to 0 whenever no regex pattern
matched on the page. All three counters therefore reported a real zero when
the true answer was "not found" on this page layout. Fixed: all three now
stay null when genuinely not found, matching this project's no-fake-data
convention. This changes some rows' shipped output from 0 to null
for likes/comments/shares versus the 0.3 behavior — flagged explicitly
since it is a real behavior change, not just an internal refactor.
Actor.push_data(..., charged_event_name="row_result")'s result was checked
for a failed push, but never for event_charge_limit_reached, so a user's
configured spend cap was never actually respected in code. The push site now
also stops the whole run cleanly (no more charged rows) when the cap is hit.
src/free_limits.py)New module implementing the project's free-plan limits: up to 20 posts per
searchQueries entry per run, and 200 posts per month in total across all
runs, for non-paying users (unlimited for local runs, Console "Test" runs,
and paying users). The counter lives in a named key-value store
(free-quota) in the running user's own Apify account. Never crashes the
run on a limit — logs a notice, sets a run status message, and exits
cleanly.
Replaced the legacy await Actor.init() / await Actor.exit() pattern with
the current async with Actor: context manager.
src/log_style.py)Added a startup banner, an input echo, a plan line, a free-limit notice, and an end-of-run summary (duration, posts pushed, comments pushed, failed searches). Per-post progress lines and warnings now use a single, consistent emoji set across every log line instead of ad-hoc emoji.
input_schema.json's titles, descriptions, and section captions used
multiple emoji per field and decorative ━━━ separator lines, and the
proxy field had no sectionCaption of its own. Rewrote every title /
description / sectionCaption / sectionDescription to use exactly one
emoji at the start of each title/section caption and no emoji inside
descriptions, and gave the proxy field its own section. No input key, type,
editor, enum value, default, prefill, minimum/maximum, or
required entry was changed — only display text.
dataset_schema.json fields blockThe dataset schema had views but no per-item fields (JSON-Schema)
description at all. Added one covering every key ever pushed, with
likes/comments/shares typed as nullable integers to match the bug fix
above.
Pinned apify/actor-python:3.13 (was apify/actor-python:latest),
WORKDIR /home/myuser (was /app), COPY --chown=myuser:myuser, and added
ENV PYTHONUNBUFFERED=1 immediately after FROM so logs reach the Apify
Console without being block-buffered.
requirements.txtRaised the apify floor from >=1.0.0 to >=2.3.0 (needed for
charged_event_name support) and added apify-client (needed by the new
free-plan limit module). Verified a clean install and src.main import
against the versions this floor actually resolves to (apify 4.1.0,
apify-client 3.3.0) — no browserforge pin is needed at this floor.
Reviewed for (a) a literal internal GraphQL query ID / doc_id leaking into a
log line, and (b) a silent "logged out" downgrade when nested comment
replies are requested. Neither was found in the current code: no log line in
src/main.py or src/comments.py prints a doc_id/query identifier (they
only ever appear in code comments, which is already within the project's
secret-process rule), and there is no getReplies/reply-fetching code path
at all in this actor today — comments.py only ever fetches top-level
comments, by design, and the actor has no login-gated behavior of any kind
since it is fully cookie-free. Documented here rather than inventing a fix
for code that doesn't exist.
text field returned generic page boilerplate instead of the real post captionRoot cause (confirmed live, via temporary debug instrumentation deployed and run on
the actual actor): _FB_MESSAGE_TEXT_PATTERN ("(?:message|text)"\s*:\s*"...") matches
the first bare "text":"..." (or "message":"...") JSON key it finds anywhere in the
entire page bootstrap payload — which contains many unrelated UI-microcopy strings keyed
"text" before the actual post content ever appears. Captured, on real live posts, the
full ordered list of candidates this pattern matches; the first one satisfying the old
filter (length > 10, doesn't start with "facebook") was consistently one of: a
"text":"See more about <PageName>" UI prompt, a follower/following count (
"2.8M followers"og:description), or a
"Page · <category>" label — never the post's caption. Example observed live: for a
real post whose actual caption began "JOÃO PESSOA CONHECEU OS CAMPEÕES! 🏆🏀 …", the
actor's text field contained only "See more about Basquete Brasil - CBB".
The real caption is present on the same page in Facebook's structured message object,
"message":{"text":"<real caption>"}, which the old pattern's "stop at the first
match" loop never reached. A second, independent bug in the same code path: the manual
unescaping (.replace("\\n","\n").replace('\\"', '"')) does not decode \uXXXX
escapes or UTF-16 surrogate pairs, so any caption containing an emoji (extremely common)
would have come through with literal garbled \ud83c\udfc6-style sequences even once
pointed at the right field.
Fix (src/main.py): added _FB_MESSAGE_OBJECT_TEXT_PATTERN (matches the structured
"message":{"text":"..."} shape) and tries it first; the old loose pattern is now only a
fallback for the rare page layout where that structured object isn't present, and is
additionally filtered through a new _GENERIC_TEXT_BOILERPLATE check so it can no longer
surface a "See more about", follower/following count, or "Page · category" string as if
it were the caption. Added _decode_json_string_fragment(), which properly JSON-decodes
the captured string (handling \uXXXX/surrogate-pair emoji correctly) instead of the old
two-.replace() approach, and switched both the new and fallback extraction to use it.
Verification: live-ran the fix (build with temporary debug fields first, to capture
real evidence of which JSON shape holds the caption; then the actual fix, redeployed and
re-run). Before: "text": "See more about Basquete Brasil - CBB". After the fix,
re-verified live with a fresh minimal run — text now contains the real, full post
caption with correctly-decoded emoji and line breaks (see the live verification note in
the project report for the exact before/after row).
Root cause (confirmed live): the actor discovers candidate Facebook posts by
scraping Google's site:facebook.com search results through Apify's GOOGLE_SERP
proxy group, then parsing the returned SERP HTML for outbound links. That parsing
(_parse_facebook_urls_from_serp_html in src/main.py) matched only the old Google
redirect format — <a href="/url?q=<real_url>..."> — via _SERP_REDIRECT_PATTERN
(/url\?q=) and a generic <a href> scan.
Google no longer emits that format. Live-tested by fetching a real SERP page through
the account's actual GOOGLE_SERP proxy (football site:facebook.com): every
organic result link is now wrapped behind an opaque, undecodable redirect —
href="/goto?url=CAESXAHrOzAV..."> — where the url= value is an encrypted
protobuf-style token, not a URL. Confirmed there is no longer a single /url?q=
occurrence, and no plain <a href="https://..."> to any external site, anywhere in
the returned page. Also confirmed the token cannot be resolved another way: requesting
Google's /goto (or /url) endpoint directly through the GOOGLE_SERP proxy returns
HTTP 400 "Unsupported Google service or resource" — that proxy group only forwards
/search. Because of this, _parse_facebook_urls_from_serp_html matched zero URLs on
every single query, _fetch_posts never had a URL to fetch, and the actor produced an
empty dataset on every run regardless of searchQueries, maxPosts, or proxy settings.
Fix (src/main.py): the real destination URL turns out to still be present as
plain, unwrapped text elsewhere on the same SERP page — Google emits an adjacent
embedded JS data array (used for its own client-side hover-preview-card hydration)
shaped like
...,"/goto?url=CAES...",...],["https://www.facebook.com/page/","Title", "Snippet",...]_parse_facebook_urls_from_serp_html was rewritten to extract the
URL from that structure (via the already-present-but-unused _FACEBOOK_URL_PATTERN)
instead of from <a href> markup. The dead _SERP_REDIRECT_PATTERN, _HREF_PATTERN,
and _extract_url_from_serp_href were removed along with the now-unused unquote
import.
Verification: fetched real, multi-page SERP HTML for football site:facebook.com
through the account's actual GOOGLE_SERP proxy session and ran it through the new
parser: 12 distinct, real facebook.com page/post/video URLs extracted from page 1
(facebook.com/NFL/, facebook.com/PumaFootball/, facebook.com/SkySportsFootball/,
etc.), 8 more distinct URLs from page 2 (start=10) — pagination continues to work.
No /login, /help, or /policy junk URLs in the output.
While live-testing the fix above, fetching a discovered post/Page URL (phase 2, via a
shared Apify datacenter proxy session) was observed to be inconsistent: the identical
URL with identical headers returned real page content on one attempt and, on the very
next attempt, either a redirect to /login or Facebook's small generic "Sorry,
something went wrong" error page — ordinary per-request IP-reputation noise on a
shared proxy pool, not a permanent block on the URL itself. The existing code treated
any HTTP-200 response (including a login-wall/error page) as success, so a blocked
fetch would still get pushed to the dataset as a charged row with almost every field
null/empty.
Fix (src/main.py): added _looks_blocked_or_login() — a real post/Page document
is always tens of KB and references fbcdn.net; Facebook's login-wall/error responses
are a few KB of boilerplate with neither. Added _fetch_post_html(), which retries a
blocked fetch on a freshly-rotated Apify proxy session (up to 3 attempts total) before
giving up. _fetch_posts now uses this helper and skips (logs a warning, does not
push or charge) a URL that is still blocked after retries, instead of silently saving
a near-empty row. Proxy rotation is skipped (matching prior behavior) when the user
explicitly disabled proxyConfiguration.useApifyProxy.
site:facebook.com discovery (GOOGLE_SERP
proxy group) and extract post text, engagement counts, media, and page identity.postTimeRange recency filter enforced before a row is saved/charged.maxCommentsPerPost).enableContactLookup): email/phone/website/
address from each post's own Page, never fabricated.targetCountry / targetLanguage geo/language biasing of the discovery search.