Facebook Ad Library Scraper

A Python + Scrapling scraper for the Facebook Ad Library, packaged as an Apify actor. Paste Ad Library URLs → get the full ad object (a superset of curious_coder/facebook-ads-library-scraper), including EU and UK transparency, advertiser info, and per-country age/gender reach breakdowns.

Drop-in replacement for curious_coder/facebook-ads-library-scraper. Same input fields, same output records in the same field order (plus a couple of extra fields). To switch an existing integration, see ./MIGRATION.md — usually just changing the actor id.

• Actor: concrete_pavilion/facebook-ad-library-scraper · ID VPPuOei1FoCdSE815

How it works

Facebook's public Ad Library is a React SPA that loads ads via an internal GraphQL API. This scraper uses a "bootstrap → replay" strategy:

1. Bootstrap (no browser) — a plain HTTP GET of the search URL (http_bootstrap.py) returns the session tokens (lsd, jazoest, __*) and the server-rendered first page of ads embedded in the HTML. This avoids launching a stealth browser (which downloads ~40 MB of ad-creative video/images through the proxy) — cutting residential bandwidth ~25–50× and running ~8× faster. A media-blocked stealth browser (bootstrap.py) is kept as a fallback.
2. Replay — a fast HTTP client (Scrapling Fetcher, curl_cffi with TLS-impersonation) replays the AdLibrarySearchPaginationQuery GraphQL POST, mutating only the pagination cursor, until every ad is collected.
3. Details (optional) — for each ad, AdLibraryV3AdDetailsQuery returns advertiser, aaa_info, and full transparency_by_location (EU + UK reach breakdowns), merged losslessly. Detail queries are fetched concurrently (detailsConcurrency, default 15 on the actor).

Each output record is then arranged to curious_coder's exact field order at every nesting level (see reorder.py / _order_template.json); our extra fields are appended, never substituted.

Facebook rate-limits by IP (error 1675004); with a proxy_provider (Apify Residential) the scraper rotates to a fresh IP and continues.

Works logged-out. Only two GraphQL queries are needed:

Query	doc_id	Purpose
AdLibrarySearchPaginationQuery	24922295957467452	The ad list (cursor pagination)
AdLibraryV3AdDetailsQuery	25068828942793558	Per-ad detail (--scrape-ad-details)

Facebook rate-limits aggressively (error code 1675004), so the scraper supports proxy rotation: on a rate-limit it rotates to a fresh IP (Apify residential) and continues.

Install (local)

Requires Python 3.11+ and uv (or plain pip).

uv venv --python 3.11 .venv
uv pip install --python .venv/bin/python -e ".[dev]"
.venv/bin/scrapling install        # downloads the stealth browser (patchright chromium)

CLI usage

.venv/bin/fb-ads-scrape \
  --url 'https://www.facebook.com/ads/library/?active_status=active&ad_type=all&country=ALL&media_type=all&q=nike&search_type=keyword_unordered&sort_data[mode]=total_impressions&sort_data[direction]=desc' \
  --scrape-ad-details \
  --limit-per-source 50 \
  --proxy 'http://user:pass@host:port' \
  --out ads.jsonl

Options: --url (repeatable), --out (.jsonl/.json/.csv), --count, --limit-per-source, --scrape-ad-details, --active-status, --country, --proxy, --run-tag. With no --count/--limit-per-source, it scrapes all ads for the query.

Python API

from fb_ads_scraper.models import ScrapeOptions
from fb_ads_scraper.runner import scrape

opts = ScrapeOptions(urls=["https://www.facebook.com/ads/library/?q=nike&search_type=keyword_unordered"],
                     scrapeAdDetails=True, limitPerSource=50)
for ad in scrape(opts, proxy_provider=lambda: "http://user:pass@host:port"):
    print(ad["ad_archive_id"], ad.get("aaa_info", {}).get("eu_total_reach"))

Apify actor

The .actor/ directory + src/main.py wrap the package as an Apify actor (concrete_pavilion/facebook-ad-library-scraper, ID VPPuOei1FoCdSE815) with Apify Residential proxy + rotation built in.

apify login                 # once
apify run                   # local run; reads storage/key_value_stores/default/INPUT.json
apify push                  # deploy to the Apify platform (use --force to overwrite console edits)

Input (see .actor/input_schema.json): urls, scrapeAdDetails, count, limitPerSource, maxConcurrency, detailsConcurrency, scrapePageAds.{period,activeStatus,sortBy,countryCode}, runTag, proxyConfiguration (defaults to RESIDENTIAL — also accepts curious_coder's proxy field name). Each ad is pushed to the default dataset.

• Memory: default 1024 MB (0.25 vCPU) — this is I/O-bound and uses ~120 MB; Apify bills memory × time, so it's right-sized for cost.
• Logs: a startup banner, per-URL progress at ~10% milestones (▶ … N ads available → k/N ads (Xs, R ads/s) → ✓ … N ads in Mm SSs), and a final run summary. Scrapling's per-request noise is suppressed.

Output

The raw Facebook ad node (lossless) plus enrichments — arranged to curious_coder's exact field order: url, ad_library_url, start_date_formatted, end_date_formatted, 1-based position, total (actual ads returned for the URL), ads_count, stringified page_id, and (with scrapeAdDetails) advertiser, aaa_info, transparency_by_location.{eu_transparency, uk_transparency, br_transparency}, verified_voice_context. The one field we add beyond curious_coder is is_siep_advertiser_eligible_for_ai_disclosure. See ./MIGRATION.md for the full compatibility contract and docs/superpowers/specs/ for the schema + a real captured example.

Testing

.venv/bin/pytest -q            # unit tests (no network)
.venv/bin/pytest -m network    # opt-in live tests (need a proxy or a cooled-down IP)

Project layout

fb_ads_scraper/     # the scraper package
  url_parser.py     # Ad Library URL -> GraphQL variables
  bootstrap.py      # stealth-browser token/session capture
  client.py         # curl_cffi GraphQL replay
  paginator.py      # cursor pagination + rate-limit backoff/rotation
  extractor.py      # pull ad nodes from GraphQL payloads
  details.py        # AdLibraryV3AdDetailsQuery enrichment (EU/UK transparency)
  normalizer.py     # superset output + enrichments (calls reorder)
  reorder.py        # re-key each ad to curious_coder's field order (+ _order_template.json)
  session.py        # captured-request -> replay form builder
  proxies.py        # ProxyRotator
  runner.py         # orchestration across URLs (+ proxy rotation, progress logs)
  output.py, cli.py, models.py
src/                # Apify actor entrypoint (main.py)
.actor/             # actor.json (memory 1024 MB), input_schema.json, Dockerfile
tests/              # pytest suite + real captured fixtures
docs/superpowers/   # design spec, implementation plan, live capture reference
MIGRATION.md        # drop-in migration guide from curious_coder's actor

Notes & compliance

Scraping Facebook is against its Terms of Service — treat this as a legal/compliance decision, not just a technical one. Use residential proxies and reasonable pacing.