# Changelog of Facebook Marketplace Scraper (`worshipful_knife/facebook-marketplace-scraper`) Actor

- **URL**: https://apify.com/worshipful_knife/facebook-marketplace-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/worshipful_knife/facebook-marketplace-scraper.md

## Changelog

### 2026-11-20 (cheaper full browser path)

#### Changed (cost)

- Anonymous `fullBrowserMode` no longer loads Facebook's JavaScript (~2 MB of residential traffic per page): anonymous Facebook serves one server-rendered page per search and never paginates, so the JS added nothing (same fields on 20/20 listings, measured). It also reads the other sort orders like the HTTP path, instead of scrolling 5 empty rounds (~90 s).
- Listing (detail) pages in the browser never load JavaScript (cookies too); the inlined JSON is complete, and `condition` is still filled. If a scriptless listing page gives no detail at all, it is retried once with JavaScript.
- With cookies, only search pages load JavaScript (pagination).
- Popup dismissal, human-like pauses and network-idle waits are skipped on scriptless pages; `ping`/`manifest` requests and Facebook logging endpoints are blocked.

#### Measured (platform, NYC iPhone / Chicago sofa, 1024 MB; cost = sum of run.usageUsd)

| Test | Before | After |
|---|---|---|
| `fullBrowserMode`, 20 listings | $1.05 / 1000 (run tBV5nvZBuoHuezann) | $0.33 / 1000 (NE2TDW3Bjl21shCBv, fixed browser start dominates) |
| `fullBrowserMode`, 100 listings | - | $0.088 / 1000 (lHO0xg3C1lSd3gJw1) |
| `fullBrowserMode` + 6 details | $8.87 / 1000 (pjHIypK8QOisBJBx4) | $2.18 / 1000 (r7WAuazJbuY5AjKTu) |

- Not worth it: 512 MB (twice as slow, same cost), datacenter proxy (Facebook answers every request with a login wall).
- A desktop listing page is ~180 KB on the wire; that is the floor of a full-mode detail row (the mobile HTTP path is ~5x lighter but has no `condition`).

### 2026-11-15 (no browser for anonymous runs)

Checkpoint: residential proxy traffic is ~75% of a detail row's platform cost (1.0.34 London 20 details: $0.029 proxy, $0.009 compute). A desktop listing page is ~190 KB on the wire even without a browser; the mobile listing page is ~35 KB and has the description, photos, Details lines, seller and map position.

#### Changed (cost)

- Anonymous runs fetch pages over plain HTTP (got-scraping, browser TLS/headers, same residential proxy) instead of a browser. Search pages: server-rendered desktop HTML (unchanged data). Detail pages: the mobile listing page (~5x less proxy traffic); the desktop page is read only when the mobile layout is not recognised, and a browser only for pages that fail every HTTP retry.
- Default memory 1 GB (the browser is only a fallback).
- HTTP concurrency is 4x `maxConcurrency` (max 20); each request uses its own proxy session.
- EU/EEA/UK/CH searches: listing pages go through US residential IPs (Facebook shows logged-out mobile visitors there a cookie-consent screen instead of the page). If consent screens still appear, the run switches to the desktop page.

#### Measured (platform, beta build 1.0.40, 1024 MB; cost = sum of run.usageUsd)

| Test | Secs | Items | Full / with description | Cost | Per 1000 | Residential | Peak mem |
|---|---|---|---|---|---|---|---|
| London sofa, 20 details (1.0.34 baseline, 2 GB) | 78 | 20 | 17 / 16 | $0.0394 | $1.97 | 3.65 MB | 498 MB |
| London sofa, 20 details | 56 | 20 | 19 / 19 | $0.0133 | $0.67 | 1.12 MB | 261 MB |
| New York iPhone, 50 details | 77 | 50 | 49 / 48 | $0.0333 | $0.67 | 3.24 MB | 458 MB |
| Paris velo, 10 basic | 5 | 10 | – | $0.0017 | $0.17 | 0.14 MB | 177 MB |
| London, 4 details, `fullBrowserMode` (browser path check) | 96 | 4 | 3 / 3 | $0.1088 | $27 | 12.8 MB | 641 MB |

Per detail row: ~47 KB of residential traffic (mobile page + TLS) against ~175 KB before. Proxy is still ~78% of the cost.

#### Notes (detail rows from the mobile page)

- Up to 5 photos (535 px) per listing; `imageCount` gives the listing's total photo count.
- Category attributes come from the Details lines (`attributes.details` + mileage, transmission, colours, owners); vehicle make/model are not split out.
- The renderer's line wraps are removed from descriptions; a seller's line break right after a long line can become a space.

### 2026-11-15 (location + cost)

#### Fixed

- City searches returned listings near the proxy IP (e.g. "newyork" → Stockton, CA): Facebook only honours its own slugs (`nyc`, `la`, `philly`, `dc`, `vegas`, `mexicocity`, …) and redirects unknown ones. Cities now map to verified slugs (80+ checked; Leeds/Nice use city-page ids), `searchUrls` with known misspellings are fixed, unknown towns are geocoded (OpenStreetMap) and snapped to the nearest verified city, and each search page logs where Facebook actually searched (warning when it is not the requested city).

#### Added

- `distanceKm` and `outsideSearchArea` on every row; `strictLocation` to drop far listings; `latitude`/`longitude` (API) or `"lat,lng"` in `city`.
- For US towns without a Facebook slug the residential proxy is targeted to the state.
- Anonymous searches also run 4 other sort orders when maxResults > ~24 (Facebook shows one page of ~24 per anonymous search).

#### Changed (cost)

- Anonymous runs no longer load Facebook's JavaScript (listings are in the server-rendered HTML): ~0.14 MB per search page and ~0.18 MB per detail page instead of ~2.2 MB. The warm-up page only loads on retries. `fullBrowserMode` restores the old behaviour. Proxy traffic is logged in the run summary.

### 2026-11-15

#### Changed (behavior)

- `maxResults`: empty / `null` now means **100** (was unlimited). Set `0` for no cap; the search then stops by itself when Facebook stops showing new matching listings.
- `scrapeDetails` defaults to `false` everywhere (schema, code, README).
- Monitoring mode: the default store is now per search (`fb-marketplace-seen-<hash>`). IDs from the old shared `fb-marketplace-seen` store are imported on the first run. Set `monitoringStoreName` to keep using a specific store.
- Runs that scrape nothing now **fail** with an actionable message (login wall, blocked IP, no results, strictMatch rejected everything). Monitoring runs with only already-seen listings, and `untilDate` runs with only older listings, still succeed.
- Default memory is 2 GB.

#### Fixed

- Search scrolling no longer runs until the timeout and retries: it stops after 5 scrolls without a new matching listing, with a hard cap on scrolls and time per search.
- Monitoring: IDs are saved during the run (and on abort/migration), only after a listing is delivered, so aborted runs don't re-charge.
- Soft blocks ("This content isn't available") retry on a new session instead of ending the search or marking listings removed.
- The run stops cleanly when the maximum total charge for the run is reached.
- `detail-enriched` is charged only when the detail page added a description, more than one photo or category attributes; condition parsing is strict (e.g. "New York, NY" is no longer read as "New").
- Prices in European and other formats ("1.200 €", "1 200,50 €", ₱, ₹, zł, kr, ¥).
- Cookie input accepts Playwright storageState and EditThisCookie / Cookie-Editor exports.

#### Added

- Progress status message during the run.
- `inputUrl`, `isRemoved` fields; `attributes` / `isSold` in dataset views.
- `FAILED_REQUESTS` record in the default key-value store.
