# Changelog of Glovo Scraper Spain — Menu Prices, Fees & Stores (`rastriq/glovo-spain-scraper`) Actor

- **URL**: https://apify.com/rastriq/glovo-spain-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/rastriq/glovo-spain-scraper.md

## Changelog

### 0.1 — 2026-11-15 (not deployed)

First version. Glovo stores of a chain per city found with an adaptive grid of delivery points,
and the menu of each store. Plain HTTP to `api.glovoapp.com`, no login, no browser, no proxy.
Store records (one per `storeId` + `addressId`) and product records (one per `addressId` +
`storeProductId`).

**Source behaviour, as measured on 2026-11-15 from a home connection:**

- `GET /v3/stores/{slug}` with the delivery point in headers: 200 with the store that serves the
  point, 404 when none does.
- `GET /v4/stores/{storeId}/addresses/{addressId}/content/main`: ~400 KB, ~100 KB on the wire.
  A product listed in two sections ("Promociones" and its own) appears twice: one record, both
  sections.
- `POST /v1/web/store_wall/search`: with the headers listed in the survey it answered `503 no
  available server`. The web client's own header set (read from the page on 2026-11-15: request
  id, perseus client id and consent, dynamic session id, device URN, app context, request TTL,
  and `Glovo-Client-Info: web-customer-web-react/{version} project:customer-web`) gives 200. All
  ids are random UUIDs per run; no cookie, token or key.
- App version `v1.2968.0`, read from `NEXT_PUBLIC_APP_VERSION` in the city page on every run.
- City codes: `MAD` and `BCN` verified. `VLC` and `SVQ` answer 400; the real codes were not found,
  so Valencia and Sevilla are not offered.
- **Menu throttling**: at one menu every 0.5 s, 429 (`code: throttle`, domain `store-view`) after
  about 25 menus; 14 throttled answers and 2 menus lost out of 83. At 1.5 s: 83 of 83, no 429. Menus
  now have their own pacing (`menuIntervalSecs`, default 1.5).

**Live runs** (`tools/live_check.py`, home connection):

| Run | Points | Served | Stores | Products | Time | Errors |
|---|---:|---:|---:|---:|---:|---|
| Burger King Madrid, grid 2 km + 0.5 km | 2,308 (196 coarse) | 1,392 | 83 | — | 1,154 s | 0, no 429 |
| Burger King Madrid, 83 menus at 1.5 s | — | — | — | 16,941 | 123 s, 8.5 MB | 0, no 429 |
| Goiko Madrid, same grid | 1,988 | 1,016 | 20 | — | 994 s | 0, no 429 |
| Goiko Madrid, 20 menus | — | — | — | 1,355 | 29 s, 0.8 MB | 0 |
| Burger King Madrid, second sweep (~75 min after the first) | 2,308 | 1,392 | 79 | — | 1,154 s | 0, no 429 |

- **Id stability**: the 79 stores of the second sweep all have the same `storeId:addressId` as
  in the first, and 71 of them the same first grid point. **The store set depends on the time of
  day**: the 4 stores missing in the second sweep (00:05 vs ~01:20) were open in the first; when a
  store closes Glovo assigns its points to another one. Schedule sweeps at a fixed hour inside
  opening hours (e.g. 13:00) so runs are comparable.

- **Goiko**: 20 stores, 31 of 70 products with more than one price across stores (the survey
  found 31 of 66 between two stores).

- **Coverage**: the survey's full 0.5 km grid (3,080 points) found 77 Burger King stores; the
  adaptive grid found 83 with 2,308 points (75 % of the calls). Burger King delivers almost
  everywhere in Madrid, so few coarse cells are skipped; chains with fewer stores save more.

- **Price per store**: 119 of 227 Burger King products (by SKU) have more than one price across
  the 83 stores.

- 75 of the 83 stores have a 280xx postal code; the rest are just outside the municipality but
  deliver into it.

**Tests:** 56 offline checks (`tests/test_v01.py`): headers, store and menu parsing, search tiles,
grid geometry, slug table, privacy (no phone; fiscal name only for companies), dataset schema,
and a stubbed run with a 404-everywhere slug, a 503 slug and a failing menu.

**Not in 0.1:**

- matching Glovo stores to locator stores by address (Glovo gives no coordinates);
- Valencia, Sevilla and other cities (city codes not found);
- datacenter-IP test of this header set (first platform run).

**Platform smoke test (2026-11-15, Apify datacenter IP, no proxy, build 0.1.1 `beta`): inconclusive.**

- Run `UXia0xEniDo3Taxpu` (08:06 UTC), Goiko Madrid capped at 20 points: the city page answered
  200 (app version `v1.2968.0` read) and the 20 store requests answered **404**, not 403. No block
  seen, but no store either: with the cap, the 20 points are the first rows of the grid, on the
  south edge of the box, where Goiko does not deliver. The run failed as designed
  ("404 at every point"). The smoke input with `maxPointsPerSlug: 20` was a bad choice.
- Run `R9f751TWgM3NI8Uam` (Burger King, full coarse grid) was aborted at the owner's request about
  a minute after it started. Datacenter access to `stores/{slug}` and the menus is still unproven.
- Run `ZjeDFXQleg73tfid9` (08:13 UTC), Apify **RESIDENTIAL ES** proxy, Burger King, coarse grid
  4 km, 1 menu: the city page answered through the proxy (app version read in 1.5 s), then **no
  store request finished**: no warning in 5 minutes and the run hit the 300 s platform timeout
  (TIMED-OUT, no SUMMARY, USD 0.03). With a 60 s socket timeout and 4 retries, one request that
  never answers takes about 330 s, so the very first `api.glovoapp.com` call hung through the
  proxy. **Corrected later the same day:** the actor did not log successful points, so a slow
  proxy (several seconds per point) looks the same as a hang. This run proves neither.
- Run `8356m0ogh2MarMLEe` (08:21 UTC), **Apify datacenter, no proxy**, Burger King, coarse grid
  2 km: app version read, then the first ~64 s of store requests (about 120 at one every 0.5 s)
  answered without error; **from 08:22:50 every request got 403** with a CloudFront-style HTML
  "ERROR" page (195 in a row, none recovered). Aborted at 08:24. Reading: a volume/rate block on
  the datacenter IP after about a minute, not a header problem (the home run did 2,308 requests at
  the same pace with no error). From datacenter the sweep does not work as is.
- "Required request header 'Glovo-Perseus-Session-Id' is missing" is what the API answers to a
  request without the web headers (e.g. opening the URL in a browser tab); the actor sends that
  header on every API call.
- **Residential proxy (Decodo, ASN 12338, Spain) as a secret env var of the actor:**
  - From home, through Decodo (08:33 UTC): city page 3.3 s, 3 store requests 200 in 0.8-1.2 s.
  - Run `wbDnfe6pxbt18VcJl` (08:29 UTC), coarse grid 2 km (196 points), 240 s timeout:
    TIMED-OUT with no warning. At about 1.2 s per point the grid needs about 4 minutes: the timeout
    was too short for the run, not a block.
  - Run `SYBgg86Qbw2Kqdu1n` (08:34 UTC), Burger King, coarse grid 4 km, 1 menu: **SUCCEEDED**.
    49 points, 21 served, **16 stores**, 0 errors, 1 network error retried; 75 s, 0.1 MB.
    The menu answered 200 with "no product list for this store": at 09:35 Madrid time the store
    was closed. Menus through the proxy still have to be checked inside opening hours.

**Code changes after the platform tests (still 0.1, builds 0.1.2-0.1.4 on `beta`):**

- Proxy from the secret env var `GLOVO_PROXY_URL` (URL or Decodo `host:port:user:pass`), used when
  the input asks for no proxy. The URL is never logged; `SUMMARY.egress` says `direct`,
  `env-proxy` or `input-proxy`.
- `fineStepKm: 0` now means coarse grid only, as the schema says. It used to fall back to 0.5.
- Progress log every 25 points. Without it a slow run looked like a hung one.
- `SUMMARY.menus` lists each menu with its status and error; `menusEmpty` counts closed stores.
- Tests: 63 offline checks (fine step 0, proxy env parsing, egress).
- **Egress fallback** (`egressMode`, default `auto`): direct first; on the first 403 the client
  switches once to the residential proxy of `GLOVO_PROXY_URL` and resends that request; no
  rotation, no other change to the request. `egressMode: proxy` uses the proxy from the start. An
  input `proxyConfiguration` overrides both. Tests: 67 offline checks (switch on 403, no second
  switch, 403 without fallback fails).
- Run `QwB2Q6BJnmhqrHabg` (2026-11-15 08:45 UTC, build 0.1.5 `beta`, `egressMode: auto`), Burger
  King, coarse grid 2 km, 1 menu: **SUCCEEDED**. Direct (Apify datacenter) for the first 135
  requests (~67 s), then one 403; the client switched to Decodo and the rest went through it with
  0 errors. 196 points, 82 served, **43 stores**; 1 menu `ok` with **222 products** through the
  proxy; 168 s, 0.3 MB. Second datacenter block measured at the same point (~120-135 requests,
  \~1 minute) as run `8356m0ogh2MarMLEe`.
