Urbania Peru Scraper

Apify Actor (v1) that scrapes Urbania.pe property listing pages (rentals and sale) over plain HTTP with a browser-grade Chrome TLS/HTTP2 client profile. No browser automation, no stealth plugins, no solver services.

How it works

• URL contract: https://urbania.pe/buscar/{alquiler|venta}-de-propiedades-en-{city} with ?page=N for pages 2+ (confirmed via canonical and rel=prev/next link elements on live captures — the React paging widgets render hrefs without query strings).
• Optional keyword search: the search input appends

  ?keyword=<urlencoded term>
  to the same operation+city URL — the filter composes with both (probed 2026-09-03: 8,329 baseline rentals/Lima vs 1,355 with keyword=miraflores, 3,358 for sale+keyword=miraflores; nonsense terms return a handful of fuzzy matches, so the filter is real, not ignored. Rejected probes: q= is silently ignored, text= triggers a 403 Cloudflare challenge).
• One GET per page through a Chrome-impersonation HTTP client (curl_cffi chrome131 in src/urbania_actor/client.py: a genuine Chrome TLS/HTTP2 fingerprint with the full Chrome header set — UA, sec-ch-ua, Sec-Fetch-*, header ordering) and a configurable delay (default 4s). The legacy plain-httpx browser-headers client stays selectable via the client input (see Proxy).
• Parsing merges both embedded formats:
  • ld+json RealEstateListing.mainEntity blocks → url, name, image, publisher, contentLocation.
  • ld+json Apartment/Residence blocks → postal address (joined to listings via the shared CDN image id).
  • data-qa attributed cards (POSTING_CARD_PRICE, POSTING_CARD_FEATURES, POSTING_CARD_LOCATION, POSTING_CARD_GALLERY) → price, rooms, bathrooms, area, card location line.
• id is the numeric tail of the listing URL; listingKey is urbania.pe:{id}.
• Prices like S/ 2,529 / USD 810 parse to (PEN, 2529.0); dual-currency cards (S/ 4,859 · USD 1,400) report the first currency shown. Consultar precio cards yield null price.
• Feature ranges (1 a 2 dorm.) report the upper bound.

Input

Proxy and client lanes

urbania.pe sits behind Cloudflare, and the wall is the HTTP client fingerprint, not just the IP:

• Datacenter egress (plain Apify runs) is Cloudflare-blocked with 403 challenge pages — always was.
• The Apify residential proxy (groups-RESIDENTIAL,country-PE, ~$0.0002 per run) passed with browser-grade headers alone — until 2026-09, when the challenge rate on that plain-httpx lane drifted to ~100% (endurance canary 31/60 = 52% success vs required ≥95%; see docs/FINDINGS.md, issue #201). Retrying could not absorb a 100% challenge rate.
• The default lane since #201 is therefore curl_cffi Chrome impersonation over the Apify residential proxy (client=chrome): a genuine Chrome TLS/HTTP2 fingerprint over residential exits. The same fingerprint class is the verified pass on two other Cloudflare-walled sources in this repo (ImportYeti, Farside) where plain httpx draws the JS wall.
• client=plain keeps the pre-#201 lane (httpx + the browser-grade header set from src/urbania_actor/scraper.py) selectable as a fallback; proxy=external + proxyUrl routes through an external residential gateway (e.g. DataImpulse PE) for lane probing/failover.

The Apify proxy URL is built from platform env vars only (APIFY_PROXY_HOSTNAME / APIFY_PROXY_PORT / APIFY_PROXY_PASSWORD) — nothing is hardcoded, because from platform runs the external proxy.apify.com:8000 endpoint is rejected on the Free plan. When proxy=residential and those env vars are missing (local run), the Actor falls back to no proxy with a warning.

Challenges, retries and BlockError

A challenge shell — a 403 with Cloudflare markers ("Just a moment" / "Attention Required"), or any tiny marker page carrying no listing content (is_challenge_shell() in src/urbania_actor/proxy.py) — is retried with exponential backoff: up to 2 additional attempts (3 total, capped for #201 — retrying a ~100% challenge rate just burns time and bandwidth; ~20s / ~40s ± jitter), each through a fresh residential-proxy session so the retry exits through a new IP.

If every attempt still lands on a shell, the run FAILS with a BlockError naming the egress lane (e.g.

curl_cffi chrome131 over Apify residential proxy (groups-RESIDENTIAL,country-PE)

Cost note: failed attempts still consume residential-proxy bandwidth (worst case 3 attempts of a ~1.3MB page ≈ $0.004 per URL — acceptable; backoff waits are wait time, not extra requests).

Detail enrichment (includeDetails)

With includeDetails: true the Actor fetches each collected listing's detail URL (/inmueble/<slug>-<id>) after the list crawl, through the same proxy lane and honoring requestDelaySecs and maxResults. Each record gains:

• description — the full ad text (#reactDescription .section-description).
• postedAt — exact ISO-8601 publication timestamp. Urbania does not ship ld+json datePosted or article:published_time; the exact value comes from the inline publicationDateFormatted script value (UTC, ...Z), with the standard ld+json/meta sources checked first for robustness. List cards carry no timestamps, so this is the only source.
• sellerContact — email/phone only when plainly rendered in visible HTML. Urbania usually hides contacts behind a "Ver teléfono" action and embeds only a partial phone (+5199) plus form placeholders in scripts, so this is commonly null.
• additionalImageUrls — gallery URLs from the inline pictures JSON array, capped at 10.

Failed detail fetches (HTTP errors, Cloudflare challenge shells, non-HTML) are logged as warnings and leave the enrichment fields null — they never fail the run.

Dataset fields

id, listingKey, url, name, priceAmount, priceCurrency, operation, city, source_query, address, rooms, bathrooms, areaM2, publisher, imageUrl, source, collectedAt

description, postedAt, sellerContact, additionalImageUrls

Known limitation

urbania.pe sits behind Cloudflare. Datacenter egress is blocked (403 challenge pages); the supported egress is the Apify residential proxy with the Chrome-impersonation client (client=chrome, the default) — the plain-httpx residential lane went ~100% challenge-walled in 2026-09 (issue #201; see docs/FINDINGS.md and docs/PROXY-VERDICT.md for the lane history and the probe protocol). The runner fails loudly with a lane-naming BlockError instead of silently pushing zero rows when a challenge shell appears on every attempt.

Development

cd actors/urbania-peru-scraper
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt jsonschema
PYTHONPATH=src .venv/bin/python -m unittest discover -s tests

Tests run against real captured pages (tests/fixtures/alquiler_lima_page1.html, Wayback Machine snapshot of /buscar/alquiler-de-propiedades-en-lima, 30 listing cards; tests/fixtures/alquiler_lima_keyword_miraflores.html, the same capture filtered to the 6 Miraflores cards as a ?keyword=miraflores page; and tests/fixtures/detail_page.html, a Wayback Machine snapshot of a /inmueble/clasificado/... detail page).