# Urbania Peru Scraper (`barefoot_grade/urbania-peru-scraper`) Actor

Extract property listings (rentals and sale) from Urbania.pe, Peru's largest real-estate portal. Includes keyword search, optional detail enrichment (full description, exact posting date, seller contact, gallery), and structured fields: price, currency, rooms, bathrooms, area, address, publisher.

- **URL**: https://apify.com/barefoot\_grade/urbania-peru-scraper.md
- **Developed by:** [Philip Kirkbride](https://apify.com/barefoot_grade) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.40 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Urbania Peru Scraper

Apify Actor (v1) that scrapes Urbania.pe property listing pages (rentals and
sale) over plain HTTP with a browser-grade Chrome TLS/HTTP2 client profile.
No browser automation, no stealth plugins, no solver services.

### How it works

- URL contract: `https://urbania.pe/buscar/{alquiler|venta}-de-propiedades-en-{city}`
  with `?page=N` for pages 2+ (confirmed via canonical and `rel=prev/next`
  link elements on live captures — the React paging widgets render hrefs
  without query strings).
- Optional keyword search: the `search` input appends `?keyword=<urlencoded
  term>` to the same operation+city URL — the filter composes with both
  (probed 2026-09-03: 8,329 baseline rentals/Lima vs 1,355 with
  `keyword=miraflores`, 3,358 for sale+`keyword=miraflores`; nonsense terms
  return a handful of fuzzy matches, so the filter is real, not ignored.
  Rejected probes: `q=` is silently ignored, `text=` triggers a 403
  Cloudflare challenge).
- One `GET` per page through a Chrome-impersonation HTTP client
  (`curl_cffi` `chrome131` in `src/urbania_actor/client.py`: a genuine
  Chrome TLS/HTTP2 fingerprint with the full Chrome header set — UA,
  `sec-ch-ua`, `Sec-Fetch-*`, header ordering) and a configurable delay
  (default 4s). The legacy plain-httpx browser-headers client stays
  selectable via the `client` input (see Proxy).
- Parsing merges both embedded formats:
  - `ld+json` `RealEstateListing.mainEntity` blocks → url, name, image,
    publisher, contentLocation.
  - `ld+json` `Apartment`/`Residence` blocks → postal address (joined to
    listings via the shared CDN image id).
  - `data-qa` attributed cards (`POSTING_CARD_PRICE`, `POSTING_CARD_FEATURES`,
    `POSTING_CARD_LOCATION`, `POSTING_CARD_GALLERY`) → price, rooms,
    bathrooms, area, card location line.
- `id` is the numeric tail of the listing URL; `listingKey` is
  `urbania.pe:{id}`.
- Prices like `S/ 2,529` / `USD 810` parse to `(PEN, 2529.0)`; dual-currency
  cards (`S/ 4,859 · USD 1,400`) report the first currency shown.
  `Consultar precio` cards yield null price.
- Feature ranges (`1 a 2 dorm.`) report the upper bound.

### Input

| field | type | default | notes |
| --- | --- | --- | --- |
| operation | `rentals` | `sale` | `rentals` | alquiler / venta |
| city | string slug | `lima` | e.g. `arequipa` |
| search | string | *(none)* | optional keyword filter (`?keyword=...`); composes with operation+city, URL-encoded automatically, recorded per listing as `source_query`; blank means no filter |
| maxPages | 1–3 | 1 | Cloudflare-flagged site, stay polite |
| maxResults | int ≥ 1 | 50 | stops early once reached |
| requestDelaySecs | 1–30 | 4 | delay between page fetches; ≥3s enforced when proxied |
| client | `chrome` | `plain` | `chrome` | HTTP client profile: `chrome` = curl\_cffi Chrome TLS impersonation (the default lane since the plain client started drawing ~100% challenge shells); `plain` = pre-#201 httpx browser-headers fallback |
| proxy | `residential` | `external` | `none` | `residential` | egress lane (see below) |
| proxyCountry | 2-letter code | `PE` | residential proxy session country |
| proxyUrl | string | *(none)* | external residential gateway for `proxy=external`, e.g. `http://gw.dataimpulse.com:823` (DataImpulse; the username gains `__cr.<proxyCountry>` automatically); credentials embedded or via the two fields below |
| proxyUsername | string | *(none)* | external proxy username when `proxyUrl` has none embedded |
| proxyPassword | string | *(none)* | external proxy password when `proxyUrl` has none embedded |
| includeDetails | boolean | `false` | fetch each listing's detail page and enrich the record (see below) |

### Proxy and client lanes

urbania.pe sits behind Cloudflare, and the wall is the **HTTP client
fingerprint**, not just the IP:

- Datacenter egress (plain Apify runs) is Cloudflare-blocked with 403
  challenge pages — always was.
- The Apify **residential** proxy (`groups-RESIDENTIAL,country-PE`,
  \~$0.0002 per run) passed with browser-grade headers alone — until
  2026-09, when the challenge rate on that plain-httpx lane drifted to
  \~100% (endurance canary 31/60 = 52% success vs required ≥95%; see
  `docs/FINDINGS.md`, issue #201). Retrying could not absorb a 100%
  challenge rate.
- The default lane since #201 is therefore **`curl_cffi` Chrome
  impersonation over the Apify residential proxy** (`client=chrome`):
  a genuine Chrome TLS/HTTP2 fingerprint over residential exits. The
  same fingerprint class is the verified pass on two other
  Cloudflare-walled sources in this repo (ImportYeti, Farside) where
  plain httpx draws the JS wall.
- `client=plain` keeps the pre-#201 lane (httpx + the browser-grade
  header set from `src/urbania_actor/scraper.py`) selectable as a
  fallback; `proxy=external` + `proxyUrl` routes through an external
  residential gateway (e.g. DataImpulse PE) for lane probing/failover.

The Apify proxy URL is built from platform env vars only
(`APIFY_PROXY_HOSTNAME` / `APIFY_PROXY_PORT` / `APIFY_PROXY_PASSWORD`) —
nothing is hardcoded, because from platform runs the external
`proxy.apify.com:8000` endpoint is rejected on the Free plan. When
`proxy=residential` and those env vars are missing (local run), the
Actor falls back to no proxy with a warning.

#### Challenges, retries and BlockError

A challenge **shell** — a 403 with Cloudflare markers ("Just a
moment" / "Attention Required"), or any tiny marker page carrying no
listing content (`is_challenge_shell()` in `src/urbania_actor/proxy.py`)
— is retried with exponential backoff: up to 2 additional attempts
(3 total, capped for #201 — retrying a ~100% challenge rate just burns
time and bandwidth; ~20s / ~40s ± jitter), each through a **fresh
residential-proxy session** so the retry exits through a new IP.

If every attempt still lands on a shell, the run FAILS with a
`BlockError` naming the egress lane (e.g. `curl_cffi chrome131 over
Apify residential proxy (groups-RESIDENTIAL,country-PE)`) — a walled
lane is an egress-switching problem (`client`/`proxy` inputs), not a
re-run problem. The string `challenge-platform` is not treated as a
block signal: Cloudflare injects that telemetry script even into clean
200 listings pages.

Cost note: failed attempts still consume residential-proxy bandwidth
(worst case 3 attempts of a ~1.3MB page ≈ $0.004 per URL — acceptable;
backoff waits are wait time, not extra requests).

### Detail enrichment (includeDetails)

With `includeDetails: true` the Actor fetches each collected listing's
detail URL (`/inmueble/<slug>-<id>`) after the list crawl, through the same
proxy lane and honoring `requestDelaySecs` and `maxResults`. Each record
gains:

- `description` — the full ad text (`#reactDescription .section-description`).
- `postedAt` — exact ISO-8601 publication timestamp. Urbania does not ship
  ld+json `datePosted` or `article:published_time`; the exact value comes
  from the inline `publicationDateFormatted` script value (UTC, `...Z`),
  with the standard ld+json/meta sources checked first for robustness.
  List cards carry no timestamps, so this is the only source.
- `sellerContact` — email/phone only when plainly rendered in visible HTML.
  Urbania usually hides contacts behind a "Ver teléfono" action and embeds
  only a partial phone (`+5199`) plus form placeholders in scripts, so this
  is commonly `null`.
- `additionalImageUrls` — gallery URLs from the inline `pictures` JSON
  array, capped at 10.

Failed detail fetches (HTTP errors, Cloudflare challenge shells, non-HTML)
are logged as warnings and leave the enrichment fields `null` — they never
fail the run.

### Dataset fields

`id, listingKey, url, name, priceAmount, priceCurrency, operation, city,
source_query, address, rooms, bathrooms, areaM2, publisher, imageUrl, source,
collectedAt` (`source_query` is the applied `search` term, null on non-search
runs) plus, on `includeDetails` runs, `description, postedAt, sellerContact,
additionalImageUrls` (schema: `.actor/dataset_schema.json`, JSON Schema
draft-07).

### Known limitation

urbania.pe sits behind Cloudflare. Datacenter egress is blocked (403
challenge pages); the supported egress is the Apify residential proxy
with the Chrome-impersonation client (`client=chrome`, the default) —
the plain-httpx residential lane went ~100% challenge-walled in
2026-09 (issue #201; see `docs/FINDINGS.md` and
`docs/PROXY-VERDICT.md` for the lane history and the probe protocol).
The runner fails loudly with a lane-naming `BlockError` instead of
silently pushing zero rows when a challenge shell appears on every
attempt.

### Development

```
cd actors/urbania-peru-scraper
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt jsonschema
PYTHONPATH=src .venv/bin/python -m unittest discover -s tests
```

Tests run against real captured pages
(`tests/fixtures/alquiler_lima_page1.html`, Wayback Machine snapshot of
`/buscar/alquiler-de-propiedades-en-lima`, 30 listing cards;
`tests/fixtures/alquiler_lima_keyword_miraflores.html`, the same capture
filtered to the 6 Miraflores cards as a `?keyword=miraflores` page; and
`tests/fixtures/detail_page.html`, a Wayback Machine snapshot of a
`/inmueble/clasificado/...` detail page).

# Actor input Schema

## `operation` (type: `string`):

Listing operation to search for.

## `city` (type: `string`):

City slug as used in the Urbania URL, e.g. 'lima' or 'arequipa'.

## `search` (type: `string`):

Optional free-text keyword filter appended as ?keyword=<term> to the operation+city search URL (it composes; neither is dropped). URL-encoded automatically. Records carry the applied term in source\_query; null on non-search runs. Blank means no filter.

## `maxPages` (type: `integer`):

Pages to fetch (1-3; the site is Cloudflare-flagged, stay polite).

## `maxResults` (type: `integer`):

Stop after this many listings.

## `requestDelaySecs` (type: `number`):

Seconds to wait between page fetches (Cloudflare-flagged site, be polite). At least 3s is enforced when the residential proxy is enabled.

## `client` (type: `string`):

TLS/HTTP2 fingerprint of the fetch client. 'chrome' uses a browser-grade Chrome profile that passes the site's anti-bot checks; 'plain' is the legacy profile kept as a fallback.

## `proxy` (type: `string`):

Egress lane. Datacenter egress is Cloudflare-blocked; the Apify residential proxy is the verified lane on the FREE plan (docs/PROXY-VERDICT.md). 'external' routes through your own residential gateway set under proxyUrl. Falls back to no proxy when platform proxy env vars are absent (local runs).

## `proxyCountry` (type: `string`):

Two-letter country code for the residential proxy session.

## `proxyUrl` (type: `string`):

External residential proxy for proxy=external, e.g. http://gw.dataimpulse.com:823 (DataImpulse; the username automatically gains \_\_cr.<proxyCountry> for country-pinned residential exits). Either a full URL with embedded credentials (http://user:pass@host:port) or a bare host:port together with proxyUsername/proxyPassword. Ignored unless proxy=external.

## `proxyUsername` (type: `string`):

Username for proxyUrl when the URL does not embed credentials.

## `proxyPassword` (type: `string`):

Password for proxyUrl when the URL does not embed credentials.

## `includeDetails` (type: `boolean`):

When true, fetch each collected listing's detail page (same proxy lane and delay) and add the full description, exact postedAt timestamp, seller contact when plainly public, and extra gallery image URLs. Missing fields are null; failed detail fetches are logged as warnings and never fail the run.

## Actor input object example

```json
{
  "operation": "rentals",
  "city": "lima",
  "maxPages": 1,
  "maxResults": 50,
  "requestDelaySecs": 4,
  "client": "chrome",
  "proxy": "residential",
  "proxyCountry": "PE",
  "includeDetails": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "operation": "rentals",
    "city": "lima",
    "search": "",
    "maxPages": 1,
    "maxResults": 50,
    "requestDelaySecs": 4,
    "client": "chrome",
    "proxy": "residential",
    "proxyCountry": "PE",
    "proxyUrl": "",
    "proxyUsername": "",
    "proxyPassword": "",
    "includeDetails": false
};

// Run the Actor and wait for it to finish
const run = await client.actor("barefoot_grade/urbania-peru-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "operation": "rentals",
    "city": "lima",
    "search": "",
    "maxPages": 1,
    "maxResults": 50,
    "requestDelaySecs": 4,
    "client": "chrome",
    "proxy": "residential",
    "proxyCountry": "PE",
    "proxyUrl": "",
    "proxyUsername": "",
    "proxyPassword": "",
    "includeDetails": False,
}

# Run the Actor and wait for it to finish
run = client.actor("barefoot_grade/urbania-peru-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "operation": "rentals",
  "city": "lima",
  "search": "",
  "maxPages": 1,
  "maxResults": 50,
  "requestDelaySecs": 4,
  "client": "chrome",
  "proxy": "residential",
  "proxyCountry": "PE",
  "proxyUrl": "",
  "proxyUsername": "",
  "proxyPassword": "",
  "includeDetails": false
}' |
apify call barefoot_grade/urbania-peru-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,barefoot_grade/urbania-peru-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/g5M5Set1JfdgNwfvZ/builds/axeOsv5NXMT64koq3/openapi.json