# Changelog of PropertyFinder UAE Scraper: Dubai Sale, Rent & Off-Plan (`kotaro001/propertyfinder-ae-scraper`) Actor

- **URL**: https://apify.com/kotaro001/propertyfinder-ae-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/kotaro001/propertyfinder-ae-scraper.md

## Changelog

**Current Active Pricing (04/11/2026):** `apify-actor-start` $0.005 Free/Bronze, $0.004 Silver+; `listing-scraped` $0.0020 Free/Bronze, $0.0014 Silver+; `detail-scraped` $0.0050 Free/Bronze, $0.0040 Silver+, charged only when `includeDetails` is true and `maxDetails` above 0. Older entries below are history; where they disagree, this block wins. Silver+ rows cost $0.0014: 200,000 rows a month is $280.12 with 30 daily starts, under a $300 budget.

### 0.1.68 — 2026-11-07

- Added `verify=False` to `curl_cffi` requests via Apify Proxy to eliminate SSL `CertificateVerifyError` issues.
- Reinforced resilient pagination in `PropertyFinderSource.fetch`: warn and `continue` to the next page when a search page encounters a network error or fetch failure, preventing early scan termination.

### 0.1.67 — 2026-11-06

- Flagship SEO documentation suite integrated across 10 sections with verified dataset benchmark and multi-channel workflows.
- Published 10 gold-standard Public Tasks on Apify Store for high-intent UAE property valuation, DLD benchmarks, and Golden Visa searches.
- Hardened input schema with masked `stateKey` (`isSecret: true`) for persistent price tracking and change detection.
- Pre-warmed verified snapshots with 100% real UAE portal property data across prime Dubai and Abu Dhabi communities.
- Resilient 3-tier proxy retry architecture (`ATTEMPTS = 3`: UNBLOCKER -> RESIDENTIAL -> UNBLOCKER) conforming to Sprint 17 standards.

### 0.1.64 — 2026-11-04

- README lists the ten ready-made searches with the input each one stores, and sample links carry ?v=38.
- Sample links carry a version tag, so an edge copy cannot serve an old file.
- Input schema now matches the form: every filter a task sets is declared for the buyer, including `rentFrequency`.
- README lists all 96 output columns by name; two columns that never existed are gone.
- Sample CSV cells that hold an object or a list are JSON, so `json.loads` parses them.

### 0.1.47 — 2026-11-04

- Fixed dataset width at 96 keys in both modes: the four detail-derived columns (`sizeSqft`, `agentLicenseNumber`, `reraValidationUrl`, `agent1PhoneComplete`) now ship as null in list mode, so a CSV header does not shift when detail mode is off.

### 0.1.46 — 2026-11-04

- `miss_count` no longer reaches the dataset: it is lifecycle bookkeeping, like `detail_from_cache`. A run on a tracked state used to ship 97 keys instead of 96.

### 0.1.45 — 2026-11-04

- Fixed the detail charge: every enriched row now pays `detail-scraped`, whether the payload was fetched now or served by the store. The billing path no longer reads the internal `detail_from_cache` flag, so a store hit can neither discount a customer nor reveal that a store exists.
- Detail mode is off by default: `includeDetails` false, `maxDetails` 0. `maxDetails` 0 now means zero detail rows (it used to fall back to 50).
- README and the price table say "enriched detail row", not "live detail page".

### 0.1.36 — 2026-11-04

- Price at the time of this build: `listing-scraped` $0.0020 on Free and Bronze, $0.0014 from Silver. `apify-actor-start` $0.002 / $0.001. `detail-scraped` was removed in that build; it returned in 0.1.45 at $0.0050 / $0.0040 per enriched row.
- Five DLD benchmark columns from Dubai Land Department ready sales: `dldMedianSoldPriceAed`, `dldMedianSoldPricePerSqftAed`, `vsDldSoldMedianPercent`, `dldSampleTransactionsCount`, `dldBenchmarkBasis`.
- `agent1PhoneDigits` and `agent1PhoneComplete` prove the phone is whole even when a viewer masks it.

### 0.1.27 — 2026-11-03

- Parser: `rera` is read as a dict (Dubai: QR link, number usually blank) or a
  bare string (Abu Dhabi: 14-digit Madhmoun permit). Abu Dhabi rows no longer
  crash the list parser.
- Parser: `chequesCount` from rent descriptions, `parkingSpaces` from the
  portal number / a "N parking" phrase / the covered-parking amenity,
  `serviceChargePerSqftAed` from `service_fee_per_sqft`.
- Parser: detail images are read from the nested gallery shape; a detail row
  keeps the longer image list.
- Search URL: only the keys the portal honours are sent (`l`, `c`, `t`, `fu`,
  `pf`, `pt`, `page`, `ob`). Completion, bedrooms, area and verified filters run
  on the fetched rows, and the actor reads more pages until it has enough rows.
- Location map: 75 communities across five emirates, city-level fallback.
- Price drops: the portal's own signed `price_change` becomes `previousPrice`,
  `priceDiff` and `priceDiffPercent`. `onlyPriceChanged` keeps only rows whose
  previous price is higher; there is no widening fallback.
- `minDaysOnMarket` is a hard filter: no fallback list when nothing is old
  enough.
- Details are written to the state service (`/v1/details`), and the snapshot
  push now retries once, so a slow upload no longer costs the state write.
- README 1,830 words with the seven-point gate; 11 parser tests on real payload
  fixtures.

### 0.1.0 — 2026-11-01

Khung trống, tách từ actor `ai-jobs-daily` đang chạy (12.942 dòng) nhưng **bỏ hết
phần nghiệp vụ**: không adapter ATS, không schema ngành, không danh sách nguồn.

- Core: nguồn, chuẩn hoá, gắn tag, cổng kiểm, khử trùng, vòng đời, field health,
  kho Apify + kho local, chỗ cắm heal.
- Một nguồn duy nhất: `http_json` khai bằng cấu hình.
- `scripts/new_actor.py` tạo pack rỗng + sửa config trỏ vào pack đó.
- Cổng: 35 test, giới hạn 250 dòng mỗi file.

### Chưa phát hành — khối 25 cột dùng chung cho BĐS Úc

- `src/normalize/unified.py`: 25 cột phẳng, cùng tên cho mọi actor
  (`listingId`, `dealType`, `changeStatus`, `priceHistory`, `landSizeSqm`…).
- `src/normalize/unified_fields.py`: hàm thuần parse địa chỉ, diện tích,
  lịch sử giá, trạng thái đổi giá.
- Giữ nguyên cột riêng của từng nguồn; chỉ thêm khối chung.
- `changeStatus` tách `PRICE_DROP` / `PRICE_INCREASE`, không dùng
  `PRICE_CHANGED` của portal.
- 10 test mới, tổng 63 test xanh; lint 250 dòng xanh.
