Cashify Scraper (v2 — fast + per-city + per-brand)

Extracts the estimated price of a phone variant (based on the city) and an optional warranty status from Cashify and writes them into a CSV.

Output: data/prices_<city>_<brand>_<date>.csv — only 3 columns: (each city + brand + day gets its OWN file, so nothing is overwritten or mixed; e.g. prices_default_apple_2026-06-19.csv)

device	estimate_price	warranty
samsung galaxy s20 ultra 5g (12 gb 128 gb)	14070	out_of_warranty
oppo a15 (2 gb 32 gb)	2990	out_of_warranty

The storage variant now appears separately in brackets: model (8 gb 256 gb). warranty is filled only when you pass --warranty (a browser is required). Otherwise it is unknown. Models that return the price directly now also have their warranty checked (unknown only when the condition flow does not open at all).

---

Setup

pip install playwright            # only for --warranty
python -m playwright install chromium

Running

# 1) one time: find all phone URLs -> data/urls.json
python cashify_scraper.py discover

# 2) get the price for your city + your brand/model
python cashify_scraper.py scrape --city delhi --brand samsung
python cashify_scraper.py scrape --pincode 282001 --match "redmi note 13"
python cashify_scraper.py scrape --city agra              # all phones in agra

Options

flag	meaning
--city <name>	price for that city (delhi, agra, mumbai, lucknow ...)
--pincode <pin>	use a pincode instead of a city
--brand <brand>	only one brand (apple, samsung, xiaomi ...)
--match "<text>"	name must contain this text (e.g. "galaxy a14")
--limit N	only N phones (for testing)
--workers N	how many phones at once (default 5; more = slightly risky)
--warranty	also extract warranty (browser, slow)
--headful	show the browser window (debug)

Debug helpers

python cashify_scraper.py cities agra              # see city -> region info
python cashify_scraper.py check <variant-url> --city delhi   # test the price for one URL

Crash-safe + resume

• Every price is saved to disk immediately (data/prices_<city>_<brand>_<date>.csv), and progress is saved along the way (data/done_<city>_<brand>_<date>.json). If the script is closed or crashes by accident, whatever was extracted is preserved.
• Running the same command again skips already-fetched phones and continues from where it left off — nothing is fetched twice.
• Interactive mode looks at the old data and asks: Continue (c) or start over (s). If you want a fresh run from the CLI, pass --restart.

---

How it is FAST + SAFE + per-CITY (your 3 requirements)

1. Fast, but no IP ban

• The price now comes directly over HTTP, not from a browser (the Cashify page renders its HTML server-side and the price is in it) → ~1 sec per phone.
• Many phones at once (in parallel) → hours of work done in minutes.
• A plain page GET looks like a normal visitor (not a heavy bot browser) → lower ban risk. Even so: random jitter delay, user-agent rotation, and all threads cool down as soon as a 429/403 appears (keep --workers low for extra safety).

2. City-specific value

• Cashify gives a different price for each city (Delhi ₹14070 vs Agra ₹13320).
• Pass --city/--pincode → the city id is fetched from Cashify's own region API and a cookie is set, which yields that city's price. (Region info is cached in data/regions.json.)
• Only what you ask for is scraped — not the whole site.

3. Specific brand / model

• Use --brand and --match to get only the data you care about: e.g. --brand apple or --match "iphone 13" (both together as well).

Robust even if Cashify changes its code

• The price is extracted in 5 different ways (if one breaks, another works).
• If the city API breaks → default-city price + a clear warning (no crash).
• The warranty buttons/questions are found by text (not by CSS class).

---

v3 — Cache DB + API (Apify / RapidAPI) + Proxy fallback

1. Interactive mode is now simple

python cashify_scraper.py → now only asks for the brand (city defaults to Gurgaon, price is fast). One y/n for warranty. No other questions.

2. Caching database (data/cache.db, SQLite)

Whatever is extracted the first time is saved in the DB. The next user / next time the same thing is requested, it is returned instantly without scraping.

Data	Rule
Price	Valid for 7 days. Old data is automatically deleted + re-scraped.
Warranty	Valid for 30 days. out_of_warranty → PERMANENT (never opens a browser again). in_warranty → rechecked after 30 days.

The scraper, API, and Apify all use this same DB. Maintenance: python cache_db.py (cleans old data + stats).

3. HTTP API (for RapidAPI) — api_server.py

pip install flask waitress
python gen_key.py new "customer name"          # create your API key
set RAPIDAPI_PROXY_SECRET=<from-RapidAPI>       # block RapidAPI bypass
python api_server.py                            # dev:  http://localhost:8000
waitress-serve --port=8000 api_server:app       # production

endpoint	purpose
GET /health	public, is the server alive?
GET /price?model=iphone-13&warranty=true	price (+ optional warranty); brand/city/variant too
GET /brands	available brands

Auth: header X-API-Key: <key> or through RapidAPI. Rate limit API_RATE_PER_MIN (default 60/min). RapidAPI handles billing + keys itself.

4. Apify Actor — .actor/, Dockerfile, apify_main.py

$npm i -g apify-cli && apify login && apify push

Input: brand/model/variant/city/warranty/limit. The result goes into an Apify dataset. Local test: python apify_main.py (or INPUT.json).

5. Free proxies (only on block) — proxies.py

The normal flow is direct + fast. Only on 429/403 does it retry via a GitHub free-proxy. Test: python proxies.py.

6. Security

• The API key file stores only the SHA-256 hash (not plaintext).
• Per-key rate limit, input validation, request-size cap, debug OFF, no stack-trace leaks, secrets via env var.

---

If it ever breaks (Cashify changed the site)

1. python cashify_scraper.py check <some-variant-url> — see which price strategy is matching.
2. Update the CONFIG section at the top of cashify_scraper.py:
   • PRICE_STRATEGIES → price pattern.
   • for the city: OAUTH_URL, REGIONS_URL, or the cookie name.
   • EXACT_VALUE_BUTTON_TEXTS / WARRANTY_QUESTION_HINTS → warranty.