ADRES Colombian Document Scraper

This is a Crawlee + PlaywrightCrawler + Camoufox + TypeScript crawler for querying the Colombian ADRES (Administradora de los Recursos del Sistema General de Seguridad Social en Salud) database.

Features

• Support for 14 Colombian document types (see E_DOCUMENT_TYPE in src/types.ts)
• Automatic document number cleaning and validation
• CLI and environment variable configuration
• Headless and headful browser support
• Proxy rotation support

If you're looking for Crawlee examples or want to learn more visit:

• Documentation
• Examples

Document Types

Valid values for DOCUMENT_TYPE:

• TI (TARJETA_IDENTIDAD) - Tarjeta de Identidad
• CC (CEDULA_CIUDADANIA) - Cédula de Ciudadanía
• MS (MENOR_SIN_IDENTIFICACION) - Menor sin Identificación
• SC (SALVO_CONDUCTO) - Salvo Conducto
• AS (ADULTO_SIN_IDENTIFICACION) - Adulto sin identificación
• CD (CARNET_DIPLOMATICO) - Carnet diplomático
• NI (NUMERO_IDENTIFICACION_TRIBUTARIA) - Número Identificación Tributaria
• PA (PASAPORTE) - Pasaporte
• RC (REGISTRO_CIVIL) - Registro Civil
• PT (PERMISO_POR_PROTECCION_TEMPORAL) - Permiso por Protección Temporal
• NU (NO_UNICO_DE_ID_PERSONAL) - No. Único de Id. Personal
• CN (CERTIFICADO_NACIDO_VIVO) - Certificado Nacido Vivo
• PE (PER_ESPECIAL_PERMANENCIA) - Per Especial Permanencia
• CE (CEDULA_DE_EXTRANJERIA) - Cédula de Extranjería

Run (local)

npm ci

# Via CLI arguments
npm run start:dev CEDULA_CIUDADANIA 1234567890

# Or via environment variables
cp .env.example .env
# Edit .env with your values
npm run start:dev

Environment variables:

# REQUIRED - CAPTCHA Service (required for form submission)
CAPTCHA_SOLUTION_SERVICE=2captcha  # 2captcha, anticaptcha, or capmonster
CAPTCHA_API_KEY=...                # Your API key for the CAPTCHA service

# REQUIRED - Document Input
DOCUMENT_TYPE=CC                   # CC, TI, PA, NU, etc. (see Document Types section)
DOCUMENT_NUMBER=...                # Colombian ID number (4-15 digits)

# OPTIONAL - Browser & Logging
HEADLESS=true|false                # Default: true
CRAWLEE_LOG_LEVEL=INFO             # Default: INFO (DEBUG, INFO, WARN, ERROR)
CRAWLEE_STORAGE_DIR=./storage      # Default: ./storage

# OPTIONAL - Concurrency
MIN_CONCURRENCY=1                  # Default: 1
MAX_CONCURRENCY=10                 # Default: 10
MAX_REQUESTS_PER_CRAWL=1           # Default: 1

Docker

docker build -t biodatos-adres:local .
docker run --rm -it \
  -e HEADLESS=true \
  -e CRAWLEE_LOG_LEVEL=INFO \
  -e CRAWLEE_STORAGE_DIR=./storage \
  -e DOCUMENT_TYPE=CEDULA_CIUDADANIA \
  -e DOCUMENT_NUMBER=1234567890 \
  -e PROXY_URLS="http://user:pass@proxy:3128" \
	-e MIN_CONCURRENCY=1 -e MAX_CONCURRENCY=10 \
	-v "$PWD/storage:/home/myuser/storage" \
	biodatos-adres:local