US H-1B Visa Sponsorship Data Scraper

Search H-1B visa sponsorship data from DOL OFLC LCA disclosures. Find employer sponsorship history, wages, job titles, and locations across fiscal years.

Data Source

• URL: https://www.dol.gov/agencies/eta/foreign-labor/performance
• Format: XLSX files (80-250 MB per fiscal year)
• Rate limits: None (direct file downloads)
• Auth required: No
• Coverage: FY2020-FY2026, ~800K+ records per fiscal year
• Updates: Quarterly (Q1-Q4), Q4 contains the full fiscal year

Output Example (real data from FY2024)

{
    "case_number": "I-200-24186-636144",
    "case_status": "Certified",
    "employer_name": "GOOGLE LLC",
    "employer_city": "MOUNTAIN VIEW",
    "employer_state": "CA",
    "job_title": "SENIOR STAFF SOFTWARE ENGINEER",
    "soc_code": "15-1252",
    "soc_title": "SOFTWARE DEVELOPERS",
    "wage_from": 296000.0,
    "wage_to": 296000.0,
    "wage_unit": "Year",
    "worksite_city": "MOUNTAIN VIEW",
    "worksite_state": "CA",
    "visa_class": "H-1B",
    "received_date": "2024-07-03",
    "decision_date": "2024-07-10",
    "begin_date": "2024-10-01",
    "end_date": "2027-09-30",
    "fiscal_year": 2024
}

Output Schema

Field	Type	Description
case_number	string	DOL case ID (e.g. I-200-24186-636144)
case_status	string	Certified, Certified - Withdrawn, Denied, Withdrawn
employer_name	string	Sponsoring employer name
employer_city	string	Employer city
employer_state	string	Employer 2-letter state code
job_title	string	Position title
soc_code	string	SOC occupation code (e.g. 15-1252)
soc_title	string	SOC occupation title
wage_from	float	Base salary (lower bound)
wage_to	float	Base salary (upper bound)
wage_unit	string	Year, Month, Bi-Weekly, Week, Hour
worksite_city	string	Work location city
worksite_state	string	Work location 2-letter state code
visa_class	string	H-1B, H-1B1, or E-3
received_date	string	Date DOL received the application
decision_date	string	Date DOL made the decision
begin_date	string	Employment start date
end_date	string	Employment end date
fiscal_year	int	DOL fiscal year of the disclosure file

Input Parameters

Parameter	Type	Required	Description
employer_name	string	No	Filter by employer (case-insensitive partial match)
job_title	string	No	Filter by job title (case-insensitive partial match)
worksite_state	string	No	Filter by 2-letter US state code (e.g. CA, NY, TX)
worksite_city	string	No	Filter by worksite city (case-insensitive partial match)
min_wage	number	No	Minimum annual salary (wages auto-annualized for comparison)
max_wage	number	No	Maximum annual salary (wages auto-annualized for comparison)
visa_class	string	No	H-1B (default), H-1B1, or E-3
case_status	string	No	Certified, Certified - Withdrawn, Denied, Withdrawn
fiscal_years	int[]	No	Fiscal years to fetch (default: [2025])
max_results	integer	No	Max results (default: 1,000 / max: 100,000)
database_url	string	No	PostgreSQL URL for persistence
proxy_configuration	object	No	Apify proxy config

Database

• Database name: datamon_us_h1b_visa
• Table: lca_records
• Upsert key: case_number
• Batch size: 1,000 rows per INSERT (optimized for 800K+ record files)
• Index: composite on (fiscal_year, employer_name)

Pricing (Apify Store)

• Model: Pay-per-event (PPE)
• Price: $3 / 1,000 results

Development

# Local runner (no Apify dependency)
cd actors/us-h1b-visa
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Search by employer (no DB)
python runner.py --no-db --employer "GOOGLE" --fiscal-years 2024 --max-results 10

# Filter by state + salary
python runner.py --no-db --employer "GOOGLE" --worksite-state CA --min-wage 200000 --fiscal-years 2024

# Filter by job title
python runner.py --no-db --job-title "DATA SCIENTIST" --worksite-state NY --fiscal-years 2025

# Persist to PostgreSQL
DATABASE_URL="postgresql+asyncpg://user:pass@host:5432/datamon_us_h1b_visa" \
    python runner.py --employer "META" --fiscal-years 2024,2025

# Apify run
apify run --purge

Technical Notes

• Memory efficient: Uses openpyxl read_only=True to stream XLSX rows without loading the full file (~80-250 MB) into memory
• Wage annualization: Hourly (x2080), monthly (x12), bi-weekly (x26), weekly (x52) -- applied only for filtering, raw values stored as-is
• Column variants: DOL column names change across fiscal years -- the scraper maps 18 known header variants to canonical field names
• URL resolution: HEAD request to resolve the correct DOL download URL before fetching (FY2020-2025 and FY2026+ use different URL patterns)

Status

• Scraper implementation (XLSX streaming + in-memory filtering)
• Database models + migrations
• Input/output schemas
• DB persistence (batch upsert, 1000/batch)
• Local runner (runner.py)
• Verified against live DOL data (FY2024 Q4, 79.2 MB)
• Apify Store listing
• Tests