Foundit Jobs Scraper

Overview

Pull detailed employment data from Foundit.in (formerly Monster India), a prominent Indian job search platform connecting millions of job seekers with employers. This actor queries the Foundit.in middleware API directly, ensuring lightweight, high-performance, and reliable data extraction without the overhead of heavy browser automation.

Features

• Direct integration with the Foundit.in middleware API for high-speed scraping
• Comprehensive company and employer information
• Salary and CTC package data in Indian Rupees
• Skills-based matching and filtering support
• Proxy rotation with automatic fallback (residential → datacenter)
• Automatic retry on failures with exponential backoff
• Deduplication of results by compound key (url, title, experience, requirements)
• Dataset validation with auto-fix capability

Supported Inputs

Field	Type	Default	Description
keyword	string	"software engineer"	Search terms for job discovery
location	string	"India"	Geographic filter for results
country	string	"IN"	Country code for proxy routing
maxItems	integer	50	Upper limit on extracted listings
proxyEnabled	boolean	true	Toggle proxy rotation on/off
sortBy	string	"relevance"	Result ordering (relevance/date/salary)
jobType	string	""	Employment type filter
experienceLevel	string	""	Seniority level filter
datePosted	string	""	Recency filter (24h/3d/7d/14d/30d)
remoteOnly	boolean	false	Restrict to remote positions only
includeCompanyDetails	boolean	true	Fetch extra company information
includeSalary	boolean	true	Include compensation data

Output Format

Each scraped listing produces a JSON object with these fields:

{
	"jobTitle": "Senior Software Engineer",
	"companyName": "Mphasis",
	"location": "Bengaluru, India",
	"salary": "5-7 LPA",
	"jobType": "Full time",
	"experienceLevel": "1-3 Years",
	"postedDate": "4 hours ago",
	"applyUrl": "https://mphasis.ripplehire.com/candidate/?token=ty4DfyWddnOrtpclQeia&source=CAREERSITE#detail/job/868349",
	"companyUrl": "https://www.foundit.in/search/mphasis-488098-jobs-career",
	"description": "Mphasis Limited is an Indian multinational information technology services and consulting company based in Bangalore.The company provides infrastructure technology and applications outsourcing services, as well as architecture guidance, application development and integration, and application management services. It serves financial services, telecom, logistics, and technology industries. Mphasis was ranked #7 in India IT companies and overall #189 by Fortune India 500 in 2019. In April 2016, Hewlett Packard Enterprise sold the majority of its stake in Mphasis to Blackstone Group LP for around US$1 billion.",
	"requirements": [
		"Fast API",
		"GitHub Actions",
		"CI CD tools",
		"containerization",
		"performance optimization",
		"Algorithms",
		"S3",
		"RDS",
		"Microservices",
		"Sql",
		"Nosql",
		"Lambda",
		"Cloudwatch",
		"Docker",
		"Sqs",
		"Sns",
		"data structures",
		"Asynchronous programming",
		"Kubernetes",
		"Python",
		"Api Gateway",
		"AWS"
	],
	"benefits": [],
	"sourcePortal": "foundit",
	"country": "IN",
	"scrapedAt": "2026-06-27T08:32:11.912Z"
}

Proxy Handling

A robust proxy pipeline handles connection failures transparently.

1. Apify Residential Proxy (country-targeted) — First choice for foundit
2. Apify Residential Proxy (any region) — Fallback if country proxy unavailable
3. Apify Datacenter Proxy — Secondary fallback for cost efficiency
4. Direct Connection — Last resort when all proxies fail

Proxies auto-rotate on each request. Blocked sessions are discarded and replaced automatically.

Retry Logic

Automatic recovery from failures with progressive retry intervals.

• Maximum 5 retries per request
• Fresh HTTP session and fingerprint generation on each retry
• Automatic proxy rotation between attempts
• Blocked status codes (401, 403, 429) trigger session refresh
• Configurable request timeout (30 seconds)

Anti-block Handling

Comprehensive stealth configuration keeping requests under anti-bot radar.

• Powered by got-scraping to mock TLS/TCP and HTTP headers automatically
• Human-like headers setup matching real browser signatures
• Session pool with automatic rotation on blocks
• Rate limit detection (HTTP 429) with backoff

Sample Input

{
    "keyword": "data analyst",
    "location": "India",
    "maxItems": 25,
    "proxyEnabled": true,
    "sortBy": "date",
    "remoteOnly": false
}

Sample Output

{
    "jobTitle": "AI Software Engineer",
    "companyName": "S&P Global Market Intelligence",
    "location": "India",
    "salary": "0-0 INR",
    "jobType": "Full time",
    "experienceLevel": "2-6 Years",
    "postedDate": "13 hours ago",
    "applyUrl": "https://careers.spglobal.com/jobs/323823?lang=en-us",
    "companyUrl": "https://www.foundit.in/search/sp-global-market-intelligence-502334-jobs-career",
    "description": "S&P Global Inc. (prior to April 2016 McGraw Hill Financial, Inc., and prior to 2013 The McGraw&#8211&#x3B;Hill Companies, Inc.) is an American publicly traded corporation headquartered in Manhattan, New York City. Its primary areas of business are financial information and analytics. It is the parent company of S&P Global Ratings, S&P Global Market Intelligence, S&P Global Mobility, S&P Global Engineering Solutions, S&P Global Sustainable1, and S&P Global Commodity Insights, CRISIL, and is the majority owner of the S&P Dow Jones Indices joint venture. "S&P" is a shortening of "Standard and Poor's",
    "requirements": [
        "LLM GenAI fundamentals",
        "observability tools",
        "OpenTelemetry",
        "dbt",
        "Elk",
        "Prometheus",
        "Grafana",
        "Sql",
        "Pandas",
        "Gcp",
        "Spark",
        "Azure",
        "Python",
        "AWS"
    ],
    "benefits": [],
    "sourcePortal": "foundit",
    "country": "IN",
    "scrapedAt": "2026-06-27T08:32:11.907Z"
}

Usage

Local Development

# Install dependencies
npm install

# Set Apify token (required for proxy)
export APIFY_TOKEN=your_token_here

# Run the actor
npm start

# Validate scraped data
node dataset-validator.js

Apify Platform

# Login to Apify
apify login

# Push actor to platform
apify push

# Run from Apify Console or API

Deployment

1. Ensure all dependencies are installed: npm install
2. Authenticate with Apify: apify login
3. Deploy the actor: apify push
4. Configure input in the Apify Console
5. Schedule runs or trigger via API / webhooks

Limitations

• Results depend on the portal's internal middleware API; endpoint changes may require updates
• Some job details (salary, description) may not be available for all listings
• Rate limiting by the portal may reduce throughput during high-volume scrapes
• API changes may require updating request headers or query parameters
• Maximum items per run is capped at 1000 to prevent excessive resource usage
• Proxy costs apply when using Apify residential or datacenter proxies