# Internshala Scraper — Indian Internship Data for AI (`fervent_bus/internshala-scraper`) Actor

Extract Indian internship data from Internshala: jobTitle, company, location, salary, skills, duration, stipend. For HR teams, recruiters, job boards. Power AI agents, Claude, ChatGPT via Apify MCP. First Internshala scraper.

- **URL**: https://apify.com/fervent\_bus/internshala-scraper.md
- **Developed by:** [Archit Khurana](https://apify.com/fervent_bus) (community)
- **Categories:** Jobs
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## 🎓 Internshala Scraper

> **Extract internship listings from Internshala.com with ease!** Perfect for job boards, market research, and AI-powered recruitment tools.

[![Apify Actor](https://img.shields.io/badge/Apify-Actor-0078D4?logo=apify)](https://apify.com)
[![Python 3.11](https://img.shields.io/badge/Python-3.11-blue?logo=python)](https://www.python.org/)
[![MCP Compatible](https://img.shields.io/badge/MCP-Compatible-green)](https://modelcontextprotocol.io)

### ✨ Features

- 🔍 **Keyword Search** - Find internships by skills, role, or industry
- 📍 **Location Filtering** - Filter by city (Bangalore, Mumbai, Delhi, etc.)
- 📊 **Structured Data** - Extracts JSON-LD schema.org JobPosting data
- 🚀 **Fast & Reliable** - Uses httpx + BeautifulSoup for efficient scraping
- 🌐 **Proxy Support** - Built-in Apify proxy integration (residential)
- 🔄 **Pagination** - Automatically handles multiple pages
- 💾 **Immediate Push** - Results streamed as they're found

### 🤖 AI & MCP Integration

This actor is **AI-ready** and **MCP-compatible**! Use it with:

- 🧠 **Claude Code** (Anthropic MCP)
- 🤖 **ChatGPT** (OpenAI Actions)
- 🦙 **LangChain** & **LlamaIndex**
- 📊 **Zapier** & **Make.com**

Perfect for building AI agents that need real-time internship data!

### 📋 Input Configuration

| Field | Type | Description | Default |
|-------|------|-------------|---------|
| **searchQuery** | string | Keywords to search (e.g., "python", "marketing") | `"python"` |
| **location** | string | City name (e.g., "Bangalore", "Mumbai") | `"Bangalore"` |
| **maxResults** | integer | Maximum internships to scrape (1-1000) | `50` |
| **proxyConfiguration** | object | Apify proxy settings | RESIDENTIAL |

#### Example Input

```json
{
  "searchQuery": "python developer",
  "location": "Bangalore",
  "maxResults": 100,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

### 📤 Output Format

Each internship listing includes:

| Field | Type | Description |
|-------|------|-------------|
| **title** | string | Internship title/role |
| **company** | string | Company name |
| **location** | string | Location of internship |
| **duration** | string | Internship duration |
| **stipend** | string | Stipend/salary range |
| **skills** | string | Required skills |
| **description** | string | Full job description |
| **url** | string | Direct link to listing |
| **scrapedAt** | string | ISO timestamp |

#### Example Output

```json
{
  "title": "Python Developer - Internship",
  "company": "Tech Startup Inc.",
  "location": "Bangalore",
  "duration": "3 months",
  "stipend": "INR 15000-25000",
  "skills": "Python, Django, REST APIs",
  "description": "We are looking for a passionate Python developer...",
  "url": "https://internshala.com/internships/detail/123456",
  "scrapedAt": "2026-08-21T10:30:00.000Z"
}
```

### 🚀 Quick Start

#### Run on Apify Platform

1. Create an Apify account at [apify.com](https://apify.com)
2. Create new actor and upload this code
3. Configure your search query and location
4. Click **Start** and get results!

#### Run Locally

```bash
## Install dependencies
pip install -r requirements.txt

## Set environment variable
export APIFY_INPUT_JSON='{"searchQuery":"python","location":"Bangalore","maxResults":20}'

## Run the scraper
python -m src.main
```

#### Docker

```bash
## Build image
docker build -t internshala-scraper .

## Run container
docker run -e APIFY_INPUT_JSON='{"searchQuery":"python","location":"Bangalore","maxResults":20}' internshala-scraper
```

### 🔧 How It Works

1. **URL Construction** - Builds search URLs based on keywords and location
2. **HTML Fetching** - Downloads pages using httpx with proxy support
3. **JSON-LD Extraction** - Parses structured schema.org JobPosting data from `<script>` tags
4. **Data Normalization** - Converts to clean, consistent format
5. **Immediate Push** - Streams results to Apify dataset in real-time
6. **Pagination** - Continues to next page until maxResults reached

### 📊 Use Cases

- 🎯 **Job Boards** - Aggregate internships for your platform
- 📈 **Market Research** - Analyze hiring trends and salary ranges
- 🤖 **AI Recruiters** - Feed data to AI agents for candidate matching
- 📧 **Job Alerts** - Build notification systems for new postings
- 📊 **Analytics** - Track internship market dynamics
- 🔗 **Integration** - Connect with CRMs, ATSs, or automation tools

### 🛡️ Best Practices

- ✅ Use **residential proxies** for best results
- ✅ Set reasonable **maxResults** (50-200 recommended)
- ✅ Run **daily/weekly** for fresh data
- ✅ Respect rate limits and be polite to the site
- ✅ Cache results to avoid redundant scraping

### 🐛 Error Handling

The actor includes robust error handling:

- ❌ Gracefully handles missing data (returns `null` for missing fields)
- ❌ Continues on single-page failures
- ❌ Logs errors without crashing
- ❌ Stops at reasonable pagination limits

### 📝 Logging

Progress is logged throughout:

- 📊 Every 10 results scraped
- 📄 Each page URL being scraped
- ⚠️ Errors and warnings
- ✅ Final completion summary

### 🤝 Contributing

Contributions welcome! Feel free to:

- 🐛 Report bugs
- 💡 Suggest features
- 🔧 Submit pull requests
- ⭐ Star the repository

### 📄 License

MIT License - feel free to use in personal and commercial projects!

### 🙋 Support

Need help? Have questions?

- 📧 Contact via Apify platform
- 💬 Open an issue on GitHub
- 📖 Check [Apify documentation](https://docs.apify.com)

***

**Built with ❤️ for the AI and automation community**

*This actor respects robots.txt and website terms of service. Use responsibly.*

# Actor input Schema

## `searchQuery` (type: `string`):

Keywords to search for (e.g., 'python', 'marketing', 'design')

## `location` (type: `string`):

Location to filter internships (e.g., 'Bangalore', 'Mumbai', 'Delhi'). Use 'All' for all locations.

## `maxResults` (type: `integer`):

Maximum number of internships to scrape

## `proxyConfiguration` (type: `object`):

Proxy settings for the scraper

## Actor input object example

```json
{
  "searchQuery": "python",
  "location": "Bangalore",
  "maxResults": 10,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `title` (type: `string`):

Internship title

## `company` (type: `string`):

Company offering the internship

## `location` (type: `string`):

Internship location

## `duration` (type: `string`):

Internship duration

## `stipend` (type: `string`):

Stipend/salary information

## `skills` (type: `string`):

Required skills

## `description` (type: `string`):

Full internship description

## `url` (type: `string`):

URL to the internship posting

## `scrapedAt` (type: `string`):

Timestamp when the data was scraped

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": "python",
    "location": "Bangalore",
    "maxResults": 10,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("fervent_bus/internshala-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQuery": "python",
    "location": "Bangalore",
    "maxResults": 10,
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("fervent_bus/internshala-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQuery": "python",
  "location": "Bangalore",
  "maxResults": 10,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call fervent_bus/internshala-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fervent_bus/internshala-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/d78rp2UjPww0Sr6N3/builds/KTIDaw8SfrKYKbdCu/openapi.json