🎓 Internshala Scraper

Extract internship listings from Internshala.com with ease! Perfect for job boards, market research, and AI-powered recruitment tools.

✨ Features

• 🔍 Keyword Search - Find internships by skills, role, or industry
• 📍 Location Filtering - Filter by city (Bangalore, Mumbai, Delhi, etc.)
• 📊 Structured Data - Extracts JSON-LD schema.org JobPosting data
• 🚀 Fast & Reliable - Uses httpx + BeautifulSoup for efficient scraping
• 🌐 Proxy Support - Built-in Apify proxy integration (residential)
• 🔄 Pagination - Automatically handles multiple pages
• 💾 Immediate Push - Results streamed as they're found

🤖 AI & MCP Integration

This actor is AI-ready and MCP-compatible! Use it with:

• 🧠 Claude Code (Anthropic MCP)
• 🤖 ChatGPT (OpenAI Actions)
• 🦙 LangChain & LlamaIndex
• 📊 Zapier & Make.com

Perfect for building AI agents that need real-time internship data!

📋 Input Configuration

Field	Type	Description	Default
searchQuery	string	Keywords to search (e.g., "python", "marketing")	"python"
location	string	City name (e.g., "Bangalore", "Mumbai")	"Bangalore"
maxResults	integer	Maximum internships to scrape (1-1000)	50
proxyConfiguration	object	Apify proxy settings	RESIDENTIAL

Example Input

{
  "searchQuery": "python developer",
  "location": "Bangalore",
  "maxResults": 100,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

📤 Output Format

Each internship listing includes:

Field	Type	Description
title	string	Internship title/role
company	string	Company name
location	string	Location of internship
duration	string	Internship duration
stipend	string	Stipend/salary range
skills	string	Required skills
description	string	Full job description
url	string	Direct link to listing
scrapedAt	string	ISO timestamp

Example Output

{
  "title": "Python Developer - Internship",
  "company": "Tech Startup Inc.",
  "location": "Bangalore",
  "duration": "3 months",
  "stipend": "INR 15000-25000",
  "skills": "Python, Django, REST APIs",
  "description": "We are looking for a passionate Python developer...",
  "url": "https://internshala.com/internships/detail/123456",
  "scrapedAt": "2026-08-21T10:30:00.000Z"
}

🚀 Quick Start

Run on Apify Platform

1. Create an Apify account at apify.com
2. Create new actor and upload this code
3. Configure your search query and location
4. Click Start and get results!

Run Locally

# Install dependencies
pip install -r requirements.txt

# Set environment variable
export APIFY_INPUT_JSON='{"searchQuery":"python","location":"Bangalore","maxResults":20}'

# Run the scraper
python -m src.main

Docker

# Build image
docker build -t internshala-scraper .

# Run container
docker run -e APIFY_INPUT_JSON='{"searchQuery":"python","location":"Bangalore","maxResults":20}' internshala-scraper

🔧 How It Works

1. URL Construction - Builds search URLs based on keywords and location
2. HTML Fetching - Downloads pages using httpx with proxy support
3. JSON-LD Extraction - Parses structured schema.org JobPosting data from <script> tags
4. Data Normalization - Converts to clean, consistent format
5. Immediate Push - Streams results to Apify dataset in real-time
6. Pagination - Continues to next page until maxResults reached

📊 Use Cases

• 🎯 Job Boards - Aggregate internships for your platform
• 📈 Market Research - Analyze hiring trends and salary ranges
• 🤖 AI Recruiters - Feed data to AI agents for candidate matching
• 📧 Job Alerts - Build notification systems for new postings
• 📊 Analytics - Track internship market dynamics
• 🔗 Integration - Connect with CRMs, ATSs, or automation tools

🛡️ Best Practices

• ✅ Use residential proxies for best results
• ✅ Set reasonable maxResults (50-200 recommended)
• ✅ Run daily/weekly for fresh data
• ✅ Respect rate limits and be polite to the site
• ✅ Cache results to avoid redundant scraping

🐛 Error Handling

The actor includes robust error handling:

• ❌ Gracefully handles missing data (returns null for missing fields)
• ❌ Continues on single-page failures
• ❌ Logs errors without crashing
• ❌ Stops at reasonable pagination limits

📝 Logging

Progress is logged throughout:

• 📊 Every 10 results scraped
• 📄 Each page URL being scraped
• ⚠️ Errors and warnings
• ✅ Final completion summary

🤝 Contributing

Contributions welcome! Feel free to:

• 🐛 Report bugs
• 💡 Suggest features
• 🔧 Submit pull requests
• ⭐ Star the repository

📄 License

MIT License - feel free to use in personal and commercial projects!

🙋 Support

Need help? Have questions?

• 📧 Contact via Apify platform
• 💬 Open an issue on GitHub
• 📖 Check Apify documentation

---

Built with ❤️ for the AI and automation community

This actor respects robots.txt and website terms of service. Use responsibly.