Instagram Profile Scraper

Extract comprehensive data from Instagram profiles including posts, reels, photos, and engagement metrics. This scraper supports automatic cookie rotation via MongoDB for reliable, uninterrupted scraping.

🚀 Features

Profile Data Extraction

• Complete Profile Information: Username, full name, biography, follower/following counts, verification status
• Profile Picture: High-resolution profile picture URL
• Account Type Detection: Identifies business accounts, private accounts, and verified profiles
• Post Count: Total number of posts on the profile

Post Data Extraction

• Unlimited Posts: Extract up to any number of posts (configurable)
• Rich Post Data:
  • Caption/description
  • Publication date and time
  • Like count
  • Comment count
  • View count (when available)
  • Post type (image, video, carousel)
  • Media URLs (images and videos)
  • Post URL

Media Extraction

• High-Quality Media URLs: Direct links to images and videos
• Carousel Support: Extracts all media from multi-image/video posts
• Video Support: Gets video URLs for reels and IGTV posts

Advanced Features

• MongoDB Cookie Rotation: Automatic cookie management with failover
• Email Notifications: Get alerts when cookies fail or are exhausted
• Humanized Behavior: Simulates human browsing patterns to avoid detection
• Rate Limiting Protection: Built-in delays and randomization
• Session Management: Save and reuse sessions across runs
• Block Detection: Automatically detects Instagram blocks and rotates cookies

📥 Input Parameters

Parameter	Type	Required	Default	Description
username	String	✅ Yes	-	Instagram username (without @) to scrape
maxPosts	Integer	No	100	Maximum number of posts to extract (0 = all posts)
cookies	String/Array	No	-	Instagram cookies for authentication (JSON format)
humanizeBehavior	Boolean	No	true	Simulate human-like behavior (recommended)
sessionName	String	No	"default_session"	Session name for storing cookies

Authentication Notes

MongoDB Cookie Rotation (Recommended):

• Cookies are automatically managed via MongoDB
• No need to provide cookies in input
• Automatic failover when cookies fail
• Email notifications on cookie issues
• Just leave the cookies field empty to use MongoDB rotation

Manual Cookie Input (Optional):

• Provide your own Instagram cookies if not using MongoDB
• Cookies should be in JSON array format
• Must include sessionid cookie for authentication

📤 Output Format

Profile Data

{
  "username": "ambuzzway",
  "full_name": "Ambuzzway",
  "biography": "News that speaks. Stories that matter.",
  "profile_pic_url": "https://...",
  "is_verified": false,
  "is_private": false,
  "is_business": false,
  "followers_count": 308,
  "following_count": 219,
  "posts_count": 45,
  "profile_url": "https://www.instagram.com/ambuzzway/"
}

Post Data

{
  "description": "Post caption with hashtags...",
  "post_url": "https://www.instagram.com/p/ABC123/",
  "pub_date": "2025-11-23T21:50:42",
  "like_count": 3302,
  "comment_count": 1332,
  "view_count": 0,
  "media_urls": [
    "https://instagram.com/.../video.mp4",
    "https://instagram.com/.../image.jpg"
  ],
  "post_type": "video",
  "username": "ambuzzway",
  "scraped_at": "2025-12-05T11:39:54.924184",
  "authorMeta": {
    "username": "ambuzzway",
    "full_name": "Ambuzzway",
    "biography": "News that speaks...",
    "profile_pic_url": "https://...",
    "is_verified": false,
    "is_private": false,
    "is_business": false,
    "followers_count": 308,
    "following_count": 219,
    "posts_count": 45,
    "profile_url": "https://www.instagram.com/ambuzzway/"
  }
}

🎯 Use Cases

1. Content Research: Analyze successful posts and engagement patterns
2. Competitor Analysis: Track competitor content and performance
3. Influencer Vetting: Verify follower counts and engagement rates
4. Brand Monitoring: Monitor brand mentions and user-generated content
5. Media Archival: Download and archive Instagram content
6. Market Research: Analyze trends and content strategies
7. Data Analysis: Export data for analytics and reporting

🔧 How to Use

On Apify Platform

1. Navigate to the Actor: Find "Instagram Profile Scraper" in Apify Store
2. Enter Username: Input the Instagram username (without @)
3. Configure Settings: Set max posts, humanize behavior, etc.
4. Start Scraping: Click "Start" and wait for results
5. Download Data: Export results as JSON, CSV, or Excel

Local Development

# Install dependencies
pip install -r requirements.txt

# Set up input
echo '{"username": "ambuzzway", "maxPosts": 50}' > storage/key_value_stores/default/INPUT.json

# Run the scraper
apify run

With Apify SDK

from apify_client import ApifyClient

client = ApifyClient('YOUR_APIFY_TOKEN')

# Prepare input
run_input = {
    "username": "ambuzzway",
    "maxPosts": 100,
    "humanizeBehavior": True
}

# Run the actor
run = client.actor("YOUR_ACTOR_ID").call(run_input=run_input)

# Fetch results
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

⚙️ MongoDB Cookie Rotation Setup

The scraper includes automatic cookie rotation via MongoDB for reliable, long-term scraping:

Features

• Automatic Failover: When one cookie fails, automatically rotates to the next
• Round-Robin Selection: Distributes load across multiple accounts
• Failure Tracking: Tracks success/failure rates for each cookie
• Email Alerts: Sends notifications when cookies fail or are exhausted
• Zero Downtime: Never stops scraping due to single cookie failure

Email Notifications

You'll receive alerts for:

• ❌ Individual cookie failures
• 🔴 All cookies exhausted (requires immediate action)
• ⚠️ MongoDB connection issues

Cookie Status

• Active: Cookie is working and available for use
• Failed: Cookie has failed 3+ times and is disabled
• Success Rate: Tracked for each cookie

⚠️ Important Limitations

View Counts

• Not Available: Instagram doesn't expose view counts via web scraping
• Always Returns 0: This is an Instagram platform limitation, not a bug
• Alternatives: View counts are only accessible via Instagram's official API

Rate Limiting

• Instagram has strict rate limits
• Use humanized behavior to reduce detection risk
• Consider using multiple cookies for high-volume scraping

Private Accounts

• Cannot scrape private accounts unless authenticated with a follower account
• Provide cookies from an account that follows the private profile

Content Restrictions

• Some content may be age-restricted or geo-blocked
• Login required for sensitive content

🛡️ Best Practices

1. Use MongoDB Cookie Rotation: Provides reliability and automatic failover
2. Enable Humanized Behavior: Reduces detection and blocking risk
3. Respect Rate Limits: Don't scrape too many profiles too quickly
4. Monitor Email Alerts: Act quickly when cookies fail
5. Rotate Fresh Cookies: Replace failed cookies regularly
6. Test Small First: Start with maxPosts: 10 to verify everything works
7. Use Sessions: Save sessions for faster subsequent runs

🔒 Privacy & Compliance

• Only scrapes publicly available data
• Respects Instagram's robots.txt (when possible)
• No password cracking or unauthorized access
• Users responsible for compliance with Instagram Terms of Service
• Recommended for research, analysis, and legitimate business purposes only

📊 Output Formats

Results can be downloaded in multiple formats:

• JSON: Full structured data with all fields
• CSV: Tabular format for Excel/spreadsheets
• Excel: .xlsx format with formatted columns
• HTML: Web-viewable table format
• RSS: Feed format for monitoring

🐛 Troubleshooting

"No posts extracted"

• Check if profile is private (requires authenticated cookies)
• Verify username is correct (case-sensitive)
• Check if Instagram is blocking requests (use MongoDB rotation)

"Cookies failed" errors

• MongoDB rotation will handle this automatically
• Check email for failure notifications
• Add fresh cookies to MongoDB if all are exhausted

"Action blocked" messages

• Instagram detected automation
• Cookie has been marked as failed and rotated automatically
• Wait a few hours before using that account again

Slow extraction

• This is normal - humanized behavior includes delays
• Disable humanizeBehavior for faster (but riskier) scraping
• Reduce maxPosts to extract fewer posts

💡 Tips for Best Results

1. Start Small: Test with 10-20 posts first
2. Use Fresh Cookies: Regularly update cookies in MongoDB
3. Monitor Emails: Set up email filters for scraper notifications
4. Check Output: Verify data quality before large-scale runs
5. Batch Processing: Process multiple usernames in separate runs
6. Schedule Runs: Use Apify's scheduler for regular updates

📞 Support

For issues, questions, or feature requests:

• Check the troubleshooting section above
• Review the Apify documentation
• Contact support with detailed error messages
• Include input parameters and error logs

🔄 Version History

Latest Version

• ✅ MongoDB cookie rotation with automatic failover
• ✅ Email notifications for cookie failures
• ✅ Enhanced full_name extraction with multiple fallbacks
• ✅ Comprehensive view_count extraction (when available)
• ✅ Improved profile data parsing
• ✅ Better error handling and logging
• ✅ Support for all post types (images, videos, carousels, reels)

📄 License

This scraper is provided as-is for legitimate research and business purposes. Users are responsible for compliance with Instagram's Terms of Service and applicable laws.

---

Note: Instagram's structure and policies change frequently. This scraper is maintained to work with current Instagram layouts, but functionality may be affected by platform changes.