Instagram Profile Scraper

A robust Apify Actor for scraping Instagram profile information including posts, followers, biography, and more. This actor extracts comprehensive profile data from public Instagram accounts with smart proxy management and real-time data saving.

Why Choose Us?

• Comprehensive Data Extraction: Get complete profile information including latest posts, follower counts, biography, external links, and related accounts
• Smart Proxy Management: Automatic fallback system (no proxy → datacenter → residential) ensures maximum reliability and success rates
• Real-Time Data Saving: Data is saved incrementally to the output table as each profile is scraped, so you never lose progress
• Bulk Processing: Process multiple profiles efficiently with built-in rate limiting (1-2 second delays between requests)
• Clean Progress Logging: See real-time progress and results without internal technical details cluttering the logs
• Production-Ready: Built with error handling, retry logic, and graceful failure management

Key Features

• ✅ Profile Information: Username, full name, biography, follower/following counts, post count
• ✅ Latest Posts: Extract up to 12 latest posts with images, captions, hashtags, mentions, and engagement metrics
• ✅ Related Accounts: Get similar/related Instagram accounts
• ✅ External Links: Extract all external URLs from profile bio
• ✅ Account Metadata: Verification status, business account info, profile pictures (standard & HD)
• ✅ IGTV Videos: Extract IGTV video information if available
• ✅ Optional About Section: Extract date joined and country (slower, requires additional parsing)
• ✅ Smart Proxy Fallback: Automatic proxy switching for maximum reliability
• ✅ Real-Time Output: See results appear in the output table as they're scraped

Input

JSON Example

{
  "startUrls": [
    "https://www.instagram.com/humansofny/",
    "https://www.instagram.com/username2/",
    "username3"
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "includeAboutSection": false
}

Input Fields

• startUrls (required): Array of Instagram profile URLs or usernames

  • Accepts full URLs: https://www.instagram.com/username/
  • Accepts just usernames: username
  • Supports bulk input: Add multiple profiles in the array
  • 💬 For custom solutions or feature requests, contact us at dev.scraperengine@gmail.com

• proxyConfiguration (optional): Proxy settings

  • useApifyProxy: Set to true to use Apify proxy (default: false - no proxy)
  • Smart Fallback System:
    • By default, starts with no proxy (direct connection)
    • If Instagram blocks/rejects → automatically falls back to datacenter proxy
    • If datacenter is blocked → automatically falls back to residential proxy
    • If residential fails → retries 3 times with residential proxy
    • Once fallback occurs, sticks with residential proxy for all remaining requests

• includeAboutSection (optional): Extract date joined and country

  • Default: false
  • Set to true to extract account creation date and country information
  • Note: Slower option that requires additional HTML parsing

Output

Output Format

Data is saved in real-time to the Apify dataset. Each profile is saved as a separate item in the output table.

JSON Example

[
  {
    "inputUrl": "https://www.instagram.com/humansofny",
    "id": "242598499",
    "username": "humansofny",
    "url": "https://www.instagram.com/humansofny",
    "fullName": "Humans of New York",
    "biography": "New York City, one story at a time. Created by Brandon Stanton...",
    "externalUrls": [
      {
        "title": "Dear New York Book",
        "lynx_url": "https://l.instagram.com/...",
        "url": "https://bit.ly/dearnewyorkbook",
        "link_type": "external"
      }
    ],
    "externalUrl": "https://bit.ly/dearnewyorkbook",
    "externalUrlShimmed": "https://l.instagram.com/...",
    "followersCount": 12802590,
    "followsCount": 673,
    "postsCount": 5839,
    "verified": true,
    "isBusinessAccount": false,
    "private": false,
    "profilePicUrl": "https://instagram.fdac142-1.fna.fbcdn.net/...",
    "profilePicUrlHD": "https://instagram.fdac142-1.fna.fbcdn.net/...",
    "latestPosts": [
      {
        "id": "3742534382104984578",
        "type": "Sidecar",
        "shortCode": "DPwJzEVDbwC",
        "caption": "A few years ago I received a DM...",
        "hashtags": [],
        "mentions": ["zohrankmamdani"],
        "url": "https://www.instagram.com/p/DPwJzEVDbwC/",
        "commentsCount": 3841,
        "likesCount": 254147,
        "timestamp": "2025-10-13T20:14:46Z",
        "images": ["https://..."],
        "childPosts": [...]
      }
    ],
    "relatedProfiles": [],
    "latestIgtvVideos": [],
    "igtvVideoCount": 0,
    "fbid": "17841401154973790"
  }
]

Output Fields

Profile Information

• inputUrl: Original input URL or username
• id: Instagram user ID
• username: Instagram username
• url: Full Instagram profile URL
• fullName: Display name/full name
• biography: Profile bio text
• verified: Whether account is verified (boolean)
• isBusinessAccount: Whether it's a business account (boolean)
• private: Whether account is private (boolean)
• profilePicUrl: Standard profile picture URL
• profilePicUrlHD: High-definition profile picture URL

Statistics

• followersCount: Number of followers
• followsCount: Number of accounts following
• postsCount: Total number of posts
• igtvVideoCount: Number of IGTV videos

Links & External Data

• externalUrl: Main external URL from bio
• externalUrlShimmed: Instagram-shimmed external URL
• externalUrls: Array of all external links from bio with titles

Posts Data

• latestPosts: Array of latest 12 posts, each containing:
  • id, type (Image/Video/Sidecar), shortCode, caption
  • hashtags, mentions, url, commentsCount, likesCount
  • timestamp, images, childPosts (for Sidecar posts)
  • videoUrl, videoViewCount (for Video posts)
  • taggedUsers, isPinned, isCommentsDisabled

Related Data

• relatedProfiles: Array of similar/related Instagram accounts
• latestIgtvVideos: Array of IGTV videos (if any)

Optional Fields (if includeAboutSection is true)

• dateJoined: Account creation date (ISO format)
• country: Country information (if available)

🚀 How to Use the Actor

Via Apify Console

1. Log in to https://console.apify.com and navigate to Actors
2. Find your actor (instagram-profile-scraper) and click it
3. Configure inputs:
   • Add Instagram profile URLs or usernames in the startUrls field
   • Optionally configure proxy settings (default: no proxy with automatic fallback)
   • Toggle "Include About Section" if you need date joined and country info
4. Click Start to run the actor
5. Monitor progress in real-time through the logs
6. View results in the OUTPUT tab as they're scraped
7. Export results to JSON, CSV, or other formats when complete

Input Format

The startUrls field accepts:

• Full URLs: https://www.instagram.com/username/
• Just usernames: username
• Mix of both formats

Example:

https://www.instagram.com/humansofny/
username2
https://www.instagram.com/username3/

How It Works

1. Initialization: Actor starts with no proxy (direct connection to Instagram)
2. Profile Processing: For each profile:
   • Extracts username from URL if needed
   • Fetches Instagram headers and app ID
   • Makes API request to get profile data
   • Transforms data to standardized format
   • Saves to output table immediately
3. Proxy Fallback: If requests are blocked:
   • Automatically switches to datacenter proxy
   • If still blocked, switches to residential proxy
   • Retries up to 3 times with residential if needed
4. Real-Time Saving: Each profile is saved as soon as it's scraped
5. Progress Tracking: Logs show progress and results in real-time

Best Use Cases

• Social Media Research: Analyze competitor profiles, engagement metrics, and content strategies
• Influencer Discovery: Find and analyze potential influencers for marketing campaigns
• Content Analysis: Study post patterns, hashtag usage, and content performance
• Market Research: Gather data on brand presence, audience demographics, and market trends
• Data Collection: Build comprehensive datasets for analysis, reporting, and business intelligence
• Account Monitoring: Track profile changes, follower growth, and engagement over time

Frequently Asked Questions

Q: Does this work with private accounts?
A: No, this actor only scrapes publicly available information from public Instagram profiles. Private accounts cannot be accessed.

Q: Will I get blocked by Instagram?
A: The actor includes smart proxy management and rate limiting (1-2 second delays) to minimize blocking. If requests are blocked, it automatically falls back through the proxy chain (datacenter → residential) to ensure maximum success rates.

Q: How many profiles can I scrape?
A: The actor can process multiple profiles efficiently. Rate limiting is built-in to avoid overwhelming Instagram's servers. There's no hard limit, but be mindful of Instagram's rate limits and terms of service.

Q: What if a profile fails to scrape?
A: The actor will log a warning and continue with the next profile. All successfully scraped data is saved incrementally, so you won't lose progress if some profiles fail.

Q: Can I scrape posts older than the latest 12?
A: Currently, the actor scrapes the latest 12 posts per profile. For historical posts beyond that, you would need Instagram's pagination API (not included in this actor).

Q: How does the proxy fallback work?
A: The actor starts with no proxy. If Instagram blocks the request, it automatically tries datacenter proxy. If that's blocked, it switches to residential proxy. Once a fallback occurs, it sticks with residential for all remaining requests to maintain consistency.

Q: Is the data saved in real-time?
A: Yes! Each profile is saved to the output table immediately after it's scraped. You can see results appearing in the OUTPUT tab as the actor runs.

Q: What's the difference between standard and HD profile pictures?
A: profilePicUrl is the standard resolution, while profilePicUrlHD provides a higher resolution version of the profile picture.

Support and Feedback

For issues, questions, custom solutions, or feature requests, please contact us at dev.scraperengine@gmail.com.

Cautions

• Data is collected only from publicly available sources
• No data is taken from private accounts or password-protected content
• The end user is responsible for ensuring legal compliance (spam laws, privacy, data protection, GDPR, etc.)
• Respect Instagram's Terms of Service and rate limits
• Use responsibly and ethically
• Do not use for spam, harassment, or any illegal activities

Technical Details

• Language: Python 3
• Framework: Apify SDK
• HTTP Client: aiohttp (async)
• Proxy Support: Apify Proxy with automatic fallback
• Data Storage: Apify Dataset (real-time incremental saving)
• Rate Limiting: 1-2 second delays between requests
• Retry Logic: 3 retries with exponential backoff
• Error Handling: Graceful failure with detailed logging

---

Version: 0.1
Last Updated: 2025
Actor Name: instagram-profile-scraper