Instagram Profile & Posts Scraper

Extract complete Instagram profiles and posts with follower stats, engagement metrics, captions, hashtags, media URLs, and related accounts. Supports instant fetch (recent posts) or full scraping, batch processing, concurrency control, polling, and detailed error tracking.

---

✨ Features

• 👤 Profile extraction: Get follower count, following, bio, verified status, and more
• 📸 Posts scraping: Extract recent posts with likes, comments, captions, and media URLs
• 🎯 Two modes: Instant fetch (recent posts) or full scraping (all posts)
• 🔁 Batch processing: Handle multiple profiles simultaneously
• ⚙️ Concurrency control: Adjust processing speed with configurable limits
• 🔄 Automatic polling: Waits for full profile scraping when enabled
• 🔔 Webhook support: Receive results via webhook instead of polling
• 🚦 Rate limiting: Built-in delays to respect API limits
• 💰 Cost tracking: Monitor API usage per profile
• 🧠 Error tracking: All failures logged with detailed error information
• 📊 Rich metadata: Hashtags, related accounts, highlights, engagement scores

---

📚 API Documentation

For complete API reference, endpoint details, and advanced usage examples, visit our official documentation:

Transcript Downloader API Documentation

Get Your API Key • API Pricing

---

🔧 Input Parameters

The actor accepts the following input:

Parameter	Type	Required	Default	Description
profileUrls	array	✅ Yes	-	List of Instagram profile URLs
apiToken	string	✅ Yes	-	Your Transcript Downloader API bearer token
waitForCompletion	boolean	No	true	Whether to wait for full profile scraping (all posts)
maxWaitTime	number	No	10	Max time to wait for scraping completion (1-30 minutes)
pollingInterval	number	No	30	How often to check status when waiting. API recommends 30s for profile operations (10-120 seconds)
maxConcurrency	number	No	2	Max concurrent profile requests (range: 1-5)
includeWebhook	string	No	-	Webhook URL to receive results when profile processing completes. Must be publicly reachable and accept POST requests

📥 Sample Input

Instant Mode (Recent posts)

{
  "profileUrls": [
    "https://www.instagram.com/mrbeast",
    "https://www.instagram.com/cristiano"
  ],
  "apiToken": "your-api-token",
  "waitForCompletion": false,
  "maxConcurrency": 2
}

Full Mode (All posts, waits for completion)

{
  "profileUrls": [
    "https://www.instagram.com/mrbeast"
  ],
  "apiToken": "your-api-token",
  "waitForCompletion": true,
  "maxWaitTime": 10,
  "pollingInterval": 30,
  "maxConcurrency": 1
}

---

📤 Output Format

Each profile will produce a dataset item with the following structure:

Successful Response

{
  "profileUrl": "https://www.instagram.com/mrbeast",
  "status": "success",
  "profile": {
    "account": "mrbeast",
    "instagram_id": "12345678",
    "fbid": "17841400455970028",
    "profile_name": "MrBeast",
    "full_name": "Jimmy Donaldson",
    "biography": "Doing things that no one has done before...",
    "profile_image_link": "https://scontent.cdninstagram.com/...",
    "followers": 150000000,
    "following": 500,
    "posts_count": 1250,
    "is_verified": true,
    "is_private": false,
    "is_business_account": true,
    "is_professional_account": true,
    "avg_engagement": 5.2,
    "category_name": "Entrepreneur",
    "highlights_count": 8,
    "is_joined_recently": false,
    "has_channel": true,
    "business_address": "Los Angeles, CA",
    "email": "contact@example.com",
    "external_url": ["https://website.com"],
    "profile_url": "https://www.instagram.com/mrbeast/"
  },
  "posts": [
    {
      "id": 123,
      "instagram_id": "post_12345",
      "url": "https://www.instagram.com/p/ABC123/",
      "caption": "Post caption text with #hashtags",
      "likes": 5000000,
      "comments": 25000,
      "date_posted": "2025-09-10T15:30:00Z",
      "content_type": "image",
      "image_url": "https://...",
      "video_url": null,
      "thumbnail": "https://...",
      "hashtags": ["#challenge", "#giveaway"],
      "location": {"name": "Los Angeles"},
      "shortcode": "ABC123",
      "is_paid_partnership": false,
      "post_content": [
        {
          "index": 0,
          "type": "image",
          "url": "https://image.jpg",
          "instagram_id": "content_id_123"
        }
      ]
    }
  ],
  "related_accounts": [
    {
      "instagram_id": "87654321",
      "user_name": "related_user",
      "profile_name": "Related User",
      "is_private": false,
      "is_verified": true,
      "profile_pic_url": "https://..."
    }
  ],
  "highlights": ["highlight1", "highlight2"],
  "bio_hashtags": ["#entrepreneur", "#creator"],
  "post_hashtags": ["#challenge", "#giveaway"],
  "summary": {
    "totalPosts": 1250,
    "totalCost": "0.050",
    "listCost": "12.500",
    "processingTime": "15.2s"
  },
  "downloadInfo": {
    "id": "01K575ZB8HT9BY4QATJ7848BVK",
    "type": "instagram_profile",
    "status": "success"
  },
  "listInfo": {
    "list_id": "01K575ZB8HT9BY4QATJ784DJN",
    "list_cost": "12.500"
  }
}

Pending Response (when waitForCompletion is false)

{
  "profileUrl": "https://www.instagram.com/mrbeast",
  "status": "pending",
  "download_id": "01HXYZ123456789",
  "message": "Profile scraping initiated. All posts will be processed. Please be patient as it may take a few minutes.",
  "summary": {
    "processingTime": "2.1s"
  }
}

Failed Response

{
  "profileUrl": "https://www.instagram.com/invalid",
  "error": "Profile does not exist or has been deleted",
  "status": "failed"
}

---

🚀 How to Use

1. Get your API token from Transcript Downloader
2. Add Instagram profile URLs in the format:
   • https://www.instagram.com/username
   • https://instagram.com/username (also supported)
3. Choose your mode:
   • Instant mode: Set waitForCompletion: false for instant results with recent posts
   • Full mode: Set waitForCompletion: true to get all posts (takes longer)
4. Run the actor and access results in the dataset

---

❌ Error Handling

The actor gracefully handles common API errors:

Status Code	Meaning	Action
400	Invalid Instagram URL	Check URL format
401	Insufficient credits or invalid token	Check credits and API token
403	Invalid API token	Regenerate API token
404	Profile does not exist or has been deleted	Verify profile exists
429	Rate limit exceeded	Actor handles with delays
500	Third party service error or processing error	Retry automatically
503	Service temporarily unavailable	Retry later

Failed profiles are captured in the dataset with error information.

---

⚠️ Rate Limiting & Performance

API Rate Limits

Scope	Limit	Window
Per User (API Token)	90 requests	1 minute
Per IP (unauthenticated)	90 requests	1 minute

When rate limits are exceeded, the API returns 429 Too Many Requests. The actor handles this automatically with built-in retry logic.

Processing Times

Operation	Typical Duration	Maximum Wait	Recommended Poll Interval
Profile Fetch	5-30 seconds	60 minutes	30 seconds
Content Metadata	5-60 seconds	60 minutes	30 seconds

• Instant mode (recent posts): ~5-15 seconds per profile
• Full mode (all posts): ~2-10 minutes per profile depending on post count

Retry Behavior

The actor automatically retries on transient errors (429, 500, 503) with exponential backoff (base delay 1s, max delay 60s, up to 5 attempts). It does not retry on client errors (401, 403, 404) since those require user action.

Response Headers

The API returns rate limit headers you can monitor in logs:

Header	Description
X-RateLimit-Limit	Max requests allowed in window
X-RateLimit-Remaining	Requests remaining in current window
Retry-After	Seconds to wait before retrying (on 429)

---

🔔 Webhook Support

Instead of polling, you can receive results automatically via webhook. Pass a publicly reachable URL in the includeWebhook field, and the API will POST the results directly to your server when profile processing completes.

How It Works

1. The webhook URL is sent with the initial API request only (not during polling)
2. One webhook is fired per profile when processing completes or fails
3. The webhook payload is the exact same JSON the API endpoint would return
4. Failed deliveries are retried up to 3 times (at 10s, 30s, and 60s intervals)

Webhook Headers

Each delivery includes these custom headers to identify the event:

Header	Description
X-Webhook-Endpoint	instagram/profile
X-Webhook-Original-Status	HTTP status code of the result (e.g. 200)
X-Webhook-Download-Id	The download record ID

Example with Webhook

{
  "profileUrls": ["https://www.instagram.com/mrbeast"],
  "apiToken": "your-token",
  "waitForCompletion": false,
  "includeWebhook": "https://your-domain.com/webhook"
}

Tip: When using webhooks, you can set waitForCompletion: false so the actor returns immediately after initiating the request. Your webhook URL will receive the full results when ready.

Important Notes

• The URL must be publicly reachable (no localhost or private IPs)
• Your endpoint must accept POST requests and return a 2xx status
• Test your webhook URL first using the API test endpoint
• The webhook is registered once at request time — it cannot be added after a job has started

---

💡 Best Practices

For Quick Analysis (Instant Results)

{
  "profileUrls": ["https://www.instagram.com/profile1", "https://www.instagram.com/profile2"],
  "apiToken": "your-token",
  "waitForCompletion": false,
  "maxConcurrency": 2
}

Use case: Quick profile checks, follower counts, recent engagement

For Deep Analysis (All Posts)

{
  "profileUrls": ["https://www.instagram.com/profile"],
  "apiToken": "your-token", 
  "waitForCompletion": true,
  "maxWaitTime": 15,
  "pollingInterval": 30,
  "maxConcurrency": 1
}

Use case: Complete content analysis, historical data, all posts metadata

With Webhook (No Polling)

{
  "profileUrls": ["https://www.instagram.com/profile"],
  "apiToken": "your-token",
  "waitForCompletion": false,
  "includeWebhook": "https://your-domain.com/webhook"
}

Use case: Fire-and-forget — results delivered to your server automatically

General Tips

• ✅ Use waitForCompletion: false for quick checks
• ✅ Use waitForCompletion: true for comprehensive analysis
• ✅ Use includeWebhook with waitForCompletion: false to avoid long actor runtimes
• ✅ Keep maxConcurrency low (1-2) to avoid rate limits
• ✅ Keep pollingInterval at 30 seconds as recommended by the API
• ✅ Monitor costs using the summary.totalCost field
• ✅ Validate profile URLs before running large batches
• ✅ Private accounts may return limited data or errors
• ✅ Business accounts typically provide richer metadata

🆕 New Profile Fields Available

The API now returns additional profile information:

• fbid: Facebook ID associated with the Instagram account
• is_professional_account: Whether the account is a professional account
• highlights_count: Number of story highlights on the profile
• is_joined_recently: Indicates if the user joined Instagram recently
• has_channel: Whether the account has a broadcast channel
• business_address: Physical address for business accounts
• list_cost: Cost for fetching the complete posts list (returned upfront)

These fields are available in the profile object of the response.

---

🔗 Supported URL Formats

The actor supports Instagram profile URLs in the following formats:

✅ https://www.instagram.com/username
✅ https://instagram.com/username
✅ http://www.instagram.com/username
✅ http://instagram.com/username

❌ Not supported:

❌ https://www.instagram.com/p/POST_ID/ (individual posts)
❌ https://www.instagram.com/username/reels/
❌ https://www.instagram.com/username/tagged/

---

📈 Monitoring & Analytics

Track your usage with the built-in summary data:

{
  "summary": {
    "totalPosts": 1250,
    "totalCost": "0.050",
    "listCost": "12.500",
    "processingTime": "45.2s"
  }
}

Use this data to:

• Monitor API costs per profile
• Track processing performance
• Plan batch processing strategies
• Optimize concurrency settings

---

💳 Pricing & Billing

The Transcript Downloader API used by this actor requires a valid API token. API usage is billed separately:

• Instant fetch (recent posts): Lower cost per profile
• Full scraping (all posts): Higher cost based on post count
• List processing: Additional cost shown in listCost field

📊 Pricing varies by profile size and data volume. Visit our site to checkout pricing.
View full details and subscription plans on our pricing page

---

🎯 Use Cases

• Influencer Marketing: Analyze engagement rates, follower counts, content strategy
• Competitor Research: Track competitors' posting frequency, hashtags, content types
• Brand Monitoring: Monitor brand mentions, user-generated content, partnerships
• Content Analysis: Analyze successful post patterns, optimal posting times
• Market Research: Identify trends, popular hashtags, related accounts
• Social Listening: Track conversations, comments, engagement metrics

---

🙋 Support

Need help? Visit Transcript Downloader Support. We respond within 24 business hours.

For technical issues with this actor, check the run logs for detailed error messages.

---

📄 License

This actor is provided under the ISC License.