# Instagram Profile & Hashtag Scraper (`second_coming/instagram-profile-scraper`) Actor Extract posts, comments, follower counts, engagement rates, and hashtag analytics from Instagram profiles using public page HTML parsing (meta tags, JSON-LD, embedded GraphQL data). Designed for influencer marketing research and campaign analysis. - **URL**: https://apify.com/second\_coming/instagram-profile-scraper.md - **Developed by:** [Richard P](https://apify.com/second_coming) (community) - **Categories:** Social media, Marketing - **Stats:** 1 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Instagram Profile & Hashtag Scraper An Apify Actor that extracts public profile metadata, posts, follower counts, engagement rates, and hashtag analytics from Instagram. ### Features - **Profile scraping** — username, full name, biography, follower/following/post counts, verified status, profile picture, external URL - **Post extraction** — shortcode, caption, likes, comments, media URL, timestamp, hashtags, mentions - **Hashtag exploration** — recent posts for any hashtag with owner info - **Engagement rate** — calculated from likes + comments per follower - **Multi-strategy extraction** — tries meta tags, JSON-LD, embedded GraphQL data, and regex fallbacks - **Rate limited** — 2-second delay between requests to avoid blocking - **Graceful abort** — clean shutdown when the Actor is stopped ### Input | Field | Type | Description | Default | |-------------------|-----------|----------------------------------------------------|----------| | `profileUrls` | string[] | Instagram profile URLs to scrape | `[]` | | `hashtags` | string[] | Hashtags to explore (with or without #) | `[]` | | `maxPosts` | integer | Max posts to extract per profile / hashtag (1-200) | `30` | | `includeComments` | boolean | Include comment counts when available | `true` | | `includeLikes` | boolean | Include like counts when available | `true` | At least one of `profileUrls` or `hashtags` is required. #### Example input ```json { "profileUrls": [ "https://www.instagram.com/nasa/", "https://www.instagram.com/natgeo/" ], "hashtags": ["travel", "photography"], "maxPosts": 20, "includeComments": true, "includeLikes": true } ```` ### Output Each dataset item is either a profile result or a hashtag result. #### Profile output fields | Field | Type | Description | |-------------------|-----------|----------------------------------------| | `username` | string | Instagram handle | | `fullName` | string? | Display name | | `biography` | string? | Profile bio text | | `followerCount` | integer? | Number of followers | | `followingCount` | integer? | Number of accounts followed | | `postCount` | integer? | Total number of posts | | `isPrivate` | boolean? | Whether the account is private | | `isVerified` | boolean? | Whether the account is verified | | `profilePicUrl` | string? | Profile picture URL | | `externalUrl` | string? | External link in bio | | `engagementRate` | number? | (Likes + Comments) / Followers × 100 | | `posts` | array | List of extracted posts (see below) | | `scrapedAt` | string | ISO 8601 timestamp of scrape | | `error` | string? | Error message if scraping failed | #### Hashtag output fields | Field | Type | Description | |-------------------|-----------|----------------------------------------| | `hashtag` | string | The hashtag (without #) | | `postCount` | integer? | Estimated total post count | | `posts` | array | List of extracted posts | | `scrapedAt` | string | ISO 8601 timestamp of scrape | #### Post object fields | Field | Type | Description | |-------------|-----------|-------------------------------------| | `shortcode` | string | Unique post identifier | | `caption` | string? | Post caption text | | `likes` | integer? | Like count | | `comments` | integer? | Comment count | | `mediaUrl` | string? | URL of the image/video | | `mediaType` | string | GraphImage / GraphVideo / GraphSidepage | | `timestamp` | string? | ISO 8601 timestamp of the post | | `isVideo` | boolean | Whether the post is a video | | `videoUrl` | string? | URL of the video (if applicable) | | `hashtags` | string\[] | Hashtags extracted from caption | | `mentions` | string\[] | @mentions extracted from caption | ### Example output ```json { "username": "nasa", "fullName": "NASA", "biography": "Explore the universe and discover our home planet.", "followerCount": 75000000, "followingCount": 78, "postCount": 4200, "isPrivate": false, "isVerified": true, "profilePicUrl": "https://instagram.com/...", "externalUrl": "https://www.nasa.gov/", "engagementRate": 2.4513, "posts": [ { "shortcode": "CxYzAbCdEfG", "caption": "Exploring new worlds #space #science", "likes": 1250000, "comments": 8452, "mediaUrl": "https://instagram.com/...", "mediaType": "GraphImage", "timestamp": "2026-06-15T14:30:00+00:00", "isVideo": false, "videoUrl": null, "hashtags": ["space", "science"], "mentions": [] } ], "scrapedAt": "2026-07-04T12:00:00+00:00" } ``` ### Limitations Instagram aggressively blocks automated access. This actor works **without login** by parsing the public HTML of profile and hashtag pages, but: | Limitation | Details | |------------|---------| | **No login** | Some data (stories, full follower lists, private profiles) requires authentication | | **Rate limiting** | Instagram may return 429 errors after several requests; the actor implements exponential back-off but heavy use will be blocked | | **JavaScript rendering** | Instagram is fully client-side rendered. The actor extracts what's available from embedded JSON and meta tags — the richest data comes from `__NEXT_DATA__` / `__INITIAL_STATE__` blobs, which aren't always present | | **Post count** | The full set of posts requires JS scrolling; this actor extracts what's available in the initial page load (usually first 12 posts) | | **Comment/like accuracy** | Live like/comment counts shown on the page may differ from what's embedded in the HTML | | **Hashtag total count** | The total post count for hashtags is estimated from meta tag descriptions | For production use with authentication and JS rendering, consider: - Using Playwright/Selenium (requires the Apify Playwright Actor base image) - Using Instagram's Basic Display API (requires Facebook app registration) - Using a commercial Instagram data provider ### Local development ```bash ## Create virtualenv python3 -m venv .venv source .venv/bin/activate ## Install dependencies pip install -r requirements.txt ## Set Python path for apify CLI export APIFY_ACTOR_PYTHON_VENV_PATH=.venv ## Run locally apify run --purge ``` ### Changelog #### 0.1.0 — Initial release - Profile scraping with multi-strategy extraction - Hashtag exploration - Engagement rate calculation - Graceful abort handling - Rate limiting with exponential back-off # Actor input Schema ## `profileUrls` (type: `array`): List of Instagram profile URLs to scrape (e.g. https://www.instagram.com/nasa/) ## `hashtags` (type: `array`): List of hashtags to scrape (without or without the # prefix) ## `maxPosts` (type: `integer`): Maximum number of posts to extract per profile or hashtag page ## `includeComments` (type: `boolean`): Whether to include comment counts in the output (when available) ## `includeLikes` (type: `boolean`): Whether to include like counts in the output (when available) ## Actor input object example ```json { "profileUrls": [ "https://www.instagram.com/nasa/" ], "hashtags": [ "travel", "photography" ], "maxPosts": 30, "includeComments": true, "includeLikes": true } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "profileUrls": [ "https://www.instagram.com/nasa/" ], "hashtags": [ "travel", "photography" ] }; // Run the Actor and wait for it to finish const run = await client.actor("second_coming/instagram-profile-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "profileUrls": ["https://www.instagram.com/nasa/"], "hashtags": [ "travel", "photography", ], } # Run the Actor and wait for it to finish run = client.actor("second_coming/instagram-profile-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "profileUrls": [ "https://www.instagram.com/nasa/" ], "hashtags": [ "travel", "photography" ] }' | apify call second_coming/instagram-profile-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=second_coming/instagram-profile-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Instagram Profile & Hashtag Scraper", "description": "Extract posts, comments, follower counts, engagement rates, and hashtag analytics from Instagram profiles using public page HTML parsing (meta tags, JSON-LD, embedded GraphQL data). Designed for influencer marketing research and campaign analysis.", "version": "0.1", "x-build-id": "CdOEfiyCfX7XK6TM2" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/second_coming~instagram-profile-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-second_coming-instagram-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/second_coming~instagram-profile-scraper/runs": { "post": { "operationId": "runs-sync-second_coming-instagram-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/second_coming~instagram-profile-scraper/run-sync": { "post": { "operationId": "run-sync-second_coming-instagram-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "profileUrls": { "title": "Instagram Profile URLs", "type": "array", "description": "List of Instagram profile URLs to scrape (e.g. https://www.instagram.com/nasa/)", "items": { "type": "string" } }, "hashtags": { "title": "Hashtags to Explore", "type": "array", "description": "List of hashtags to scrape (without or without the # prefix)", "items": { "type": "string" } }, "maxPosts": { "title": "Max Posts Per Profile/Hashtag", "minimum": 1, "maximum": 200, "type": "integer", "description": "Maximum number of posts to extract per profile or hashtag page", "default": 30 }, "includeComments": { "title": "Include Comments", "type": "boolean", "description": "Whether to include comment counts in the output (when available)", "default": true }, "includeLikes": { "title": "Include Likes", "type": "boolean", "description": "Whether to include like counts in the output (when available)", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```