Instagram URL Scraper

An Apify Actor that classifies each Instagram URL and routes it to a dedicated extractor. It captures Instagram JSON responses and embedded page data, then emits normalized records for public media, profiles, comments, stories, highlights, hashtags, locations, audio pages, guides and explore feeds.

Supported URLs

URL	Default output	Compatible resultsType
/p/<shortcode>/	Post/image/video/carousel detail	details, comments
/reel/<shortcode>/, /reels/<shortcode>/	Reel detail	details, comments
/<user>/reel/<shortcode>/	Reel detail	details, comments
/tv/<shortcode>/	Legacy IGTV/video detail	details, comments
/p/<shortcode>/c/<comment-id>/	One comment	comments
/share/<token>/, /share/p/<token>/, /share/reel/<token>/	Resolved target	details, comments
/<username>/ or /<numeric-profile-id>/	Profile detail	details, posts, reels, tagged, stories, highlights
/<username>/posts/	Profile posts	posts
/<username>/reels/	Profile reels	reels
/<username>/tagged/	Tagged media	tagged
/<username>/highlights/	Highlight media	highlights
/stories/<username>/[<story-id>/]	Active story media	stories
/stories/highlights/<highlight-id>/	Highlight media	highlights
/s/<encoded-highlight>/?story_media_id=...	Shared highlight/story	highlights
/explore/tags/<tag>/	Hashtag details	details, posts, reels
/explore/locations/<id>/<slug>/	Location details	details, posts, reels
/reels/audio/<audio-id>/	Audio details	details, reels
/<username>/guide/<slug>/<id>/	Legacy guide content	posts
/explore/ or /explore/search/?q=...	Explore media	posts, reels

/embed/ variants and tracking query parameters are accepted. Non-content paths such as /accounts/, /direct/ and /settings/ are rejected instead of being mistaken for profiles.

Input

{
  "input": [
    { "url": "https://www.instagram.com/p/SHORTCODE/" },
    {
      "url": "https://www.instagram.com/natgeo/",
      "resultsType": "posts",
      "resultsLimit": 25
    },
    {
      "url": "https://www.instagram.com/p/SHORTCODE/",
      "resultsType": "comments",
      "resultsLimit": 100
    }
  ],
  "resultsType": "auto",
  "resultsLimit": 50,
  "onlyPostsNewerThan": "30 days",
  "includeRaw": false,
  "maxConcurrency": 3,
  "maxRequestRetries": 2,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

resultsType and resultsLimit can be set globally or overridden on an individual input object. An incompatible combination, such as comments for a profile URL, produces an INVALID_RESULTS_TYPE record before crawling while other valid inputs continue.

Output model

Every row has these envelope fields:

{
  "schemaVersion": 1,
  "type": "post",
  "status": "succeeded",
  "sourceUrl": "https://www.instagram.com/reel/SHORTCODE/",
  "url": "https://www.instagram.com/reel/SHORTCODE/",
  "scrapedAt": "2026-09-05T00:00:00.000Z"
}

Possible successful type values are post, profile, comment, story, highlight, hashtag, location, audio, and guide.

Posts use contentType to distinguish image, video, carousel, reel, and igtv. A carousel is one dataset record whose ordered children are in media[].

For posts, text contains only the authored caption and textSource identifies where it was recovered (caption, open_graph, or null). Instagram generated accessibility description remains available only as media[].accessibilityText; it is never substituted for the authored caption.

Post records preserve the same base contract as the Threads scraper: type, status, sourceUrl, url, id, shortcode, text, urls, metrics, author, postedAt, language, and media. Instagram-specific fields are additive. The common metric keys are likes, replies, reposts, quotes, shares, and views; Instagram also exposes comments and plays. Unsupported or unavailable values are null.

urls contains unique outbound links recovered from caption text and structured CTA, ad, or link-sticker metadata. CDN media asset URLs remain in media[] and are excluded from urls.

Collection URLs emit one row per child item, up to resultsLimit. Every child includes parent, which identifies the profile, hashtag, location, audio, guide, highlight or explore URL that produced it.

Failures are explicit and do not silently become empty results:

{
  "schemaVersion": 1,
  "type": "story",
  "status": "failed",
  "sourceUrl": "https://www.instagram.com/stories/example/",
  "url": "https://www.instagram.com/accounts/login/",
  "errorCode": "LOGIN_WALL",
  "error": "Instagram requires authentication for this content",
  "retryable": false,
  "scrapedAt": "2026-09-05T00:00:00.000Z"
}

Other stable failure codes include INVALID_URL, NOT_FOUND, PRIVATE_CONTENT, RATE_LIMITED, EXTRACTION_FAILED, UNSUPPORTED_URL, and REQUEST_FAILED.

Extraction strategy

The Actor deliberately avoids relying on one hard-coded GraphQL document ID. For each page it:

1. Captures Instagram JSON network responses.
2. Parses JSON and data-sjs script payloads embedded in the page.
3. Scrolls collection surfaces conservatively to trigger pagination.
4. Scores media/profile/metadata candidates against identifiers from the input URL.
5. Uses Open Graph data as a reduced-quality fallback for direct public pages.
6. Normalizes private-API, GraphQL and page metadata shapes into the same public schema.

Access limitations

The Actor does not bypass privacy controls. Instagram may require a logged-in session for stories, highlights, deep comments, hashtag feeds, followers, private accounts, or some regions/IP ranges. In those cases the Actor emits LOGIN_WALL or PRIVATE_CONTENT. Residential proxies and low concurrency improve reliability but do not make private content public.

CDN media URLs are often signed and can expire. Download or consume them soon after the run.

Local development

npm install
npm test
apify run

Without the Apify CLI, place input in storage/key_value_stores/default/INPUT.json and run npm start.

Deploy

npx apify-cli login
npx apify-cli push