Instagram Story Details Scraper With Highlights

Instagram Story Scraper — Extract Live Stories and Highlight Reels

Instagram Story Details Scraper With Highlights pulls structured story data from any Instagram profile — the live story tray that vanishes after 24 hours, plus every permanent Highlight reel a profile has pinned, including stories that already expired off the live feed. Every response is structured JSON, ready to pass directly to an LLM, index into a vector store, or feed a monitoring pipeline. Point it at a list of usernames and get every story item, live and archived, back as one dataset.

🧭 What is Instagram Story Details Scraper With Highlights?

Instagram Story Details Scraper With Highlights is an Apify Actor that takes a list of Instagram profile URLs or usernames and returns two things: individual story items (the media, captions, stickers and metadata behind each story) and the permanent Highlight trays a profile has curated — the collections that already fell off the 24-hour story feed but are still pinned to the profile. That second part is the Actor's real differentiator: most story scrapers only see what is currently live; this one also enumerates and opens every Highlight tray, so a story a profile posted months ago and pinned to "Travel" or "FAQ" is still retrievable.

No Instagram account is required to collect Highlights. Reading a profile's live (<=24h) story tray does require your own logged-in sessionId cookie — without it, the Actor logs a notice and simply skips the live tray for that profile while continuing to collect Highlights.

• 🎞️ Live story capture (<=24h tray) when you supply a session cookie
• 📚 Highlight tray enumeration — every permanent Highlight on a profile, or a filtered subset
• 🏷️ Title filtering to target specific Highlight trays (e.g. "Travel", "FAQ")
• 🔢 A per-profile cap on how many Highlight trays get opened
• 🌍 Apify Proxy on every request, with automatic escalation to a residential proxy tier if a profile starts blocking
• 📦 Batch as many profiles as you like into a single run

📦 What data can you get with Instagram Story Details Scraper With Highlights?

Instagram Story Details Scraper With Highlights returns two result types in every run: individual story items (from the live tray or from a Highlight) and Highlight tray records describing each tray itself.

Result Type	Extracted Fields	Primary Use Case
Story item	pk, id, code, media_type, taken_at, expiring_at, device_timestamp, user, caption, image_versions2, video_versions, video_duration, music_metadata, product_suggestions, coauthor_producers, source, highlight_id, highlight_title, plus every other field Instagram returns for that item (full list in Output)	Archiving story content, building a per-profile story timeline, spotting co-authored or paid-partnership posts
Highlight tray	username, user_id, highlight_id, title, cover_url, item_count	Mapping a profile's permanent Highlight structure before deciding which trays to pull in full

Highlight reels

A profile's live story tray only ever shows the last 24 hours. Highlights are the curated, permanent collections a user pins under their profile picture — travel diaries, FAQs, product drops — and they routinely contain stories that no longer exist anywhere else on the platform. Instagram Story Details Scraper With Highlights enumerates every Highlight tray on a profile (or a subset, via highlightTitleFilter / maxHighlights), then opens each tray and pulls its story items using the same collection logic as the live tray. Every row from a Highlight carries source: "highlight" plus the tray's highlight_id and highlight_title, so you can group story items back to the tray they came from without a second lookup:

{ "highlight_id": "highlight:17912345678901234", "title": "Travel", "item_count": 12 }

Live story tray

When includeLiveStories is on and a sessionId is supplied, the Actor also collects the profile's current live (<=24h) story tray — the same items that would appear in the Instagram app's story bar. These rows carry source: "live" with highlight_id and highlight_title set to null. This is the perishable half of the picture: useful for catching a story before it disappears, and for comparing what's currently live against what a profile has chosen to keep permanently in a Highlight.

🛠️ Why not build this yourself?

Instagram does not publish a general-purpose API for reading an arbitrary public profile's stories or Highlight trays — the official Instagram Graph API is scoped to a business account's own connected content and requires app review, not third-party profile lookups. Getting this data yourself means reverse-engineering Instagram's private web endpoints: a GraphQL relay call for Highlight enumeration that expects specific request-body tokens, a reel-media endpoint for story items, and constant handling of checkpoint pages, login walls and rate-limit blocks that Instagram serves to anything it flags as automated. Instagram Story Details Scraper With Highlights already does this work — proxy rotation with automatic escalation on a block, session warm-up before the first request, CDN-URL expiry decoding, and music-metadata backfill when Instagram omits it from the main object — so you get rows in your dataset instead of a debugging session against an undocumented API.

⚡ How to scrape Instagram stories and Highlights

1. Open Instagram Story Details Scraper With Highlights on its Apify Store listing and click Try for free (or Run, if you already have it saved).
2. Paste one or more profile links or usernames into urls — the only required field.
3. Set the real query controls: add sessionId if you want the live 24h tray included, leave includeLiveStories on or off, set maxHighlights to cap how many trays get opened, and use highlightTitleFilter to target specific Highlight names.
4. Start the run.
5. Open the Output tab to watch rows arrive live, or export the finished dataset as JSON, CSV, Excel or XML once the run completes.

How to run multiple profiles in one job

Add every profile you want to urls — it's a plain list, so one run can cover as many profiles as you paste in. The Actor works through them one at a time, with a fixed 5-second pause between profiles, collecting live stories (if applicable) and then every matching Highlight tray for each before moving to the next. There is no separate batch-upload format; the array is the batch.

⬇️ Input

Parameter	Required	Type	Description	Example Value
urls	Yes	array	One list, many profiles — paste full profile links or plain usernames (with or without @). Add more than one to batch a run across live stories and Highlights alike.	["https://www.instagram.com/kyliejenner/", "natgeo"]
sessionId	No	string, default ""	Your logged-in sessionid cookie from instagram.com. Highlight reels can be read without it, but the live (<=24h) story tray needs a session — leave blank to collect Highlights only. Treat it like a password; don't share or commit it.	""
sessionid	No	string, default ""	Legacy alias for sessionId, accepted for backward compatibility with older input JSON. Prefer sessionId; leave blank if you already filled that field.	""
includeLiveStories	No	boolean, default true	When on and a session is provided, the Actor also collects the profile's current live story tray. Turn off to collect Highlights only.	true
maxHighlights	No	integer, default 0, minimum 0	Caps how many of a profile's Highlight trays are opened, in tray order. 0 collects every tray found.	3
highlightTitleFilter	No	string, default ""	Comma-separated text to match against Highlight tray titles (case-insensitive, partial match). Travel, FAQ only opens trays whose title contains "travel" or "faq". Leave blank to include every tray.	"Travel, FAQ"
proxyConfiguration	No	object	Apify Proxy configuration for this run. Honored by every request the Actor makes — leave defaults on for the most stable runs, or turn off for direct connections.	{ "useApifyProxy": true }

Example input:

{
  "urls": ["https://www.instagram.com/kyliejenner/", "natgeo"],
  "sessionId": "",
  "includeLiveStories": true,
  "maxHighlights": 3,
  "highlightTitleFilter": "Travel, FAQ",
  "proxyConfiguration": { "useApifyProxy": true }
}

⚠️ Common pitfall: leaving sessionId blank does not error the run — it silently skips the live 24h tray for every profile (logged as a notice) and returns Highlights only. If your dataset has no source: "live" rows, check that a valid session cookie was actually supplied.

⬆️ Output

Every run writes to two datasets. The default dataset holds one row per story item — every push here is a billed row_result event, whether the item came from the live tray or a Highlight. A second, always-free named dataset called highlight-trays holds one row per Highlight tray discovered, independent of how many (or how few) story items that tray contained.

Each story row carries every field Instagram's private API returned for that item. The Actor keeps a known set of fields in a fixed order at the front of the row, then appends any additional field Instagram includes (alphabetically) — nothing Instagram returns is ever dropped:

pk, id, is_visual_reply_commenter_notice_enabled, like_and_view_counts_disabled,
is_post_live_clips_media, is_reshare_of_text_post_app_media_in_ig, is_reel_media, fbid,
device_timestamp, caption_is_edited, strong_id__, is_quicksnap_recap, deleted_reason,
expiring_at, mezql_token, should_request_ads, is_terminal_video_segment,
integrity_review_decision, client_cache_key, has_privately_liked, filter_type, taken_at,
commenting_disabled_for_viewer, hide_view_all_comment_entrypoint,
is_comments_gif_composer_enabled, has_liked, video_sticker_locales, is_dash_eligible,
video_dash_manifest, video_codec, number_of_qualities, video_versions, video_duration,
has_audio, can_viewer_save, shop_routing_user_id, is_organic_product_tagging_eligible,
product_suggestions, can_see_insights_as_brand, media_type, code, caption, fundraiser_tag,
sharing_friction_info, timeline_pinned_user_ids, has_translation,
original_media_has_visual_reply_media, coauthor_producers,
coauthor_producer_can_see_organic_insights, invited_coauthor_producers,
is_in_profile_grid, profile_grid_control_enabled, user, image_versions2, original_width,
original_height, product_type, is_paid_partnership, music_metadata,
organic_tracking_token, ig_media_sharing_disabled, crosspost, crosspost_metadata,
boost_unavailable_identifier, boost_unavailable_reason, boost_unavailable_reason_v2,
subscribe_cta_visible, is_cutout_sticker_allowed, cutout_sticker_info, can_hype,
gen_ai_detection_method, fb_aggregated_like_count, fb_aggregated_comment_count,
has_high_risk_gen_ai_inform_treatment, open_carousel_show_follow_button,
is_tagged_media_shared_to_viewer_profile_grid,
should_show_author_pog_for_tagged_media_shared_to_profile_grid,
is_open_to_public_submission, media_attributions_data, archive_story_deletion_ts,
can_send_prompt, is_first_take, is_from_discovery_surface, is_rollcall_v2,
supports_reel_reactions, is_photo_mash_story, can_play_spotify_audio, is_superlative,
show_one_tap_fb_share_tooltip, story_feed_media, story_music_stickers, can_reply,
is_viewer_mentioned, can_reshare

Then three fields the Actor always appends itself:

Field	Description
source	"live" or "highlight" — where this story item came from.
highlight_id	The Highlight tray's ID (already prefixed highlight: by Instagram), or null for a live-tray item.
highlight_title	The Highlight tray's title, or null for a live-tray item.

Two enrichments are applied to whatever Instagram returns, without inventing values: any nested object carrying a CDN url (most commonly inside video_versions) gets a computed url_expiration_timestamp_us added when the URL's oe= parameter can be decoded; music_metadata is backfilled from story_music_stickers[].music_asset_info only when Instagram's response omits the top-level music_metadata value directly. Where Instagram genuinely provides no value, the field is null — nothing is guessed.

The highlight-trays dataset carries exactly these fields per tray:

Field	Description
username	The profile the tray belongs to.
user_id	The profile's numeric Instagram user ID.
highlight_id	The tray's ID.
title	The tray's title as set by the profile owner.
cover_url	The tray's cover image URL, if Instagram provided one.
item_count	The tray's reported story count.

Scraped results

[
  {
    "pk": "3401234567890123456",
    "id": "3401234567890123456_9876543210",
    "code": "Cxxxxxxxxxxx",
    "media_type": 2,
    "taken_at": 1753800000,
    "expiring_at": 1753886400,
    "device_timestamp": 1753799990123,
    "user": {
      "pk": "9876543210",
      "username": "kyliejenner",
      "full_name": "Kylie Jenner",
      "is_private": false,
      "is_verified": true
    },
    "caption": null,
    "video_versions": [
      { "url": "https://scontent.cdninstagram.com/o1/v/t16/...oe=66B2F0A0", "url_expiration_timestamp_us": 1723017376000000 }
    ],
    "video_duration": 8.233,
    "music_metadata": null,
    "source": "live",
    "highlight_id": null,
    "highlight_title": null
  },
  {
    "pk": "3401234567890654321",
    "id": "3401234567890654321_9876543210",
    "code": "Cyyyyyyyyyyy",
    "media_type": 1,
    "taken_at": 1701500000,
    "expiring_at": null,
    "device_timestamp": 1701499990000,
    "user": {
      "pk": "9876543210",
      "username": "kyliejenner",
      "full_name": "Kylie Jenner",
      "is_private": false,
      "is_verified": true
    },
    "caption": "Bali 2023",
    "image_versions2": {
      "candidates": [{ "url": "https://scontent.cdninstagram.com/o1/v/t51/...jpg", "width": 1080, "height": 1920 }]
    },
    "music_metadata": null,
    "source": "highlight",
    "highlight_id": "highlight:17912345678901234",
    "highlight_title": "Travel"
  },
  {
    "pk": "3401234567890999999",
    "id": "3401234567890999999_9876543210",
    "code": "Czzzzzzzzzzz",
    "media_type": 2,
    "taken_at": 1690000000,
    "expiring_at": null,
    "device_timestamp": 1690000000000,
    "user": {
      "pk": "9876543210",
      "username": "kyliejenner",
      "full_name": "Kylie Jenner",
      "is_private": false,
      "is_verified": true
    },
    "caption": "FAQ #3",
    "video_versions": [
      { "url": "https://scontent.cdninstagram.com/o1/v/t16/...oe=650A1B2C", "url_expiration_timestamp_us": 1698012345000000 }
    ],
    "video_duration": 14.6,
    "music_metadata": { "music_asset_info": { "title": "Sample Track", "display_artist": "Sample Artist" } },
    "source": "highlight",
    "highlight_id": "highlight:17909876543210987",
    "highlight_title": "FAQ"
  }
]

(Trimmed here to the fields most useful for reading — every other field listed above is present in the real row.) The paired highlight-trays dataset for this run looks like:

[
  { "username": "kyliejenner", "user_id": "9876543210", "highlight_id": "highlight:17912345678901234", "title": "Travel", "cover_url": "https://scontent.cdninstagram.com/o1/v/t51/...jpg", "item_count": 12 },
  { "username": "kyliejenner", "user_id": "9876543210", "highlight_id": "highlight:17909876543210987", "title": "FAQ", "cover_url": "https://scontent.cdninstagram.com/o1/v/t51/...jpg", "item_count": 5 }
]

💡 How can I use the data extracted with Instagram Story Details Scraper With Highlights?

• 📊 Social media managers and marketers: track a brand's or influencer's story activity and Highlight structure — which trays exist, what's in them — without opening each one by hand in the app.
• 🤖 AI engineers and LLM developers: feed the JSON to an agent so it can answer questions like "what does @brand's FAQ Highlight say" using highlight_title and caption as grounding context.
• 🔬 Market and competitive researchers: compare how competing accounts organize and update their Highlight trays — new trays, renamed trays, growing item_count — over repeated runs.
• 🗂️ Brand safety, legal and compliance teams: archive story content — captions, coauthors, product tags — before it expires off the live feed or before a Highlight is edited or deleted.

🔁 How do you monitor Highlight reels over time?

Highlight trays are permanent, but not static — a profile owner can rename a tray, add new stories to it, or delete it entirely, and none of that is visible from a single run. Monitoring means running the Actor against the same profile list on a schedule and comparing the highlight-trays dataset from one run to the next. The fields worth diffing are highlight_id (did the set of trays change — new IDs appearing, old ones missing), title (was a tray renamed), and item_count (did new stories get added to an existing tray). For the story items themselves, taken_at tells you whether a story inside a tray you already indexed is genuinely new since your last run, so you don't re-process items you've already archived.

A practical loop: schedule a run across your profile list using an Apify Schedule, export or pull each run's highlight-trays dataset via the API, and diff it against the previous run's export keyed on highlight_id. Alert on any item_count increase for a tray you're tracking, or on a highlight_id that disappeared — that's a deleted Highlight, which is the kind of change a single one-off scrape would never catch.

🔌 Integrate Instagram Story Details Scraper With Highlights and automate your workflow

Instagram Story Details Scraper With Highlights works with any language or tool that can send an HTTP request — it runs on Apify, so it's reachable through the Apify API and every official Apify client.

REST API with Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("your-username/instagram-story-details-scraper-with-highlights").call(
    run_input={
        "urls": ["https://www.instagram.com/kyliejenner/"],
        "maxHighlights": 3,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["source"], item.get("highlight_title"), item["pk"])

Scheduled monitoring and delivery

The Actor has no built-in webhook or push mechanism of its own, but the Apify platform it runs on does: set up an Apify Schedule to run it on a recurring interval, and attach an Apify webhook to fire on run completion, or poll the dataset via apify_client after each scheduled run.

⚖️ Is it legal to scrape Instagram stories and Highlights?

Scraping publicly accessible data is generally lawful — Instagram Story Details Scraper With Highlights returns only what a visitor to a public profile, or the account whose session you supplied, can already see in the app. In the U.S., hiQ Labs, Inc. v. LinkedIn Corp., 938 F.3d 985 (9th Cir. 2019), held that accessing publicly available web data without authentication generally does not violate the Computer Fraud and Abuse Act. Story content and profile data returned here can constitute personal data about the account owner and about anyone tagged, mentioned, or co-authoring a story, so GDPR and CCPA considerations apply if you store or process it — particularly for EU or California residents. Scraping for AI training and scraping for operational monitoring carry different risk profiles under these regimes. Live-story collection additionally uses your own authenticated session, which is separately governed by Instagram's Terms of Use. Consult your legal team for commercial use cases involving bulk data storage.

❓ Frequently asked questions

Do I need to log in to Instagram to use this Actor?

Not to collect Highlights — those are read without any session. Live (<=24h) story collection is the one part that needs your own logged-in sessionid cookie, passed as sessionId; without it, the Actor skips the live tray per profile and still returns Highlights.

Can I collect only Highlights and skip live stories?

Yes — set includeLiveStories to false, or simply leave sessionId blank. Either way, only Highlight trays are collected.

How do I filter which Highlight trays are collected?

Use highlightTitleFilter with a comma-separated list of substrings (e.g. "Travel, FAQ") — matching is case-insensitive and partial, so "faq" matches a tray titled "FAQ #3". Combine it with maxHighlights to also cap the number of trays opened per profile, in tray order.

How does the Actor handle Instagram's anti-bot measures?

It warms up a session before making any data request (a homepage visit followed by a short pause), primes a CSRF token, and randomizes a short delay before every subsequent request. If a response comes back blocked — a 401/403/429 status, or a checkpoint, challenge, or login-wall page — it automatically retries on a residential Apify Proxy tier and stays on that tier for the rest of the run, unless you explicitly disabled Apify Proxy for the run.

Does the Actor extract Highlight reels, not just live 24-hour stories?

Yes. Every story row from a Highlight carries source: "highlight" along with highlight_id and highlight_title, and each discovered tray also gets its own row in the separate highlight-trays dataset. Rows from the live tray instead carry source: "live" with both Highlight fields set to null.

How many story items does the Actor return per profile?

There is no fixed cap on story items themselves. maxHighlights limits how many Highlight trays are opened per profile (0 = all), but every story item inside an opened tray, and every item in the live tray, is collected and pushed.

How do I monitor a profile's Highlights over time?

Schedule repeated runs with an Apify Schedule, then diff the highlight-trays dataset between runs on highlight_id — a new ID is a new tray, a missing one was deleted, and a changed item_count means stories were added. See How do you monitor Highlight reels over time? for the full workflow.

Does the Actor work with Claude, ChatGPT, and AI agent frameworks?

It isn't published as an MCP server, but it's callable as a standard HTTP endpoint through the Apify API from any agent framework — an agent can trigger a run, wait for it to finish, and pull the resulting JSON straight into its context before answering.

Can I use it without managing proxies or Instagram credentials myself?

Mostly — Apify Proxy is handled for you (including the automatic residential fallback on a block), so you don't need to source or rotate proxies yourself. You do still need to supply your own sessionId cookie if you want live 24h stories included; Highlights alone require no credentials at all.

💬 Your feedback

Found a bug, or a field Instagram added that isn't showing up in your rows? Open an issue on this Actor's Issues tab in Apify Console — Scrapier actively maintains this listing and reads every report.