Instagram Highlights Scraper

Instagram Highlights Scraper — Titles, Covers and Full-Res Archive

Instagram Highlights Scraper extracts story highlight titles, IDs, cover images, and full-resolution cover archives from any public Instagram profile, highlight link, or post, reel, or IGTV shortcode. The highlights tray is fetched across every page via cursor pagination and streamed as structured JSON in real time, with optional per-profile summary rows and a cross-profile leaderboard for side-by-side comparison. Paste a profile URL or username below and start a run to see it in action.

What is Instagram Highlights Scraper?

Instagram Highlights Scraper is an Apify Actor that reads a public Instagram profile's highlights tray and returns each highlight's title, ID, cover image, and full-resolution cover as one JSON row per highlight, plus optional per-profile and cross-profile analytics rows. It works entirely from Instagram's public, logged-out profile and embed pages — no Instagram account, login, or session cookie of your own is required. It's built for social media marketers, competitive researchers, and developers who need highlight-level Instagram data on a schedule without maintaining their own scraper.

What Instagram highlight data is publicly available to scrape?

Anyone who opens a public Instagram profile in a browser, with no login required, can see its story highlights tray: the highlight titles, cover thumbnails, and, via each highlight's own reel, a full-resolution version of that cover. Other highlight-adjacent data either is not exposed publicly at all or requires being logged in as a follower.

Data Category	Publicly Available	Restricted behind [login]
Highlight titles	Yes	—
Highlight IDs	Yes	—
Cover thumbnail (~150×150)	Yes	—
Full-resolution cover photo	Yes (via the highlight's own reel)	—
Highlights on private accounts	No	Instagram login + an approved follow request
Expired stories not pinned to a highlight	No	Never exposed publicly — only what's currently in the tray
Story view counts / viewer lists	No	Restricted to the profile owner's own logged-in account
Direct messages referencing a highlight	No	Restricted behind Instagram login

Instagram Highlights Scraper only returns publicly visible data — what any visitor sees. Nothing behind a login wall.

What data can I extract with Instagram Highlights Scraper?

Instagram Highlights Scraper returns one JSON row per highlight — plus optional per-profile summary and cross-profile leaderboard rows — covering the highlight's identity, its cover media, and derived analytics.

Field Name	Description
input_url	The original startUrls entry you supplied (profile URL, username, highlight link, or shortcode)
username	Resolved Instagram username that owns the highlight
user_id	Resolved numeric Instagram user ID
type	Row type: instagram_highlights, instagram_highlights_profile_summary, or instagram_highlights_leaderboard_entry
id	The highlight's unique ID (e.g. highlight:18069978917514346)
title	The highlight's display title
cover_media	The highlights tray's default cover thumbnail URL (~150×150)
cover_media_full_res	Highest-resolution cover photo, resolved via the highlight's own reel and matched to the tray cover by CDN asset ID
cover_media_expires_at	ISO timestamp decoded from the cover URL's CDN oe= expiry parameter
cover_media_archive_url	Permanent copy of the full-res cover saved to this run's Key-Value Store (when archiveCoverImages is on)
success	Whether this row was scraped successfully
error	Error message when success is false, otherwise null
isSummary	true only on the per-profile summary row
profile_highlight_count	Number of highlights successfully scraped for this profile (summary row only)
profile_titles	All of this profile's highlight titles, joined with `"
profile_first_highlight_id	ID of the first highlight scraped for this profile (summary row only)
profile_last_highlight_id	ID of the last highlight scraped for this profile (summary row only)
profile_success_rate	Share of this profile's rows that scraped successfully, 0–1 (summary row only)
isLeaderboard	true only on the cross-profile leaderboard row
leaderboard_rank	This profile's rank by highlight count across the whole run (leaderboard row only)
timestamp	UTC ISO timestamp of when the row was produced

Highlight identity fields

input_url, username, user_id, type, id, title — identify the highlight and the profile it belongs to.

Cover media & status fields

cover_media, cover_media_full_res, cover_media_expires_at, cover_media_archive_url, success, error, timestamp — the cover image at every resolution tier, whether the row succeeded, and when it was scraped.

Profile & leaderboard analytics fields

isSummary, profile_highlight_count, profile_titles, profile_first_highlight_id, profile_last_highlight_id, profile_success_rate, isLeaderboard, leaderboard_rank — derived, uncharged rows for per-profile stats and cross-profile ranking.

🤖 Add-on: Need additional Instagram data?

If you need the profile itself — bio, follower count, verification status — pair this Actor with Instagram Profile Scraper (Scraper-Engine/Instagram-Profile-Scraper). For the video side of a profile's content, Instagram Reels Scraper Pay Per Events (Scraper-Engine/Instagram-Reels-Scraper) extracts reel metadata, views, and captions from the same accounts.

Why not build this yourself?

Instagram's Graph API has no public endpoint for retrieving an arbitrary public profile's story highlights; the Graph API only exposes a business account's own highlights back to that same account's own authenticated app, not to a third party. Building this in-house means reverse-engineering Instagram's internal GraphQL relay protocol, including doc IDs and fb_dtsg/lsd tokens, keeping that up to date every time Instagram changes its web app internals, and separately handling the 429/403 blocks that come with any meaningful scraping volume — which in practice means running and paying for your own rotating proxy pool on top of the scraping logic itself. Instagram Highlights Scraper already does all three: relay query construction, defensive response parsing, and an automatic no-proxy → datacenter → residential fallback ladder, so none of it is your maintenance burden.

How to use Instagram Highlights Scraper

Instagram Highlights Scraper runs as a standard Apify Actor — no separate signup or credentials beyond your Apify account.

1. Open Instagram Highlights Scraper on the Apify Store and click Try for free (or Start if it's already in your account).
2. Add one or more targets to startUrls (required) — a profile URL, username, highlight link, or post/reel/IGTV shortcode, one per line.
3. Optionally adjust maxHighlightsPerProfile, archiveCoverImages, enableProfileSummary, enableLeaderboard, and proxyConfiguration.
4. Click Start and watch results stream into the dataset as each highlight is scraped.
5. Export the finished dataset as JSON, CSV, or Excel, or pull it programmatically via the Apify API or client.

How to scale to bulk Instagram highlight extraction

startUrls is an array, so add as many profile URLs, usernames, highlight links, or shortcodes as needed in a single run — one line per target, no separate run per profile required. Every target is processed in sequence with its own proxy escalation state, and enableLeaderboard ranks the entire batch by highlight count once every profile finishes, so a 5-profile competitor comparison and a 500-profile bulk extraction use the exact same input shape.

What can you do with Instagram highlight data?

• A social media manager auditing brand highlights uses title and cover_media_full_res to review how a brand organizes and visually themes its highlight covers across campaigns.
• A competitive analyst tracking multiple creators in one run uses leaderboard_rank and profile_highlight_count to instantly see which accounts maintain the most active highlight libraries.
• A digital archivist preserving at-risk content uses cover_media_archive_url and cover_media_expires_at to keep a permanent copy of each cover before its Instagram CDN link expires.
• A growth marketer monitoring account health across a client list uses profile_success_rate and error to flag profiles that stopped returning highlights, for example because they went private.
• An AI engineer building a brand-monitoring agent feeds title, profile_titles, and username into an LLM as structured context to summarize a brand's highlight strategy without writing any HTML-parsing code.

Because output is typed JSON reachable over the Apify API, any agent framework that can call an HTTP endpoint can pull this data directly into its context.

How does Instagram Highlights Scraper handle rate limits and blocking?

Instagram Highlights Scraper escalates through three proxy tiers on a block: it first tries no proxy, then Apify datacenter proxy, then residential proxy, automatically triggered whenever Instagram responds with HTTP 429 or 403. Your own apifyProxyGroups and apifyProxyCountry selections are carried through both fallback tiers rather than being discarded. Once residential proxy is used for one target, the run stays in "sticky" mode and uses residential for every remaining target, avoiding repeated escalation. Residential proxy gets up to 3 retries with a fresh proxy URL each attempt before that target is marked failed. Individual HTTP requests also retry with exponential backoff, up to 3 attempts, on transient errors, separate from the proxy-tier escalation. Highlight tray pagination is capped at 50 pages per profile as a runaway-loop safety net, though normal profiles paginate in far fewer. If a target ultimately fails after every fallback, the Actor still pushes one row for it with success: false and a descriptive error message rather than silently dropping it, so failures are visible and filterable in the dataset.

⬇️ Input

Parameter	Required	Type	Description	Example Value
startUrls	Yes	array	Profile URLs, usernames, highlight links, or post/reel/IGTV shortcodes — one target per line	["https://www.instagram.com/mrbeast/"]
maxHighlightsPerProfile	No	integer	Cap on highlights fetched per profile, across all tray pages; 0 = unlimited	0
archiveCoverImages	No	boolean	Download each highlight's full-resolution cover into this run's Key-Value Store	true
enableProfileSummary	No	boolean	Push one extra, uncharged per-profile summary row after that profile's highlight rows	true
enableLeaderboard	No	boolean	Push one extra, uncharged cross-profile leaderboard row per profile, ranked by highlight count	true
proxyConfiguration	No	object	Proxy selection, including custom apifyProxyGroups/apifyProxyCountry; automatic datacenter/residential fallback on 429/403	{"useApifyProxy": true}

Example input

{
  "startUrls": [
    "https://www.instagram.com/mrbeast/",
    "nike",
    "https://www.instagram.com/stories/highlights/17895948565602293/"
  ],
  "maxHighlightsPerProfile": 0,
  "archiveCoverImages": true,
  "enableProfileSummary": true,
  "enableLeaderboard": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

⬆️ Output

Instagram Highlights Scraper writes one flat, typed JSON row per highlight to the Actor's dataset — plus one uncharged per-profile summary row and one uncharged leaderboard row when those options are enabled. Every row shares the same 21-key schema regardless of row type, with fields that don't apply to that row type set to null. Export as JSON, CSV, Excel/XLSX, or any other format the Apify dataset export menu supports.

Example output

[
  {
    "input_url": "https://www.instagram.com/swarnika.verma_/",
    "username": "swarnika.verma_",
    "user_id": "72037592921",
    "type": "instagram_highlights",
    "id": "highlight:18069978917514346",
    "title": "Luxe🛍️",
    "cover_media": "https://scontent-iad3-1.cdninstagram.com/v/t51.2885-15/528310306_....jpg?stp=dst-jpg_s150x150_tt6&...",
    "cover_media_full_res": "https://scontent-iad3-1.cdninstagram.com/v/t51.2885-15/528310306_....jpg?stp=dst-jpg_e35_1170x2080&...",
    "cover_media_expires_at": "2026-08-05T15:23:07+00:00",
    "cover_media_archive_url": "https://api.apify.com/v2/key-value-stores/<store-id>/records/cover-highlight_18069978917514346.jpg",
    "success": true,
    "error": null,
    "isSummary": false,
    "profile_highlight_count": null,
    "profile_titles": null,
    "profile_first_highlight_id": null,
    "profile_last_highlight_id": null,
    "profile_success_rate": null,
    "isLeaderboard": false,
    "leaderboard_rank": null,
    "timestamp": "2026-07-25T10:15:22.104931+00:00"
  },
  {
    "input_url": "https://www.instagram.com/swarnika.verma_/",
    "username": "swarnika.verma_",
    "user_id": "72037592921",
    "type": "instagram_highlights_profile_summary",
    "id": null,
    "title": null,
    "cover_media": null,
    "cover_media_full_res": null,
    "cover_media_expires_at": null,
    "cover_media_archive_url": null,
    "success": true,
    "error": null,
    "isSummary": true,
    "profile_highlight_count": 14,
    "profile_titles": "Luxe🛍️ | ootd🧜🏻‍♀️ | Samsung | Travel",
    "profile_first_highlight_id": "highlight:18069978917514346",
    "profile_last_highlight_id": "highlight:17864659551325076",
    "profile_success_rate": 1.0,
    "isLeaderboard": false,
    "leaderboard_rank": null,
    "timestamp": "2026-07-25T10:15:40.881212+00:00"
  },
  {
    "input_url": "https://www.instagram.com/mrbeast/",
    "username": "mrbeast",
    "user_id": "18428658",
    "type": "instagram_highlights_leaderboard_entry",
    "id": null,
    "title": null,
    "cover_media": null,
    "cover_media_full_res": null,
    "cover_media_expires_at": null,
    "cover_media_archive_url": null,
    "success": true,
    "error": null,
    "isSummary": false,
    "profile_highlight_count": 9,
    "profile_titles": null,
    "profile_first_highlight_id": null,
    "profile_last_highlight_id": null,
    "profile_success_rate": 1.0,
    "isLeaderboard": true,
    "leaderboard_rank": 1,
    "timestamp": "2026-07-25T10:16:02.554013+00:00"
  }
]

How does it work?

For each target, Instagram Highlights Scraper first loads the profile's own public page fresh, reading the CSRF cookie and app tokens it needs straight out of that response; a highlight link or post/reel/IGTV shortcode is resolved to its owning username first, through Instagram's public embed pages. It then submits Instagram's internal highlights-tray query, following the cursor across every page so large trays are not truncated. Each highlight's cover is upgraded to full resolution by looking up that highlight's own reel and matching the same photo by its CDN asset ID, never a mismatched image. If Instagram returns a block at any step, the run automatically escalates to a different proxy tier and retries. Only what a logged-out visitor could see is ever returned, and the output schema stays the same no matter how Instagram's page markup changes.

Integrations

Instagram Highlights Scraper is a standard Apify Actor, so it works with anything that can call the Apify API — your own scripts, no-code automation tools, and AI agent frameworks.

Calling Instagram Highlights Scraper programmatically

from apify_client import ApifyClient

client = ApifyClient("<APIFY_API_TOKEN>")
run = client.actor("scraper-engine/instagram-highlights-scraper").call(
    run_input={"startUrls": ["https://www.instagram.com/mrbeast/"]}
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["username"], item["title"])

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request.

No-code tools (n8n, Make, LangChain)

In n8n, add an HTTP Request node set to POST against this Actor's run-sync-get-dataset-items endpoint with your JSON input as the request body, to get highlight rows back in the same workflow step. In Make, the equivalent is an HTTP module pointed at the same endpoint, or the dedicated Apify app's "Run Actor and get dataset items" action. In LangChain, wrap that same endpoint as a custom Tool so an agent can call it directly and receive typed JSON highlight rows as tool output.

Is it legal to scrape Instagram highlights?

Scraping publicly available Instagram data — content any visitor can see without logging in — is generally permissible in most jurisdictions, and Instagram Highlights Scraper only ever returns what is already public: highlight titles, cover images, and IDs from profiles that have not restricted their content. Because rows include username and user_id, which can qualify as personal data, GDPR and CCPA may still apply to how you store, process, or reuse that data, particularly for bulk collection tied to identifiable people. You are responsible for having a lawful basis for your specific use case, and for complying with Instagram's own Terms of Use. Consult legal counsel if your use case involves bulk storage of personal data.

Frequently asked questions

What Instagram highlight fields does Instagram Highlights Scraper return?

Instagram Highlights Scraper returns title, id, cover_media, cover_media_full_res, and username as its core fields, plus 16 more covering cover archival, success/error status, and optional profile/leaderboard analytics — see the full fields table above.

Does Instagram Highlights Scraper require an Instagram account or login?

No. It reads highlight data from Instagram's public, logged-out profile and embed pages, using only a CSRF cookie issued to that anonymous page load — no Instagram username, password, or session cookie of your own is needed.

How many highlights can I extract in one run?

As many as a profile has, across every page of its highlights tray — pagination follows Instagram's cursor up to a safety cap of 50 pages per profile, well beyond what normal accounts need. Set maxHighlightsPerProfile to cap the count per profile, or leave it at 0 for unlimited.

What happens if a profile is private, doesn't exist, or has no highlights?

The Actor still pushes one row for that target with success: false and a plain-text error message (for example, "No highlights found for this profile") instead of stopping the run, so you can filter on success in code rather than parsing exceptions.

Can I scrape multiple Instagram profiles at once?

Yes. startUrls accepts an array, so you can add any number of profile URLs, usernames, highlight links, or shortcodes and they're all processed in the same run, sharing one leaderboard if enableLeaderboard is on.

Does Instagram Highlights Scraper work with Claude, ChatGPT, and other AI agent tools?

Yes, as an HTTP endpoint. Any agent framework capable of making an API call can trigger a run and read back typed JSON through the Apify API — there's no separate integration to install.

How does Instagram Highlights Scraper compare to manual ways of saving highlights?

Rather than screen-recording a highlights tray or copying titles by hand, Instagram Highlights Scraper resolves highlight links and shortcodes automatically, paginates the entire tray, upgrades each cover to full resolution, and returns everything as structured JSON rows, with no manual re-entry needed per profile.

Does Instagram Highlights Scraper return data in a format LLMs can use directly?

Yes. Every row is typed, normalized JSON with the same field names across every run — no HTML, no CSS selectors — so you can pass it straight to an LLM prompt, index it into a vector store, or feed it to an agent tool.

What happens when Instagram changes its layout or anti-bot system?

The Actor is actively maintained, and its output schema is designed to stay stable across Instagram UI and internal API changes so your downstream code doesn't break. No specific fix turnaround time is guaranteed.

Can I use Instagram Highlights Scraper without managing proxies or browser infrastructure?

Yes. The Actor handles its own proxy escalation (no proxy → datacenter → residential) and session/cookie handling internally — you only need to supply a proxyConfiguration if you want to override the defaults.

Which Instagram highlight fields work best for AI training data and RAG indexing?

For RAG, index title and profile_titles — the highest-information text fields — alongside username for source attribution. For training data, id, success, and timestamp return the most consistently structured typed primitives across every row, regardless of row type.

Related scrapers

Scraper Name	What it extracts
Instagram Profile Scraper	Bio, follower/following counts, and profile identity fields
Instagram Reels Scraper Pay Per Events	Reel video metadata, view counts, and captions
Instagram Hashtag Scraper	Posts and metadata published under a hashtag
Instagram Tagged Posts Scraper (Date & Engagement Analytics)	Posts a profile is tagged in, with date and engagement analytics
Instagram Mentions Scraper — Inbound Tagged Posts	Posts that mention a given profile
Instagram Followers Count Scraper With Engagement Quality Score	Follower counts with a computed engagement quality score

Your feedback

Found a bug or missing a field? Let us know through this Actor's Issues tab on Apify so we can take a look. Feedback like this directly shapes what gets fixed and added next.