🌟 The Fastest Instagram Metadata Scraper & Downloader in Apify

One-line hero: Fast, privacy-safe Instagram metadata scraper + optional downloader — plug-and-play on Apify Console, enterprise-grade reliability, results in seconds.

---

📖 Summary

This Actor extracts clean metadata (captions, hashtags, owner, thumbnails, durations, short human-friendly dates) from Instagram post/reel URLs and — optionally — provides direct download links or performs secure downloads with optional merging. Designed for fast, resilient runs and easy Console usage.

Instant outcome: feed a list of start URLs → get normalized JSON records in Dataset / Key-Value store in seconds.

---

💡 Use cases / When to use

• Collect metadata for moderation, analytics, or cataloging (captions, hashtags, owner info).
• Quickly fetch download links or download video/audio artifacts (optional, best-effort).
• Backfill datasets for BI pipelines or social media research.
• Lightweight monitoring of a set of public posts / reels for changes.

---

⚡ Quick Start (Console — one-click)

1. Open this Actor in Apify Console.
2. Paste your list of startUrls in the Input editor (see example below).
3. Click Run → results appear in the Dataset and under Key-Value store key OUTPUT.

(Hero screenshot recommended: Console input + Run button — add a GIF for best conversion.)

---

⚙️ Quick Start (CLI + API)

CLI (one-liner)

# Run this Actor on Apify platform using Apify CLI
# Replace <OWNER/ACTOR_NAME> and use your input file or pipe JSON
echo '{"startUrls": ["https://www.instagram.com/reel/SHORTCODE/"]}' | apify call <OWNER/ACTOR_NAME> --silent --output-dataset

Python (apify-client) — compact snippet

from apify_client import ApifyClient
client = ApifyClient('<APIFY_TOKEN>')
run = client.actor('OWNER/ACTOR_NAME').call(input={
    'startUrls': ['https://www.instagram.com/reel/SHORTCODE/'],
    'mode': 'scrape',
})
print('Run started:', run['id'])

---

📝 Inputs (fields & schema)

Below is a compact Console JSON input example (also in input.example.json). The Actor accepts a JSON object. The platform's Input Schema (if provided by you) will auto-generate the Console UI.

{
  "mode": "scrape",
  "startUrls": [
    "https://www.instagram.com/reel/SHORTCODE/",
    "https://www.instagram.com/p/SHORTCODE2/"
  ],
  "desired_resolution": "1080p",
  "download": false,
  "merge_if_ffmpeg": false,
  "hide_media_links": true,
  "preserve_thumbnails": true,
  "maxConcurrency": 3,
  "diagnostic": false,
  "suppress_errors": true
}

---

⚙️ Configuration (friendly table)

🔑 Name	📝 Type	❓ Required	⚙️ Default	📌 Example	🧠 Notes
mode	string	✅ Yes	"scrape"	"scrape" / "download" / "both"	Determines scrape vs. download behavior
startUrls	array	✅ Yes	None	["https://.../reel/SHORTCODE/"]	List of post/reel URLs
proxyConfiguration	object	⚙️ Optional	{}	{"useApifyProxy": true}	Use Console proxy UI or custom proxy object
preferred_proxy_type	string	⚙️ Optional	"auto"	"residential"	Prefer residential proxies if needed
force_residential	boolean	⚙️ Optional	false	true	Alias for preferred proxy behavior
download	boolean	⚙️ Optional	false	true	If true, performs (best-effort) downloads
desired_resolution	string	⚙️ Optional	"1080p"	"720p"	Requested resolution for downloads
merge_if_ffmpeg	boolean	⚙️ Optional	false	true	If true and ffmpeg available, merges audio/video
cookie_file	string	⚙️ Optional	None	"cookies.txt"	Path in actor storage to cookie file (if used)
hide_media_links	boolean	⚙️ Optional	true	true	Redact raw media URLs in pushed records
preserve_thumbnails	boolean	⚙️ Optional	true	false	Keep thumbnail URLs even when hiding media
maxConcurrency	integer	⚙️ Optional	3	5	Controls parallelism (1–10)
diagnostic	boolean	⚙️ Optional	false	true	Enables verbose logs for debugging
suppress_errors	boolean	⚙️ Optional	true	false	Show errors in logs vs. suppress them

Console tip: Paste a single JSON object in the Input field (or select a saved Task) and click Run.

---

📄 Outputs (Dataset / KV examples)

Where results land:

• Metadata & download summaries are pushed to the Dataset (and final batch also stored to Key-Value under key OUTPUT).

Example output record (schema-friendly)

{
  "original_url": "https://www.instagram.com/reel/SHORTCODE/",
  "id": "SHORTCODE",
  "ownerUsername": "creator_name",
  "ownerFullName": "Creator Name",
  "description": "Caption text...",
  "hashtags": ["fun", "reel"],
  "mentions": ["other_user"],
  "likesCount": 1234,
  "likesDisplay": "1.2k",
  "commentsCount": 12,
  "commentsDisplay": "12",
  "videoViewCount": 5432,
  "viewsDisplay": "5.4k",
  "upload_date_iso": "2024-10-04T12:34:56Z",
  "upload_date": "4th October 2024",
  "upload_time_ago": "1 Month Ago",
  "duration_seconds": 33.123,
  "duration_display": "0 Minutes 33 Seconds",
  "thumbnail": "https://.../thumbnail.jpg",
  "download_links_summary": { "available": { "merged_video": false, "video_without_audio": true, "audio": true } },
  "randomExtraInfo": { "scrapeBatch": "Reel-Set-07", "regionCode": "US" }
}

Note: raw media URLs may be redacted depending on hide_media_links and preserve_thumbnails settings (privacy-safe by default).

---

🔑 Environment Variables (placeholders)

• APIFY_TOKEN — Set in Console integrations or use platform authentication. (Placeholder: <APIFY_TOKEN>)
• HTTP_PROXY / HTTPS_PROXY — Only for custom proxy setups (use secrets, do not store plaintext). Example: <PROXY_USER:PASS@HOST:PORT>

---

▶️ How to Run (Console, CLI, API)

1. Console — Open Actor → paste JSON input → Click Run. Results appear in Dataset; key-value OUTPUT stores the full batch.
2. CLI — Use the apify CLI: echo '{"startUrls":["https://.../reel/SHORTCODE/"]}' | apify call <OWNER/ACTOR_NAME> --silent --output-dataset.
3. API / apify-client (Python) — See snippet above; use ApifyClient('<APIFY_TOKEN>').actor('OWNER/ACTOR_NAME').call(input=...).

---

⏰ Scheduling & Webhooks

• Scheduling: Use Apify Console Scheduler to run this Actor periodically (e.g., hourly/daily). Choose your saved input task and frequency in Console.
• Webhooks: Configure a run onFinish webhook in Console to receive run status and dataset IDs for downstream processing.

---

🕾️ Logs & Troubleshooting

• Where to find logs: Open the run in Apify Console → Logs tab (filter by diagnostic to see verbose traces).

• Common issues & quick fixes:

  • No startUrls provided → add startUrls to input.
  • Rate-limit / access errors → enable proxy or try preferred_proxy_type: "residential".
  • Downloads failing → ensure download is true and the run has network access; check diagnostic logs.

---

🔒 Permissions & Storage Notes

• Storage used: Dataset for item records; Key-Value store under key OUTPUT for the result batch; temporary actor storage for downloads.
• Privacy & safety: By default, raw media URLs are redacted (hide_media_links: true) and thumbnails preserved. Store credentials (cookies, proxy auth) only as Console secrets — never in plain input.

---

🔟 Changelog / Versioning

• v1.0.0 — Initial release: metadata scraping + best-effort downloader, proxy-friendly, diagnostic mode.

Use semantic versioning for future releases.

---

🖌 Notes / TODOs

• TODO: confirm output schema — derived from runtime behavior but please verify against your integration.
• TODO: add demo GIF/screenshots (Console Run + sample record) for higher conversion.

---

🌍 Proxy Configuration

Enable Apify Proxy (quick): In Console → Integrations → Proxy → select Apify Proxy (recommended for reliability and rate-limit handling).

Custom proxies: Provide proxyConfiguration in input or set HTTP_PROXY / HTTPS_PROXY as secrets. Example placeholders:

{
  "proxyConfiguration": { "useApifyProxy": false, "proxyUrls": ["http://<PROXY_USER:PASS@HOST:PORT>"] }
}

Environment variables (examples):

export HTTP_PROXY="http://<PROXY_USER:PASS@HOST:PORT>"
export HTTPS_PROXY="http://<PROXY_USER:PASS@HOST:PORT>"

Security note: Store proxy credentials as Console secrets and do not place them directly in the input JSON.

Advanced: TODO: Consider proxy rotation for large-scale scraping — use session-based proxy URLs to reduce rate-limiting.

---

📚 References

• Apify Actor README guidance (official)
• Actor input schema & Console Input generation (official)
• Apify CLI & apify-client examples (official)

---

🤔 What I inferred from main.py

• Network activity present (HTML metadata fetch + downloader) — Proxy configuration is supported and recommended.
• Inputs: mode, startUrls, proxyConfiguration, preferred_proxy_type, download, desired_resolution, merge_if_ffmpeg, cookie_file, hide_media_links, preserve_thumbnails, maxConcurrency, diagnostic, suppress_errors.
• Outputs: Dataset items (normalized metadata), Key-Value store key OUTPUT holds full batch.
• The Actor attempts a primary extractor (metadata-first) with an HTML fallback and best-effort download link extraction; downloads are optional and non-fatal.

---

Why this Actor? — Trusted, production-ready, and privacy-conscious: runs in Console with zero setup, yields normalized JSON ready for analytics pipelines, and keeps sensitive links hidden by default. Run this Actor on Apify Console — get instant results in seconds.

---

---

input.example.json

{
  "mode": "scrape",
  "startUrls": [
    "https://www.instagram.com/reel/SHORTCODE/"
  ],
  "desired_resolution": "1080p",
  "download": false,
  "hide_media_links": true,
  "preserve_thumbnails": true,
  "maxConcurrency": 3
}

CONFIG.md

Quick config & proxy tips

• Use Console Proxy integration for reliable runs.
• If using custom proxies, add them as Console secrets and reference via proxyConfiguration or HTTP_PROXY env var.
• If you enable downloads and merging, ensure ffmpeg is available in your run environment; merging is optional and automatic only when enabled.

Advanced TODO

• TODO: Add formal INPUT_SCHEMA.json for Console auto UI.
• TODO: Add example Console screenshot and animated GIF for the README hero.

---

End of generated README + examples.

---

actor.json

{
  "name": "merged-instagram-metadata-downloader",
  "title": "Merged Instagram Metadata & Downloader Actor",
  "version": "1.0.0",
  "main": "src/main.py",
  "buildTag": "latest",
  "memory": 4096,
  "timeout": 36000
}

---

INPUT_SCHEMA.json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Merged Instagram Metadata & Downloader Actor - Input Schema",
  "type": "object",
  "properties": {
    "mode": { "type": "string", "enum": ["scrape", "download", "both"], "default": "scrape", "description": "Operation mode" },
    "startUrls": { "type": "array", "items": { "anyOf": [{"type":"string"},{"type":"object","properties":{"url":{"type":"string"}}}] }, "description": "List of post/reel URLs (string or {\"url\": \"...\"})" },
    "desired_resolution": { "type": "string", "default": "1080p" },
    "download": { "type": "boolean", "default": false },
    "merge_if_ffmpeg": { "type": "boolean", "default": false },
    "cookie_file": { "type": "string" },
    "hide_media_links": { "type": "boolean", "default": true },
    "preserve_thumbnails": { "type": "boolean", "default": true },
    "maxConcurrency": { "type": "integer", "minimum": 1, "maximum": 10, "default": 3 },
    "diagnostic": { "type": "boolean", "default": false },
    "suppress_errors": { "type": "boolean", "default": true },
    "proxyConfiguration": { "type": "object" },
    "preferred_proxy_type": { "type": "string", "enum": ["auto", "residential"], "default": "auto" },
    "force_residential": { "type": "boolean", "default": false },
    "proxy_country": { "type": "string" }
  },
  "required": ["mode", "startUrls"],
  "additionalProperties": false
}

---

Notes:

• actor.json gives minimal runtime metadata for Console packaging. Adjust memory and timeout based on your workload.
• INPUT_SCHEMA.json will enable Console UI generation (auto form fields) when uploaded with the Actor. Update descriptions as needed.

---

End of generated README + examples.