📸 Instagram Hashtag Scraper

A high-performance, production-ready Apify Actor for extracting Instagram posts by hashtag. Built using Crawlee for Python and Playwright, this actor features API-first network interception, eliminating reliance on volatile HTML DOM classes.

Updated for 2026: Uses internal API sniffing for 10x faster results and zero DOM-dependency issues.

---

⚡ Features

• 🚀 API-First Architecture: Intercepts direct XHR/Fetch GraphQL and REST responses (graphql/query & api/v1/tags/web_info). No fragile HTML scraping or DOM selector reliance.
• ⚡ High-Speed & Lightweight: Rapid JSON parsing directly from intercepted network payloads.
• 🛡️ Proxy Rotation & Session Pool: Full Apify Proxy integration (Residential proxies recommended) with automatic IP rotation to bypass Instagram anti-bot checks.
• 🔄 Robust 429 Handling: Automatically detects HTTP 429 Rate Limited responses, retires affected proxy sessions, and safely retries requests.
• 💰 Pay-per-Result Billing: Built-in Actor.charge('result-item', count=N) monetization model. You only pay for valid extracted posts.

---

📥 Input Parameters

Parameter	Type	Required	Default	Description
hashtag	String	Yes	—	Instagram hashtag to scrape (without # symbol, e.g. photography).
max_items	Integer	No	50	Maximum number of posts to extract.
proxy_configuration	Object	No	{"useApifyProxy": true}	Apify Proxy configuration. Residential proxies are strongly recommended.

Example Input

{
  "hashtag": "nature",
  "max_items": 50,
  "proxy_configuration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

---

📤 Output Data Schema

Each extracted item in the dataset follows a standardized, clean JSON schema:

{
  "url": "https://www.instagram.com/p/C_xY1z2S345/",
  "caption": "Exploring the majestic mountains! #nature #landscape #photography",
  "shortcode": "C_xY1z2S345",
  "owner_username": "photographer_john",
  "like_count": 1420,
  "comment_count": 89,
  "timestamp": 1722355200
}

Output Fields

• url (string): Direct URL to the Instagram post.
• caption (string): Full caption text of the post.
• shortcode (string): Unique Instagram post shortcode ID.
• owner_username (string): Username of the post creator.
• like_count (integer): Total number of likes.
• comment_count (integer): Total number of comments.
• timestamp (integer): Post publication Unix epoch timestamp.

---

🛠️ Local Development & Deployment

Run Locally

1. Install dependencies:

   $pip install -r requirements.txt
   

2. Run Playwright browser install:

   $playwright install chromium
   

3. Execute the actor locally:

   $apify run
   

Deploy to Apify

Deploy directly using the Apify CLI:

$apify push

---

📜 License

Apache-2.0