Instagram Event Extractor

Turn public Instagram captions and event poster images into structured event records. Export JSON or CSV, and monitor new or changed announcements across runs.

Version 0.1.0: a deployable first version. Automated tests use mocked services. The offline demo uses clearly marked synthetic events. A live Instagram + Gemini run must be verified with your own credentials before publishing a paid listing.

Start here

Read DEPLOYMENT.md for the complete beginner-friendly setup, deployment, first live test, scheduling, monetization and troubleshooting guide. STORE-LISTING.md contains copy for your Store listing.

Requirements: Node.js 22+ (a current Node.js 24 LTS installation is recommended), npm, an Apify account, and a Google Gemini API key for live extraction. Customers bring their own Gemini API keys. You do not need a web server or a database subscription.

npm ci
npm test
npm run check
npm run demo

The demo makes no external calls. It writes demo-output/events.json, events.csv, summary.json and second-run-summary.json. The first simulated run returns two events; the second uses the cache and returns zero changes. This tests plumbing and history, not AI recognition accuracy.

Deploy

From this project directory:

npx apify-cli@1.10.0 login
npx apify-cli@1.10.0 push --open

The CLI deploys the name in .actor/actor.json. Keep it different from your menu consistency checker. Check the name in your account before pushing; an existing Actor with the same name can be updated. You can use a different new name in the manifest. Deployment creates a private Actor; Store publication is a separate step.

Run modes

1. Demo

{ "source": "demo", "monitorName": "my-first-demo" }

Synthetic fixtures only. No Instagram collection or Gemini analysis. Cloud platform compute may still cost money. To keep this Actor's own demo fee at zero, configure only the custom post-analyzed monetization event; do not add automatic start or dataset-item fees.

2. Instagram profiles

{
  "source": "instagram",
  "usernames": ["replace_with_a_public_venue"],
  "defaultTimezone": "Europe/Rome",
  "maxPostsPerProfile": 3,
  "maxTotalPosts": 3,
  "maxImagesPerPost": 2,
  "scraperBudgetCents": 50,
  "monitorName": "my-first-live-test",
  "onlyChanges": true
}

Replace the username. Enter your key in the secret geminiApiKey form field. Alternatively set GEMINI_API_KEY in your own private Actor environment. Do not embed an owner-funded key in a published Actor's shared configuration.

The Actor calls apify/instagram-post-scraper using its verified username and resultsLimit fields. It explicitly requests basic data and disables the music add-on. When several profiles are supplied, the per-profile limit is reduced if necessary to stay within maxTotalPosts. Upstream results may contain videos or errors, which count toward the inspected-row limit but are skipped by extraction.

3. Existing posts dataset

Choose Existing Apify dataset in the form and select a dataset produced by the Instagram Post Scraper. This avoids starting another scraper. The same Gemini key requirement applies. The dataset picker grants read access under Apify's limited-permissions model.

Input records are expected to contain id, caption, timestamp, url, ownerUsername, type, displayUrl, images and/or childPosts. Not every field is mandatory; an ID and a usable caption or image are necessary. The collector reads a bounded prefix of the dataset, rather than scanning unlimited rows to find supported posts. Refresh the source dataset yourself when monitoring it; this Actor does not refresh an existing dataset.

Outputs

The default dataset contains one row per event occurrence. SUMMARY in the run's key-value store explains counts, skipped posts, extraction failures, upstream run ID, budget stops and history location. EVENTS.csv is a spreadsheet-safe CSV. Each successful fresh analysis is also accessible as an ANALYSIS-<hash> record in the run's key-value store, including analyses that found no events.

Field	Meaning
eventId	Stable identity within this monitor's retained history
title	Event name supported by the source
startDate, endDate	Local calendar dates, YYYY-MM-DD, or null
startTime, endTime, timezone	Local 24-hour times and IANA timezone; no silent UTC conversion
venue, city, address, country	Source-backed location details, or null
ticketPrice, currency, ticketUrl	Numeric stated price, currency code and explicit link, or null
status	scheduled, cancelled or postponed
confidence, needsReview, reviewReasons	Extraction confidence and reasons to inspect the source
evidence	Quoted words supporting the extraction
sourcePost, sourcePostId, sourceUsername, sourcePublishedAt	Provenance
change, changedFields	new, changed, or unchanged, with the fields that changed
firstSeenAt, lastSeenAt	Observation timestamps
demo	True for synthetic data

For a sample, run npm run demo and inspect the generated JSON.

Dates, duplicates and history

• Dates without an explicit year remain uncertain. Relative dates can be resolved against the post's publication timestamp. The model is instructed never to infer a year from the current wall clock.
• Missing or questionable details remain null or flagged; AI can still make mistakes. Validate a representative set of real posters before relying on it.
• Set the venue's timezone. A default timezone is an assumption and is flagged when the source does not supply one.
• Exact title/date/location matches can be merged across posts. Separate recurring dates remain separate. Same-post date/time edits can preserve event identity when unambiguous. A rescheduling announcement in a different post is not guaranteed to be matched to the original event.
• An absent or deleted post never automatically cancels an event. Cancellations need explicit source evidence.
• A named key-value store keeps extraction cache and event history. Its name includes this Actor's ID and your monitorName. State keys isolate profiles/dataset, model, timezone and extraction limits. Reuse the same configuration for repeated monitoring.
• Successful unchanged posts skip Gemini and this Actor's analysis fee. Instagram mode still refreshes its upstream posts and can incur scraper fees. Signed CDN URL query changes alone do not invalidate the cache.
• onlyChanges: false returns all eligible events in the current input window, not every event ever observed. forceReprocess: true analyzes unchanged posts again and incurs normal costs.
• History retains records seen within 90 days and at most 10,000 posts and 10,000 events. A retained event is not a permanent calendar archive.
• Run only one job at a time per monitor. There is no distributed lock; overlapping jobs can overwrite state. Output delivery is at least once after a crash between dataset write and history save. Downstream consumers should upsert by eventId.

Cost controls and monetization

There are separate cost components: this Actor's fee (if you configure one), upstream scraper charges, Gemini usage on the supplied API key, and applicable Apify platform usage. The post-analysis price is not an all-inclusive total.

scraperBudgetCents caps the upstream PPE run only. The platform's maximum charge on this Actor governs its own event fees. Neither limit caps Gemini. Post/image limits bound work; monitor your Gemini quotas and billing separately. No exact total-cost estimate is promised without measuring actual inputs.

The code is ready for one custom pay-per-event event: post-analyzed. It is charged once per successful complete fresh or forced post analysis, even when the result is zero events. A failed or visually incomplete analysis, cached result or demo fixture does not emit that event. Price configuration and publishing must be done in Apify Console. Do not combine it with automatic dataset-item fees, which would charge again for multi-event posts.

Scope and limitations

This release handles public image posts, captions, and image slides in carousels. It skips whole video/Reel posts and video slides. It does not analyze Stories, audio, linked pages, ticket inventory or whole cities automatically. Ticket links are extracted only when written explicitly in the analyzed source; "link in bio" is not resolved.

CDN downloads are limited to Instagram/Facebook image domains, JPEG/PNG/WebP/GIF, 8 MiB per image, and 16 MiB total per post. Redirects and arbitrary image hosts are rejected. An analysis with failed or rejected image downloads is stored only in its audit record with complete: false; it does not update the event feed, enter the successful cache or incur this Actor's analysis fee. It is retried on the next run, so upstream, Gemini and platform costs can recur. Configured caption/carousel truncation instead remains a complete, cacheable analysis with review warnings. Only the configured number of images and events per post can be processed. Posters with many occurrences or unreadable typography need review.

Extraction sends captions and selected image bytes to Google's Gemini API. No Instagram passwords, login cookies or private content are requested. Use data you are authorized to access and check applicable platform terms and data-handling requirements. This tool is independent of Instagram and Meta.

Development

Source is plain JavaScript on Node.js with the Apify SDK. Tests use Node's built-in test runner and mocked HTTP/services. src/main.js provides the Apify lifecycle; collector.js collects posts; extractor.js calls Gemini; events.js validates/reconciles events; pipeline.js handles cache, history and output.

npm start runs through the actual Apify SDK locally. Put input in storage/key_value_stores/default/INPUT.json. Local live Instagram collection also needs APIFY_TOKEN; the cloud injects a scoped token automatically. For the easiest credential-free check use npm run demo.

References

• Apify Instagram Post Scraper
• Gemini model
• Gemini image understanding
• Gemini structured output
• Apify permissions
• Apify deployment
• Apify pay-per-event monetization