Bluesky Scraper — Fast & Monitoring-Friendly

Scrape Bluesky posts, author feeds, search results, and reply threads via Bluesky's public AppView API. No login, API key, app password, or credentials of any kind are required — this works out of the box the moment you hit Run. Built for speed, reliability, and monitoring: every result carries a stable dedup key and an ISO timestamp, so you can schedule this actor and diff each run's output to catch new posts or replies.

What it does

• Scrapes Bluesky search results (sort: latest or top)
• Scrapes a specific account's post feed (authors)
• Scrapes a single post plus its reply thread from a direct link
• Optionally follows discovered posts to pull their replies too
• Talks directly to https://api.bsky.app — Bluesky's read-only AppView, which needs no authentication for these endpoints — with a descriptive User-Agent, retrying 429s automatically
• Pushes clean, flat JSON records to the dataset — ready for Excel/Sheets, a database, or a diffing script

No setup required

Unlike most social scrapers, there is nothing to configure before your first run: no account to create, no app password to generate, no API key to paste in. Bluesky's AppView (api.bsky.app) serves posts, feeds, search, and threads to anyone, unauthenticated — that's the whole reason this actor is fast to start and simple to operate. Just fill in what you want to scrape and click Run.

Input

At least one of searchQueries, authors, or startUrls is required.

Field	Type	Default	Notes
searchQueries	array of strings	[]	Keyword searches. If authors is also set, each query is searched scoped to each author
authors	array of strings	[]	Bluesky handles or DIDs whose feed to scrape, e.g. "bsky.app"
startUrls	array	[]	Direct bsky.app URLs: a profile (https://bsky.app/profile/{handle}) or a single post (https://bsky.app/profile/{handle}/post/{rkey})
sort	enum	latest	latest or top — only applies to search; author feeds are always newest-first
maxItems	integer	100	Hard cap on posts scraped this run — your main billing guardrail
includeReplies	boolean	false	Master switch for reply scraping — must be true for any reply-thread requests to be made
maxRepliesPerPost	integer	50	Cap on replies per post (keeps billing predictable); 0 = no cap (every reply in that post's thread)
scrapeType	enum	posts	posts, replies, or both — controls what gets pushed; replies are still gated by includeReplies
proxyConfiguration	object	{ "useApifyProxy": true }	Plain datacenter proxy is fine — the public AppView doesn't IP-block like Reddit does, so residential is not required

Example: monitor a keyword for new posts

{
    "searchQueries": ["nvidia earnings"],
    "sort": "latest",
    "maxItems": 50,
    "scrapeType": "posts"
}

Example: scrape an account's feed

{
    "authors": ["bsky.app"],
    "maxItems": 25,
    "scrapeType": "posts"
}

Example: posts + replies for a keyword

{
    "searchQueries": ["apify"],
    "maxItems": 20,
    "scrapeType": "both",
    "includeReplies": true,
    "maxRepliesPerPost": 50
}

Example: a single post's reply thread

{
    "startUrls": [{ "url": "https://bsky.app/profile/bsky.app/post/3juj4vqfjyk2m" }],
    "scrapeType": "both",
    "includeReplies": true
}

Output

Results are pushed to the actor's default dataset — one JSON object per row.

Post record

Field	Type	Description
uri	string	The post's at:// URI (globally unique, permanent)
cid	string	Content hash (CID) of the post record
url	string	Clickable https://bsky.app/profile/{handle}/post/{rkey} permalink
authorHandle	string	Author's handle, e.g. alice.bsky.social
authorDid	string	Author's DID (permanent identifier)
authorDisplayName	string | null	Author's display name
text	string	Post text
createdAt	string	ISO 8601 — client-declared post creation time (record.createdAt)
indexedAt	string	ISO 8601 — when the AppView indexed the post
likeCount	number	Like count
repostCount	number	Repost count
replyCount	number	Reply count
quoteCount	number	Quote-post count
langs	array of strings	Declared post languages
hasMedia	boolean	Whether the post has an embed (image/video/link/quote)
embedType	string | null	Embed $type, e.g. app.bsky.embed.images#view
dedupKey	string	Stable dedup key — the post's at:// URI

Reply record (when includeReplies is on)

Same shape as a post record, plus:

Field	Type	Description
parentUri	string | null	at:// URI of the immediate parent post
rootUri	string | null	at:// URI of the thread's root post
depth	number	Nesting depth relative to the post whose thread was fetched (0 = direct reply)

Use it for monitoring: schedule + dedupKey

This actor is built to be run on a schedule (e.g. every 15–30 minutes with sort: "latest") and diffed against the previous run:

1. Schedule the actor with the same input each time.
2. Compare each run's dataset against the last one using dedupKey (the post's at:// URI) as the join key — any key that's new since the last run is a new post/reply.
3. Because createdAt/indexedAt are proper ISO timestamps, you can sort/filter chronologically without any conversion step, and feed the diff straight into an alerting workflow (email, Slack, webhook — bring your own).

maxItems is intentionally the primary billing guardrail: it caps posts per run regardless of how many results the search or feed would otherwise return, so scheduled runs have a predictable cost ceiling.

Pricing (pay-per-event)

This actor uses Apify's pay-per-event pricing — no per-compute-unit charges. You're billed:

• post-scraped — once per post successfully scraped
• reply-scraped — once per reply successfully scraped (only when includeReplies is on)

Nothing is charged until an item is actually pushed to the dataset, and charging always respects maxItems / maxRepliesPerPost — see .actor/pay_per_event.json for current per-event prices.

Local development

npm install
npm start

A sample input is provided at storage/key_value_stores/default/INPUT.json (searches "apify", 10 posts, latest sort). No credentials or environment variables are needed — Bluesky's public AppView is open, but it returns 403 to the searchPosts endpoint from cloud/datacenter egress IPs (observed from Apify's own IPs, 2026-08-30). Apify Proxy (datacenter group) is therefore ON by default and required for search; it also gives consistent egress in production, but light local testing works without it too.

Notes & known limitations (v1)

• Bluesky's AppView applies its own rate limits; the crawler caps concurrency at 4 and retries 429s automatically via Crawlee's session/proxy rotation rather than a hand-rolled rate limiter.
• Author feeds skip pure reposts (Bluesky's reasonRepost items) since those posts are authored by a different account than the one you asked to scrape — only the account's own posts (including pinned ones) are pushed. Use scrapeType: "posts" with searchQueries + authors combined if you want author-scoped keyword matches instead.
• Reply threads are fetched with depth=10 (deeper than the API's own default of 6); maxRepliesPerPost caps the flattened, pushed count — not the raw fetch depth — so very large/deep threads may need more than one level of nesting to fully exhaust the cap.
• This is a scraper, not an alerting service: it produces alert-friendly output (dedup keys
  • ISO timestamps) but does not itself send notifications. Wire its dataset into your own schedule + diff + notify pipeline.