# Bluesky Scraper (`fanndev/bluesky-scraper`) Actor

Scrape Bluesky profiles, posts, threads, followers, likes, reposts, quotes, custom feeds, lists, starter packs, trends and full-text search -- 15 modes, no login or API key needed.

- **URL**: https://apify.com/fanndev/bluesky-scraper.md
- **Developed by:** [Faisal Ahdan naufal](https://apify.com/fanndev) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.10 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Bluesky Scraper

Scrape Bluesky (AT Protocol) without an account, an app password, or an
API key. One actor, 15 modes, one consistent output envelope.

Built on Bluesky's own public AppView (`public.api.bsky.app`) — the same
XRPC API the bsky.app web client calls. No login, no browser, no
CAPTCHA solving.

### Modes

Pick a `mode`, then fill the one input list it reads.

#### Account modes — read **Bluesky accounts** (`actors`)

Accepts a handle (`bsky.app`), an `@handle`, a DID (`did:plc:…`), or a
profile URL (`https://bsky.app/profile/bsky.app`) — mixed freely in one
list.

| Mode | Emits | Notes |
| --- | --- | --- |
| `profile` | `PROFILE` | Full account detail: bio, avatar, banner, follower/following/post counts, verification, pinned post. Batched 25 accounts per request. |
| `posts` | `POST` | An account's timeline. Filter with `postFilter` (replies / media only / video only). |
| `followers` | `FOLLOWER` | Who follows the account. |
| `follows` | `FOLLOWING` | Who the account follows. |
| `lists` | `LIST` | Curation and moderation lists the account created. |
| `starter-packs` | `STARTER_PACK` | Starter packs the account created. |

#### Post modes — read **Post URLs** (`postUrls`)

Accepts `https://bsky.app/profile/<handle>/post/<rkey>` or a raw `at://`
URI.

| Mode | Emits | Notes |
| --- | --- | --- |
| `post-detail` | `POST` + `REPLY` | The post plus its reply thread. `threadDepth` controls how deep; replies come back flat with a `replyDepth` field so the tree stays reconstructable. `parentHeight` pulls in ancestors. |
| `post-likes` | `LIKE` | Which accounts liked the post, with timestamps. |
| `post-reposts` | `REPOSTED_BY` | Which accounts reposted it. |
| `post-quotes` | `QUOTE` | Posts that quote it, as full post records. |

#### Search modes — read **Search queries** (`searchQueries`)

| Mode | Emits | Notes |
| --- | --- | --- |
| `search-posts` | `POST` | Full-text post search. Filter by `sort`, `since`, `until`, `lang`, `author`, `mentions`, `domain`, `url`, `tag`. **Read the rate-limit note below.** |
| `search-users` | `PROFILE` | Account search. |

#### Feed mode — reads **Feed / list URLs** (`feedUrls`)

| Mode | Emits | Notes |
| --- | --- | --- |
| `feed` | `POST` | Posts from a custom feed (`/profile/<did>/feed/<rkey>`) or a list (`/profile/<did>/lists/<rkey>`). The two are told apart automatically. |

#### Discovery modes — need no input at all

| Mode | Emits | Notes |
| --- | --- | --- |
| `trends` | `TREND` | What is trending now: topic, category, post count, start time, sample accounts. |
| `popular-feeds` | `FEED_GENERATOR` | The most-liked custom feeds on the network. |

### Output

Every record carries the portfolio envelope, then the upstream payload
**verbatim** — field names are never renamed, so a `POST` record is a
literal `app.bsky.feed.defs#postView` and a `PROFILE` record is a
`app.bsky.actor.defs#profileViewDetailed`.

```json
{
  "_input": "bsky.app",
  "_source": "S1-xrpc",
  "_scrapedAt": "2026-08-29T01:14:13Z",
  "recordType": "POST",
  "uri": "at://did:plc:z72i7hdynmk6r22z27h6tvur/app.bsky.feed.post/3mu3jzayuys2k",
  "cid": "bafyreiduk5omcv2kkdmv24nvslkgs65uxpino6ftvukqkkc32qwvzqokzu",
  "author": { "did": "…", "handle": "bsky.app", "displayName": "Bluesky", "…": "…" },
  "record": { "$type": "app.bsky.feed.post", "text": "…", "createdAt": "…", "facets": [], "…": "…" },
  "embed": { "$type": "app.bsky.embed.images#view", "images": [] },
  "replyCount": 42, "repostCount": 17, "likeCount": 310, "quoteCount": 4,
  "webUrl": "https://bsky.app/profile/bsky.app/post/3mu3jzayuys2k"
}
```

Two fields are added by this actor rather than passed through:
`webUrl` (the human-facing permalink) and `replyDepth` (on `REPLY`
records only).

Split mixed output with `recordType`. `_input` always echoes the exact
string you supplied, so results join back to inputs even when you mixed
URLs, handles and DIDs in one list.

#### Failures produce rows too

A run never silently drops an input. Failed inputs get a row with
`recordType: "ERROR"` and no payload:

| `_error` | Meaning |
| --- | --- |
| `not_found` | The account, post or thread does not exist (or was deleted). |
| `empty_result` | Upstream returned zero items — e.g. a post with no likes. |
| `bad_input_or_shape` | The input was not the shape the mode needs (e.g. a profile URL passed to a post mode). |
| `rate_limited` | `search-posts` hit the IP limit and had no proxy to rotate to. |
| `invalid_request` | Upstream rejected the parameters. |
| `auth_required` | The endpoint needs a login — out of scope for this actor. |
| `exception` | Anything else, with the repr attached. |

### Limits and gotchas

**`search-posts` is rate-limited per IP.** It is the one endpoint
Bluesky does not serve from the open AppView, and its host applies a
token bucket of roughly 5 requests before returning HTTP 403 for a
minute or more. Retrying eagerly makes it worse — measured, a 300 s rest
after a retry storm still returned zero tokens.

- **With Apify Proxy on (the default): the actor rotates to a fresh exit
  IP on each block, which is a fresh bucket.** Leave the proxy enabled
  for this mode.
- Without a proxy the actor paces itself to ~1 request per 20 s, waits
  90 s on a block, and after two waits gives up with a `rate_limited`
  row rather than hammering.
- `search-posts` is forced to concurrency 1 — parallel workers share one
  IP and only drain the bucket faster.

Every other mode is unaffected and needs no proxy at all: the public
AppView absorbed a 30× burst at `limit=100` without a single failure.

**Not available without a login** (so, not offered here): your home
timeline, an account's own likes, DMs, notifications, and follow
suggestions. These are private or session-scoped in Bluesky itself.

**Counts are point-in-time.** `likeCount`, `followersCount` and friends
are whatever the AppView had indexed at `_scrapedAt`.

**`maxItems` is per input item**, not per run. Three accounts at
`maxItems: 100` yields up to 300 rows.

**`trends` caps at 25** regardless of `maxItems` — that is Bluesky's own
limit on the endpoint, not a choice made here.

### Technical notes

- **No WAF.** Plain `httpx` returns 200; so does every `curl_cffi`
  profile tried. `chrome124` is pinned for portfolio consistency, not as
  a bypass — do not waste time rotating profiles against this target.
- **Pagination** is cursor-based with a hard `limit` cap of 100, and
  stops on any of: cursor absent, empty page, repeated cursor, or
  `maxItems` reached. Verified duplicate-free across 6 pages / 520
  records.
- **Handle-based `at://` URIs are resolved to DIDs before use.** This is
  not cosmetic: `getLikes`, `getQuotes` and `getPosts` return **HTTP 200
  with an empty array**, and `getRepostedBy` returns HTTP 500, when the
  URI authority is a handle instead of a DID. Since every bsky.app post
  URL is handle-based, skipping this produces silently empty datasets.

Full recon evidence, including the endpoint census and the rate-limit
measurements, is in [CRAWLING\_METHOD.md](CRAWLING_METHOD.md).

### Local development

```bash
pip install -r requirements.txt

python test_local.py             # smoke-test all 15 modes
python test_local.py posts bsky.app

## Full Apify lifecycle against local storage:
##   put your input in storage/key_value_stores/default/INPUT.json
APIFY_LOCAL_STORAGE_DIR=./storage python -m src
```

`test_local.py` audits the envelope on every row and flags any mode that
returns zero records — a mode answering `200 OK` with nothing is treated
as a failure, not a pass.

# Actor input Schema

## `mode` (type: `string`):

What to scrape. Each mode reads ONE of the input lists below -- see each field's description for which. ACTOR modes (read "Bluesky accounts"): profile, posts, followers, follows, lists, starter-packs. POST modes (read "Post URLs"): post-detail, post-likes, post-reposts, post-quotes. SEARCH modes (read "Search queries"): search-posts, search-users. FEED mode (reads "Feed / list URLs"): feed. DISCOVERY modes (need no input at all): trends, popular-feeds.

## `actors` (type: `array`):

Used by the ACTOR modes (profile, posts, followers, follows, lists, starter-packs). Accepts a handle (bsky.app), an @handle, a DID (did:plc:...), or a profile URL (https://bsky.app/profile/bsky.app) -- mixed freely.

## `postUrls` (type: `array`):

Used by the POST modes (post-detail, post-likes, post-reposts, post-quotes). Accepts a bsky.app post URL (https://bsky.app/profile/<handle>/post/<rkey>) or a raw at:// URI.

## `searchQueries` (type: `array`):

Used by the SEARCH modes (search-posts, search-users). One dataset page set per query. Bluesky's inline operators work inside the query text too (for example: from:bsky.app "release notes").

## `feedUrls` (type: `array`):

Used by the "feed" mode. Accepts a custom-feed URL (https://bsky.app/profile/<did>/feed/<rkey>), a list URL (https://bsky.app/profile/<did>/lists/<rkey>), or the equivalent at:// URI. Feeds and lists are told apart automatically.

## `maxItems` (type: `integer`):

Upper bound on records collected for EACH input item (each account, post, or query) -- not for the run as a whole. Pagination stops early when the account or result set runs out.

## `postFilter` (type: `string`):

"posts" mode only -- which of the account's posts to return. Default (posts\_with\_replies) returns everything.

## `includePins` (type: `boolean`):

"posts" mode only -- put the account's pinned post at the top of its feed.

## `threadDepth` (type: `integer`):

"post-detail" mode only -- how many reply levels below the post to walk. 0 returns the post alone; 6 is the bsky.app web default. Replies come back flat with a replyDepth field, so the tree is still reconstructable.

## `parentHeight` (type: `integer`):

"post-detail" mode only -- how many ancestor posts above the given post to include. 0 (default) treats the given post as the root.

## `sort` (type: `string`):

"search-posts" mode only -- rank results by engagement (top) or recency (latest).

## `since` (type: `string`):

"search-posts" mode only -- only posts created at or after this time. ISO 8601 (2026-08-01T00:00:00Z) or a plain date (2026-08-01).

## `until` (type: `string`):

"search-posts" mode only -- only posts created before this time. ISO 8601 (2026-08-29T00:00:00Z) or a plain date (2026-08-29).

## `lang` (type: `string`):

"search-posts" mode only -- restrict to posts tagged with this BCP-47 language code, e.g. id, en, ja.

## `author` (type: `string`):

"search-posts" mode only -- restrict to posts written by this handle or DID.

## `mentions` (type: `string`):

"search-posts" mode only -- restrict to posts that mention this handle or DID.

## `domain` (type: `string`):

"search-posts" mode only -- restrict to posts linking to this domain, e.g. nytimes.com.

## `url` (type: `string`):

"search-posts" mode only -- restrict to posts linking to this exact URL.

## `tag` (type: `array`):

"search-posts" mode only -- restrict to posts carrying every one of these hashtags. Enter them without the leading # sign.

## `maxConcurrency` (type: `integer`):

How many input items to process in parallel. Ignored for "search-posts", which is forced to 1 because Bluesky throttles that endpoint per IP and parallel workers only drain the same budget faster.

## `proxyConfiguration` (type: `object`):

Bluesky's public API has no WAF, so most modes need no proxy at all. "search-posts" is the exception: it is rate-limited per IP, and a proxy lets the actor rotate to a fresh IP instead of waiting out the cooldown.

## Actor input object example

```json
{
  "mode": "posts",
  "actors": [
    "bsky.app"
  ],
  "maxItems": 100,
  "postFilter": "posts_with_replies",
  "includePins": false,
  "threadDepth": 6,
  "parentHeight": 0,
  "sort": "latest",
  "maxConcurrency": 4,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Every profile / post / reply / follower / like / repost / quote / list / starter pack / trend / feed record produced by this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "posts",
    "actors": [
        "bsky.app"
    ],
    "maxItems": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("fanndev/bluesky-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "posts",
    "actors": ["bsky.app"],
    "maxItems": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("fanndev/bluesky-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "posts",
  "actors": [
    "bsky.app"
  ],
  "maxItems": 100
}' |
apify call fanndev/bluesky-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fanndev/bluesky-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/q9xNzaUEtKrWQ6cVQ/builds/rzafef8wmgGLjAhWm/openapi.json