# Bluesky Scraper (AT Protocol) (`variable_nose_u5u/bluesky-scraper`) Actor Search and scrape posts and profiles from Bluesky (bsky.social) via the public AT Protocol API. No authentication required. Supports full-text post search, profile search, author feeds, and single-profile lookup with date/language/image filtering. Pay-per-result pricing. - **URL**: https://apify.com/variable\_nose\_u5u/bluesky-scraper.md - **Developed by:** [Cyril R](https://apify.com/variable_nose_u5u) (community) - **Categories:** Developer tools, Social media, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Bluesky Scraper (AT Protocol) Search and scrape **posts** and **profiles** from [Bluesky](https://bsky.social) via the public AT Protocol API. No authentication, no API key, no rate-limit headaches — the `bsky.social` public AppView serves search and read endpoints for free. Built as an Apify Actor with **pay-per-result** pricing: you pay a tiny flat fee per post or profile returned in the dataset, and nothing for empty results. --- ### Features | Mode | Endpoint | What it does | |------|----------|--------------| | `posts` | `app.bsky.feed.searchPosts` | Full-text search of public posts | | `profiles` | `app.bsky.actor.searchActors` | Search actor/profile by name/handle | | `search` | both above | Run posts + profiles searches together | | `authorFeed` | `app.bsky.feed.getAuthorFeed` | Fetch recent posts from one or more authors | | `profile` | `app.bsky.actor.getProfile` | Fetch a single profile by handle or DID | Plus filters for **date range** (`since`/`until`), **language** (`lang`), **images/video only**, **author DID**, **linked domain**, and **sort order** (top vs. latest). --- ### Input The actor reads its input from the standard Apify input (`.actor/input_schema.json`). Key fields: ```jsonc { "mode": "posts", // posts | profiles | search | authorFeed | profile "queries": ["apify", "scraping"], // for posts/profiles/search "authors": ["apify.bsky.social"], // for authorFeed/profile (handle or did:plc:...) "maxResults": 100, // per query/author "since": "2024-01-01", // ISO 8601, posts/search only "until": "", "lang": "en", "hasImages": false, "hasVideo": false, "authorFilter": "", // did:plc:... to restrict posts to one author "domainFilter": "", // only posts linking to this domain "sort": "top" // top | latest } ```` See `.actor/input_schema.json` for the full, validated schema. *** ### Output Each row in the dataset is a flattened Bluesky post or profile with a consistent schema. Posts and profiles share the same column set (profile-only fields are empty on post rows, and vice-versa) so you can mix them in one table. Selected fields: - `type` — `post` | `profile` | `feedPost` - `query` / `authorHandle` — what produced this row - `uri`, `cid` — AT Protocol identifiers - `authorDid`, `authorHandle`, `authorDisplayName`, `authorAvatar` - `text` — post text / profile description - `indexedAt`, `createdAt` — timestamps - `likeCount`, `repostCount`, `replyCount`, `quoteCount`, `bookmarkCount` - `langs`, `hasImages`, `hasVideo`, `imageCount`, `imageAlts`, `imageUrls` - `urls` — links extracted from post facets/embeds - `replyParentUri`, `replyRootUri` — for threading - `labels` — content labels - `followersCount`, `followsCount`, `postsCount`, `viewer`, `joinedAt` — profile-only *** ### Pricing This actor uses Apify's **pay-per-event** model. A single event type, `result`, is charged once per item pushed to the dataset. You only pay for data you actually receive. The `events` block in `.actor/actor.json` declares the `result` event; the SDK's `Actor.push_data(rows, 'result')` handles both charging and dataset writes, and automatically stops charging once the user's budget is exhausted. *** ### Local development #### Prerequisites - Python 3.13+ - [`uv`](https://github.com/astral-sh/uv) or `pip` + `venv` #### Install ```bash cd actors/bluesky-scraper uv venv && source .venv/bin/activate uv pip install -r requirements.txt ``` #### Run locally with Apify SDK The Apify Python SDK reads input from a local storage. Set `ACTOR_TEST_PAY_PER_EVENT=true` to exercise the PPE charging path locally (events default to $1 each in local mode): ```bash export ACTOR_TEST_PAY_PER_EVENT=true apify run --input '{"mode":"posts","queries":["AI"],"maxResults":5}' ``` #### Run the scraper logic standalone (no Apify platform) For a quick smoke test of just the Bluesky API calls, you can import the client and shape helpers directly: ```python import asyncio, httpx from src.main import BlueskyClient, shape_post async def smoke(): async with httpx.AsyncClient() as http: c = BlueskyClient(http) data = await c.search_posts("AI", limit=3) for p in data["posts"]: row = shape_post(p, query="AI") print(row["authorHandle"], "->", row["text"][:60]) asyncio.run(smoke()) ``` *** ### Deployment ```bash cd actors/bluesky-scraper apify push ``` After publishing, enable **pay-per-event** pricing in the Apify Console and set the price per `result` event. The actor already declares the event in `actor.json`, so the Console will detect it automatically. *** ### API reference - [AT Protocol searchPosts](https://docs.bsky.app/docs/api/app.bsky.feed.searchPosts) - [AT Protocol searchActors](https://docs.bsky.app/docs/api/app.bsky.actor.searchActors) - [AT Protocol getAuthorFeed](https://docs.bsky.app/docs/api/app.bsky.feed.getAuthorFeed) - [AT Protocol getProfile](https://docs.bsky.app/docs/api/app.bsky.actor.getProfile) - [Apify Python SDK — pay-per-event](https://docs.apify.com/sdk/python/docs/concepts/pay-per-event) *** ### Notes & limitations - The Bluesky public API does **not** require authentication for any endpoint used here. - `searchPosts` paginates with a `cursor`; the actor follows cursors up to `maxResults`. - Rate limits are generous on the public AppView but not infinite; the actor uses a polite `User-Agent` and a 30 s timeout. For heavy loads, configure an Apify proxy in the input. - The `extendOutput` input field is reserved for future use and currently ignored. # Actor input Schema ## `mode` (type: `string`): What to search for. 'posts' = full-text search of posts; 'profiles' = search actors/profiles; 'search' = run BOTH posts and profiles searches for the given queries; 'authorFeed' = fetch recent posts for one or more authors (use the 'authors' field); 'profile' = fetch a single profile by handle/DID (use the 'authors' field, first entry). ## `queries` (type: `array`): List of search terms (for 'posts', 'profiles', 'search' modes). Each query is run independently and results are merged. Leave empty when using 'authorFeed' or 'profile' modes. ## `authors` (type: `array`): List of author handles (e.g. 'apify.bsky.social') or DIDs (did:plc:...). Required for 'authorFeed' and 'profile' modes; ignored otherwise. ## `maxResults` (type: `integer`): Maximum number of results to fetch per query/author. Total results = queries × maxResults. Keep modest to control runtime and cost. ## `since` (type: `string`): Only return posts indexed after this date (ISO 8601, e.g. '2024-01-01' or '2024-01-01T00:00:00Z'). Applies to 'posts' and 'search' modes. Bluesky supports 'since' on searchPosts. ## `until` (type: `string`): Only return posts indexed before this date (ISO 8601). Applies to 'posts' and 'search' modes. Bluesky supports 'until' on searchPosts. ## `lang` (type: `string`): Restrict posts to a language code (e.g. 'en', 'fr', 'ja'). Optional. Only applies to 'posts' and 'search' modes. ## `hasImages` (type: `boolean`): If true, only return posts that contain at least one image. Applies to 'posts' and 'search' modes. ## `hasVideo` (type: `boolean`): If true, only return posts that contain at least one video. Applies to 'posts' and 'search' modes. ## `authorFilter` (type: `string`): Only return posts from this author DID (did:plc:...). Optional. Applies to 'posts' and 'search' modes. ## `domainFilter` (type: `string`): Only return posts linking to this domain (e.g. 'example.com'). Optional. Applies to 'posts' and 'search' modes. ## `sort` (type: `string`): Sort order for post search. 'top' = by relevance/popularity, 'latest' = by recency. Applies to 'posts' and 'search' modes. ## `extendOutput` (type: `array`): Optional list of JavaScript-free data transformations. Reserved for future use; currently ignored. ## `proxyConfig` (type: `object`): Optional Apify proxy. The Bluesky public API does not require a proxy, but one can be used to avoid rate limits. ## Actor input object example ```json { "mode": "posts", "queries": [ "bluesky" ], "authors": [], "maxResults": 100, "since": "", "until": "", "lang": "", "hasImages": false, "hasVideo": false, "authorFilter": "", "domainFilter": "", "sort": "top", "extendOutput": [], "proxyConfig": {} } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("variable_nose_u5u/bluesky-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("variable_nose_u5u/bluesky-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call variable_nose_u5u/bluesky-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=variable_nose_u5u/bluesky-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Bluesky Scraper (AT Protocol)", "description": "Search and scrape posts and profiles from Bluesky (bsky.social) via the public AT Protocol API. No authentication required. Supports full-text post search, profile search, author feeds, and single-profile lookup with date/language/image filtering. Pay-per-result pricing.", "version": "0.1", "x-build-id": "SW2Gh6KPRcnGnAbpK" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/variable_nose_u5u~bluesky-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-variable_nose_u5u-bluesky-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/variable_nose_u5u~bluesky-scraper/runs": { "post": { "operationId": "runs-sync-variable_nose_u5u-bluesky-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/variable_nose_u5u~bluesky-scraper/run-sync": { "post": { "operationId": "run-sync-variable_nose_u5u-bluesky-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "mode" ], "properties": { "mode": { "title": "Mode", "enum": [ "posts", "profiles", "search", "authorFeed", "profile" ], "type": "string", "description": "What to search for. 'posts' = full-text search of posts; 'profiles' = search actors/profiles; 'search' = run BOTH posts and profiles searches for the given queries; 'authorFeed' = fetch recent posts for one or more authors (use the 'authors' field); 'profile' = fetch a single profile by handle/DID (use the 'authors' field, first entry).", "default": "posts" }, "queries": { "title": "Search queries", "type": "array", "description": "List of search terms (for 'posts', 'profiles', 'search' modes). Each query is run independently and results are merged. Leave empty when using 'authorFeed' or 'profile' modes.", "items": { "type": "string" }, "default": [ "bluesky" ] }, "authors": { "title": "Authors (handles or DIDs)", "type": "array", "description": "List of author handles (e.g. 'apify.bsky.social') or DIDs (did:plc:...). Required for 'authorFeed' and 'profile' modes; ignored otherwise.", "items": { "type": "string" }, "default": [] }, "maxResults": { "title": "Max results per query", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of results to fetch per query/author. Total results = queries × maxResults. Keep modest to control runtime and cost.", "default": 100 }, "since": { "title": "Since (start date)", "type": "string", "description": "Only return posts indexed after this date (ISO 8601, e.g. '2024-01-01' or '2024-01-01T00:00:00Z'). Applies to 'posts' and 'search' modes. Bluesky supports 'since' on searchPosts.", "default": "" }, "until": { "title": "Until (end date)", "type": "string", "description": "Only return posts indexed before this date (ISO 8601). Applies to 'posts' and 'search' modes. Bluesky supports 'until' on searchPosts.", "default": "" }, "lang": { "title": "Language code", "type": "string", "description": "Restrict posts to a language code (e.g. 'en', 'fr', 'ja'). Optional. Only applies to 'posts' and 'search' modes.", "default": "" }, "hasImages": { "title": "Only posts with images", "type": "boolean", "description": "If true, only return posts that contain at least one image. Applies to 'posts' and 'search' modes.", "default": false }, "hasVideo": { "title": "Only posts with video", "type": "boolean", "description": "If true, only return posts that contain at least one video. Applies to 'posts' and 'search' modes.", "default": false }, "authorFilter": { "title": "Author DID filter", "type": "string", "description": "Only return posts from this author DID (did:plc:...). Optional. Applies to 'posts' and 'search' modes.", "default": "" }, "domainFilter": { "title": "Domain filter", "type": "string", "description": "Only return posts linking to this domain (e.g. 'example.com'). Optional. Applies to 'posts' and 'search' modes.", "default": "" }, "sort": { "title": "Sort order", "enum": [ "top", "latest" ], "type": "string", "description": "Sort order for post search. 'top' = by relevance/popularity, 'latest' = by recency. Applies to 'posts' and 'search' modes.", "default": "top" }, "extendOutput": { "title": "Extend output functions", "type": "array", "description": "Optional list of JavaScript-free data transformations. Reserved for future use; currently ignored.", "items": { "type": "string" }, "default": [] }, "proxyConfig": { "title": "Proxy configuration", "type": "object", "description": "Optional Apify proxy. The Bluesky public API does not require a proxy, but one can be used to avoid rate limits.", "default": {} } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```