🦋 All-In-One Bluesky Scraper

Extract detailed data from the Bluesky social network. Scrape posts by keyword, user profiles, author feeds, followers, and follows lists effortlessly. This scraper is fast, reliable, and designed to deliver clean, structured data for your analysis, research, or marketing needs.

📜 What does the Bluesky Scraper do?

This actor is a powerful and flexible tool that allows you to extract public data from Bluesky at scale. Simply provide a keyword or user handle, choose what you want to scrape, and let the actor handle the rest. It authenticates with the Bluesky API, fetches the data you need, and saves it in a structured format.

This tool is perfect for data analysts, marketers, brand managers, and researchers looking to tap into the conversations and communities on the rapidly growing Bluesky platform.

(Consider adding a relevant screenshot here)

✨ What Bluesky data can I scrape?

With this Bluesky API, you can extract the following data:

• 📝 Posts from Search: Find posts based on any keyword.
• 👤 User Posts: Get the entire feed of a specific user.
• 🤝 Followers & Follows: Extract complete lists of a user's followers or the accounts they follow.
• 📄 Profile Details: Scrape detailed information from user profiles.

Each piece of data is enriched with key information, including:

• Handle and Display Name
• Post Text or Profile Bio
• Post/Profile URL
• Engagement Counts (Likes, Reposts, Replies)
• Follower/Following Counts
• Timestamps

🤺 Why scrape Bluesky?

• 📈 Market Research: Analyze trends and conversations in your niche.
• ❤️ Brand Monitoring & Sentiment Analysis: Track mentions of your brand, products, or competitors.
• 🤝 Influencer Discovery: Identify key opinion leaders and potential partners in your industry.
• 🔎 Social Media Analysis: Gather data for academic research or to understand community dynamics.
• 🗞️ Content Aggregation: Collect relevant posts for news feeds or content curation.

---

⚙️ How to Use the Bluesky Scraper

This scraper is designed to be easy to use. Here’s how to get started:

1. Create a Bluesky App Password (see detailed instructions below). This is a required step.
2. Add your credentials to the Apify Console as secret environment variables.
3. Open the Bluesky Scraper actor on Apify.
4. Choose the action you want to perform (e.g., "Search for Posts").
5. Enter your keywords or user handles (e.g., web scraping or apify.com).
6. Click "Start" and wait for the data to be extracted.
7. Download your data in JSON, CSV, Excel, or HTML format.

---

⚠️ IMPORTANT: Authentication (Required)

Bluesky's API requires you to be logged in. To do this securely, you must create and use an App Password. This is safer than using your main password.

How to create and add your Bluesky credentials:

1. Log in to your Bluesky account at bsky.app.
2. Go to Settings in the left-hand menu.
3. Scroll down and click on App Passwords under the "Advanced" section.
4. Click Add App Password. Give it a memorable name (e.g., ApifyScraper) and click Create.
5. Bluesky will generate a password for you (e.g., xxxx-xxxx-xxxx-xxxx). Copy this password immediately. You won't be able to see it again.
6. In your Apify account, go to the Actor's "Settings" tab and find the "Environment variables" section.
7. Create a variable named BLUESKY_HANDLE, enter your Bluesky handle (e.g., yourname.bsky.social), and check the "Secret" box.
8. Create another variable named BLUESKY_APP_PASSWORD, paste the App Password you copied, and check the "Secret" box.

(Consider adding a screenshot of the App Password creation screen)

Your credentials are now securely stored and ready for the actor to use.

---

📥 Input Configuration

The input for the Bluesky Scraper specifies what you want to scrape. You can provide these settings in the user-friendly interface or as a JSON object.

Field	Description
actionToPerform	The type of data you want to scrape. Options include scraping posts from a search, a user's feed, followers, follows, or a user's profile.
queries	A list of keywords (for searchPosts) or user handles (for all other actions).
maxItems	The maximum number of results you want to scrape in total. Leave it empty to get all available results.
sortOrder	For searchPosts only. Sort results by latest or top (most popular).

Input Example

Here is an example of scraping the top 200 most popular posts for the keywords "data science" and "AI":

{
  "actionToPerform": "searchPosts",
  "queries": [
    "data science",
    "AI"
  ],
  "maxItems": 200,
  "sortOrder": "top"
}

---

📤 Output Examples

The results will be stored in the actor's dataset. The structure of each item depends on the action performed.

Post Result (searchPosts or getAuthorFeed)

[{
  "kind": "post",
  "query": "web scraping",
  "id": "3kymu4f5k2g2q",
  "authorHandle": "apify.com",
  "authorName": "Apify",
  "text": "Web scraping is the process of automatically extracting data from websites. It's a powerful tool for data analysis, market research, and lead generation. #webscraping #data",
  "createdAt": "2024-05-20T14:30:00.000Z",
  "replyCount": 15,
  "repostCount": 45,
  "likeCount": 120,
  "url": "https://bsky.app/profile/apify.com/post/3kymu4f5k2g2q"
}]

Profile Result (getProfile)

[{
  "kind": "profile",
  "query": "apify.com",
  "id": "did:plc:f2i2f5g3g7g4g8h9h0j",
  "authorHandle": "apify.com",
  "authorName": "Apify",
  "text": "The one-stop shop for all your web scraping and automation needs. Turn any website into an API.",
  "followersCount": 5200,
  "followsCount": 150,
  "postsCount": 850,
  "url": "https://bsky.app/profile/apify.com"
}]

Follower Result (getFollowers or getFollows)

[{
  "kind": "follower",
  "query": "apify.com",
  "id": "did:plc:k1l2m3n4o5p6q7r8s",
  "authorHandle": "janedoe.bsky.social",
  "authorName": "Jane Doe",
  "text": "Data Scientist | Python Enthusiast | Developer",
  "url": "https://bsky.app/profile/janedoe.bsky.social"
}]

---

💰 How much will it cost?

This actor is priced at $1.80 per 1,000 results.

The Apify Free plan provides you with $5 in free usage credits every month, which means you can scrape approximately 2,700 results for free.

For larger scraping needs, a personal plan (e.g., $49/month) would allow you to scrape over 27,000 results monthly. It's best to run a small test for your specific use case to get a precise cost estimate.

⚖️ Is it legal to scrape Bluesky?

This scraper is designed to be ethical and only extracts public data that is visible to any logged-in user. It does not scrape private information. However, you should be aware that the results may contain personal data. It is your responsibility to use this data in compliance with applicable laws like GDPR and CCPA. If you are unsure, please consult with a legal professional. For more information, read our blog post on the legality of web scraping.

🔗 Integrations and API Usage

You can connect this actor with almost any cloud service or web app through Apify's integrations. Integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more. You can also use webhooks to trigger actions or get notifications when a run is complete.

Programmatically access the actor using the Apify API. You can manage runs, access datasets, and get results using our NPM or PyPI client libraries. Check out the API tab for code examples.

💬 Your Feedback

We are always working to improve our actors. If you find a bug, have a feature request, or have any technical feedback, please create an issue on the Actor’s Issues tab in the Apify Console.