Bluesky Feed Scraper for Apify

This is an Apify actor that scrapes data from a specified Bluesky feed URL and outputs detailed information about the posts, including metadata, authors, embedded media, and statistics such as likes, replies, and reposts.

Features

• Scrapes Bluesky feed posts from a given feed URL.
• Extracts detailed post data, including:
  • Author details (DID, handle, display name, avatar URL, etc.).
  • Post text, tags, and languages.
  • Embedded images, with metadata (alt text, aspect ratio, URLs).
  • Engagement statistics (likes, replies, reposts, quotes).
  • Thread and reply information.
  • Record metadata, including creation and indexing timestamps.

Input

The actor requires the following input:

Field	Type	Description
url	String	The URL of the Bluesky feed you want to scrape. Example: https://bsky.app/profile/username/feed.

Example Input

{
  "url": "https://bsky.app/profile/c3rmen.bsky.social/feed"
}

Output

The actor produces a JSON array where each object represents a post from the feed. The structure includes:

• uri and cid: Unique identifiers for the post.
• author: Details about the author (DID, handle, avatar, etc.).
• record: Post text, tags, languages, and embedded media.
• embed: View-ready image metadata (e.g., thumbnails, full-size URLs).
• Engagement metrics (replyCount, repostCount, likeCount, quoteCount).
• Thread and reply-related data.
• Timestamps (createdAt, indexedAt).

Example Output

[
  {
    "uri": "at://did:plc:z72i7hdynmk6r22z27h6tvur/app.bsky.feed.post/3lbsizxfxa22r",
    "cid": "bafyreifohcetdw6e5mudaz6anigzsm5ssjpm3oreyxu4a2l665k7hpxo4q",
    "author": {
      "did": "did:plc:z72i7hdynmk6r22z27h6tvur",
      "handle": "bsky.app",
      "displayName": "Bluesky",
      "avatar": "https://cdn.bsky.app/img/avatar/plain/did:plc:z72i7hdynmk6r22z27h6tvur/bafkreihagr2cmvl2jt4mgx3sppwe2it3fwolkrbtjrhcnwjk4jdijhsoze@jpeg",
      "associated": {
        "chat": {
          "allowIncoming": "none"
        }
      },
      "labels": [],
      "createdAt": "2023-04-12T04:53:57.057Z"
    },
    "record": {
      "createdAt": "2024-11-25T21:52:30.840Z",
      "embed": {
        "external": {
          "description": "Bluesky is social media as it should be. Find your community among millions of users, unleash your creativity, and have some fun again. https://bsky.app",
          "thumb": {
            "ref": {
              "$link": "bafkreihh7dthuxfqel6zwcmxapcu47tr34rat7thjtxlfmrwidvxfsmqne"
            },
            "mimeType": "image/jpeg",
            "size": 384236,
            "$type": "blob"
          },
          "title": "BlueskySocial - Twitch",
          "uri": "https://www.twitch.tv/blueskysocial"
        },
        "$type": "app.bsky.embed.external"
      },
      "facets": [
        {
          "features": [
            {
              "did": "did:plc:qjeavhlw222ppsre4rscd3n2",
              "$type": "app.bsky.richtext.facet#mention"
            }
          ],
          "index": {
            "byteEnd": 55,
            "byteStart": 40
          },
          "$type": "app.bsky.richtext.facet"
        },
        {
          "features": [
            {
              "did": "did:plc:ragtjsm2j2vknwkz3zp4oxrd",
              "$type": "app.bsky.richtext.facet#mention"
            }
          ],
          "index": {
            "byteEnd": 76,
            "byteStart": 64
          },
          "$type": "app.bsky.richtext.facet"
        },
        {
          "features": [
            {
              "did": "did:plc:4ewnpnebeh7zuk5pbardaxqz",
              "$type": "app.bsky.richtext.facet#mention"
            }
          ],
          "index": {
            "byteEnd": 226,
            "byteStart": 203
          },
          "$type": "app.bsky.richtext.facet"
        }
      ],
      "langs": [
        "en"
      ],
      "text": "Join us for another livestream with COO @rose.bsky.team and CTO @pfrazee.com, where they'll share team updates, the story of how Bluesky began, and what’s next. \n\nPlus, a special guest appearance from @flavorflav.bsky.social! 🎉\n\nToday 11/25 @ 5 pm PT / 8 pm ET / 1 am GMT / 10am JST",
      "$type": "app.bsky.feed.post"
    },
    "embed": {
      "external": {
        "uri": "https://www.twitch.tv/blueskysocial",
        "title": "BlueskySocial - Twitch",
        "description": "Bluesky is social media as it should be. Find your community among millions of users, unleash your creativity, and have some fun again. https://bsky.app",
        "thumb": "https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:z72i7hdynmk6r22z27h6tvur/bafkreihh7dthuxfqel6zwcmxapcu47tr34rat7thjtxlfmrwidvxfsmqne@jpeg"
      },
      "$type": "app.bsky.embed.external#view"
    },
    "replyCount": 324,
    "repostCount": 1041,
    "likeCount": 9147,
    "quoteCount": 84,
    "indexedAt": "2024-11-25T21:52:35.058Z",
    "labels": []
  },
  // ...more posts
]

Usage

1. Deploy the Actor: Use the Apify console to set up and deploy this actor.
2. Provide Input: Supply the url in the input configuration.
3. Run the Actor: Start the actor, and it will scrape the feed URL and return the posts as JSON.

Notes

• Ensure the url is publicly accessible.
• The actor fetches only visible posts; private or restricted feeds will not be included.

Feel free to suggest additional features or report any issues! 🚀