Twitter Communities Top Posts Scraper

Extract public posts from X/Twitter Communities by community URL or ID. Use it to monitor ranked community discussions, collect post engagement, identify active authors, and export community-scoped social intelligence from Apify.

What does Twitter Communities Top Posts Scraper do?

Twitter Communities Top Posts Scraper collects public post rows from X/Twitter Community pages.

It returns post text, author details, timestamps, engagement counts, media URLs, outbound links, hashtags, and the source community for every saved item.

The actor focuses on community-scoped feeds instead of broad keyword search. Give it one or more Community URLs and it extracts posts from those communities.

Who is it for?

• 🧭 Social listening teams tracking niche X communities
• 📈 Growth marketers looking for community topic ideas
• 🧑‍💼 Community managers monitoring active authors and conversations
• 🧪 Researchers measuring engagement in specialized public groups
• 🧲 Lead generation teams finding people discussing a domain-specific problem

Why use it?

X search and trend scraping are broad. Communities are different because the audience and context are pre-filtered by a group.

Use this actor when you already know the communities you care about and need structured post data from those exact spaces.

Data you can extract

Field group	Examples
Community	community ID, URL, name, description, sort mode
Post	post ID, URL, text, language, creation time
Author	username, display name, profile URL, avatar, verification flag
Engagement	replies, reposts, likes, quotes, bookmarks, views, total engagement
Content enrichments	hashtags, cashtags, media URLs, outbound links
Run metadata	scraped timestamp

How much does it cost to scrape Twitter Community posts?

The actor uses pay-per-event pricing.

• A small start event is charged once per run.
• A per-item event is charged for each community post saved.
• Tier discounts reduce the per-post price for higher-volume users.

Use a low maxItems value for your first run, then increase it once you confirm the community produces the data you need.

Input

The main input is communityUrls.

{
  "communityUrls": [
    { "url": "https://x.com/i/communities/1493446837214187523" }
  ],
  "maxItems": 20,
  "sortMode": "top"
}

You can also provide numeric communityIds if your workflow stores IDs already.

Output

Each dataset row is one post from a public X/Twitter Community.

{
  "communityId": "1493446837214187523",
  "communityUrl": "https://x.com/i/communities/1493446837214187523",
  "communityName": "Build in Public",
  "sortMode": "top",
  "postId": "1234567890",
  "postUrl": "https://x.com/example/status/1234567890",
  "text": "Example community post text",
  "authorUsername": "example",
  "likeCount": 42,
  "repostCount": 5,
  "replyCount": 3,
  "urls": [],
  "mediaUrls": [],
  "scrapedAt": "2026-06-20T22:00:00.000Z"
}

How to scrape top posts from an X Community

1. Open a public Community on X/Twitter.
2. Copy the URL from your browser.
3. Paste it into communityUrls.
4. Set maxItems to a small number for a test run.
5. Choose top for ranked posts or latest for the logged-out latest feed.
6. Start the actor.
7. Export the dataset as JSON, CSV, Excel, or through the Apify API.

Sort modes

top uses X's ranked logged-out community timeline and is best for popular or high-engagement posts.

latest uses the public logged-out community timeline when X exposes it. Some communities may show limited content in latest mode to logged-out users.

Tips for best results

• Start with 10-20 posts to verify the community is public and active.
• Use top for market research and lead discovery.
• Use latest when you need recency monitoring.
• Increase requestDelayMs if X starts returning transient errors.
• Use proxy settings only when your network is blocked.

Integrations

Use the actor in workflows such as:

• Send new top community posts to Slack for a social listening digest.
• Export posts to Google Sheets for manual qualification.
• Feed author usernames into a CRM enrichment workflow.
• Store community posts in a data warehouse for trend analysis.
• Trigger alerts when posts contain competitor names or product keywords.

API usage with Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/twitter-communities-top-posts-scraper').call({
  communityUrls: [{ url: 'https://x.com/i/communities/1493446837214187523' }],
  maxItems: 20,
  sortMode: 'top',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API usage with Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_APIFY_TOKEN')
run = client.actor('automation-lab/twitter-communities-top-posts-scraper').call(run_input={
    'communityUrls': [{'url': 'https://x.com/i/communities/1493446837214187523'}],
    'maxItems': 20,
    'sortMode': 'top',
})

for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)

API usage with cURL

curl -X POST "https://api.apify.com/v2/acts/automation-lab~twitter-communities-top-posts-scraper/runs?token=$APIFY_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"communityUrls":[{"url":"https://x.com/i/communities/1493446837214187523"}],"maxItems":20,"sortMode":"top"}'

MCP integration

Use the actor from Claude Desktop, Claude Code, or other MCP clients through Apify MCP Server.

MCP server URL:

https://mcp.apify.com/?tools=automation-lab/twitter-communities-top-posts-scraper

Claude Code setup:

$claude mcp add apify-twitter-communities "https://mcp.apify.com/?tools=automation-lab/twitter-communities-top-posts-scraper"

Claude Desktop JSON config:

{
  "mcpServers": {
    "apify-twitter-communities": {
      "url": "https://mcp.apify.com/?tools=automation-lab/twitter-communities-top-posts-scraper"
    }
  }
}

Example prompts:

• "Scrape the top 20 posts from this X Community and summarize repeated pain points."
• "Find authors with high engagement in this Twitter Community."
• "Export media links from recent posts in this Community."

Proxy settings

The actor works without a proxy for public guest endpoints in normal conditions.

If X blocks direct access from your environment, enable Apify Proxy and start with a small run. Datacenter proxies should be tried before residential proxies to control cost.

Limitations

• Private or members-only communities may not expose full content to logged-out users.
• X can change web GraphQL operation IDs or guest-token behavior.
• Deleted, tombstoned, or unavailable posts are skipped.
• Engagement counts reflect what X returns at scrape time.

FAQ

Why did my run return fewer posts than maxItems?

The public logged-out feed may expose fewer posts than requested, the community may be inactive, or X may stop pagination early.

Why do I see an unavailable-community error?

Check that the URL is a public X/Twitter Community URL and that the numeric ID is correct.

Should I use residential proxy?

Only after direct access and cheaper datacenter proxy access fail. Residential proxy traffic is much more expensive.

Related scrapers

• Twitter/X Scraper for broad tweet, profile, and search extraction.
• Twitter/X Trends Scraper for trend monitoring.
• Twitter Followers Scraper for follower/following lists.

Legality

This actor extracts public web data returned by X/Twitter to logged-out web clients. You are responsible for using the data lawfully, respecting privacy, following X's terms, and complying with applicable laws in your jurisdiction.

Changelog

Initial version extracts public ranked/latest community post rows with author, engagement, media, link, hashtag, and community context fields.

Support

If a public community fails, include the community URL, input JSON, run ID, and a short description of what you expected to extract.

Example workflow: social listening

1. Maintain a list of important X Communities.
2. Run this actor every morning with sortMode set to top.
3. Filter posts with high engagement.
4. Send promising posts to a researcher or salesperson.
5. Track repeated keywords over time.

Example workflow: content research

1. Scrape top posts from niche communities.
2. Group posts by hashtags and outbound links.
3. Review highly liked posts for topic ideas.
4. Turn repeated questions into blog posts, videos, or product documentation.

Example workflow: author discovery

1. Collect top community posts.
2. Sort by engagement count.
3. Deduplicate by authorUsername.
4. Review author profiles manually or with another enrichment actor.
5. Save qualified accounts for outreach.