X Communities Scraper

Extract public X Community metadata, member profiles, and top or latest posts for audience research and recurring community monitoring.

X Communities Scraper turns supplied Community URLs or IDs into typed records you can export as JSON, CSV, Excel, or consume through the Apify API.

It supports three record types in one run:

• Community metadata
• Public member profiles
• Community posts

What does X Communities Scraper do?

The Actor collects public information from exact X Communities that you supply.

For each Community, it can return:

• name, description, category, member count, tags, and banner;
• member usernames, display names, profile links, avatars, and verification state;
• top or latest posts with text, author, timestamp, media, and engagement counts.

Every result keeps the Community ID and canonical X URL, making repeated exports easy to compare.

Who is it for?

• Community managers tracking participation and active authors
• Audience researchers studying niche groups
• Growth teams building source-attributed prospect lists
• Content strategists reviewing popular Community discussions
• Analysts monitoring public member and post changes over time
• Developers integrating Community data into an internal workflow

Why use this Actor?

X Communities group people and posts around a shared topic.

A normal X search does not preserve that membership context.

This Actor begins from exact Community URLs, keeps the source Community on every row, and lets you select only the records needed for your job.

It does not require an X login, account cookie, or API key for public Communities.

Data you can extract

Record	Fields
Community	ID, canonical URL, name, description, category, member count, tags, banner
Member	username, display name, profile URL, avatar, bio when exposed, verification state
Post	ID, canonical URL, text, publication time, author, engagement counts, media URLs
Run context	Community name and ID, record type, scrape timestamp

Optional source fields can be absent when X does not expose them.

How much does it cost to scrape X Communities?

The Actor uses pay-per-event pricing:

• one start event per run;
• one item event for each saved Community, member, or post record.

The one-time start price is $0.0015. Per-item prices are:

Plan tier	Price per saved record
FREE	$0.000322
BRONZE	$0.00028
SILVER	$0.0002184
GOLD	$0.000168
PLATINUM	$0.000112
DIAMOND	$0.0000784

At BRONZE pricing, a 10-record run costs about $0.0043, a 25-record run about $0.0085, and a 100-record run about $0.0295.

The Apify Console shows your applicable tier before the run. Start with 10–30 records to verify that a Community is public and useful before increasing the limit.

Getting started

1. Open a public X Community.
2. Copy its URL, such as https://x.com/i/communities/1493446837214187523.
3. Paste the URL into X Community URLs.
4. Choose Community metadata, members, posts, or a combination.
5. Choose Top posts or Latest posts.
6. Set the maximum number of records.
7. Start the Actor.
8. Export the default dataset or connect it to your workflow.

Input parameters

Field	Type	Default	Description
communityUrls	array	Build in Public URL	Public X Community URLs
communityIds	string array	empty	Numeric Community IDs used by stored workflows
outputTypes	array	all three types	community, members, and/or posts
postSort	string	top	Ranked top feed or latest feed
maxItems	integer	100	Global record limit, from 1 to 5,000
requestDelayMs	integer	500	Delay between source requests

Provide at least one URL or ID.

Malformed URLs and unsupported record types fail clearly instead of producing misleading empty output.

Input example: posts

{
  "communityUrls": [
    { "url": "https://x.com/i/communities/1493446837214187523" }
  ],
  "outputTypes": ["posts"],
  "postSort": "top",
  "maxItems": 10
}

Input example: members and metadata

{
  "communityIds": ["1493446837214187523"],
  "outputTypes": ["community", "members"],
  "maxItems": 21
}

Output example

Each default-dataset row has a recordType discriminator.

A post row resembles:

{
  "recordType": "post",
  "communityId": "1493446837214187523",
  "communityUrl": "https://x.com/i/communities/1493446837214187523",
  "communityName": "Build in Public",
  "postId": "2085479430575599962",
  "postUrl": "https://x.com/buildinpublic/status/2085479430575599962",
  "text": "we're putting together a directory of hardware gadgets...",
  "createdAt": "2026-08-06T21:33:00.000Z",
  "authorUsername": "buildinpublic",
  "authorName": "Build in Public",
  "authorProfileUrl": "https://x.com/buildinpublic",
  "authorVerified": true,
  "replyCount": 71,
  "repostCount": 19,
  "likeCount": 311,
  "viewCount": 48511,
  "mediaUrls": ["https://pbs.twimg.com/media/sample.jpg"],
  "scrapedAt": "2026-08-19T12:00:00.000Z"
}

Counts reflect the public source at scrape time.

Record limits and ordering

maxItems is a global cap across every Community and selected record type.

When all record types are selected, the Actor divides the global limit across Communities and then across member and post records so every supplied Community and selected type receives a bounded share.

Choose only posts when the full limit should be reserved for posts.

Choose only members when building a member export.

Posts use the selected postSort feed order.

Pagination and deduplication

The Actor follows public source cursors until:

• it reaches maxItems;
• the source has no next cursor;
• a page contains no new records; or
• the bounded page limit is reached.

Members are deduplicated by username within each Community.

Posts are deduplicated by post ID within each Community.

Monitoring X Communities

Schedule this Actor daily or weekly to create repeated snapshots.

A practical workflow is:

1. keep stable Community URLs in the Task input;
2. select metadata, members, and latest posts;
3. run on an Apify schedule;
4. store rows in a database or spreadsheet;
5. compare stable IDs and scrape timestamps;
6. alert on new members, posts, or metadata changes.

The Actor returns snapshots; it does not calculate historical diffs itself.

Audience research workflow

Use member and post rows together to understand a public audience:

1. export member usernames and profile links;
2. collect recent or ranked posts;
3. group posts by author;
4. sort by likes, replies, or views;
5. review repeated needs and topics;
6. enrich selected public profiles with another workflow when lawful.

Avoid treating Community membership as consent for unsolicited contact.

Content research workflow

Use top posts to identify discussions that already resonate inside a niche Community.

Filter or group the export by:

• author username;
• publication date;
• like count;
• reply count;
• view count;
• attached media;
• Community ID.

Schedule latest-post runs when recency matters more than ranking.

API usage with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/x-communities-members-posts').call({
  communityUrls: [{ url: 'https://x.com/i/communities/1493446837214187523' }],
  outputTypes: ['community', 'members', 'posts'],
  postSort: 'latest',
  maxItems: 30,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API usage with Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_APIFY_TOKEN')
run = client.actor('automation-lab/x-communities-members-posts').call(run_input={
    'communityUrls': [{'url': 'https://x.com/i/communities/1493446837214187523'}],
    'outputTypes': ['posts'],
    'postSort': 'top',
    'maxItems': 10,
})

for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)

API usage with cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~x-communities-members-posts/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "communityIds": ["1493446837214187523"],
    "outputTypes": ["community", "members"],
    "maxItems": 21
  }'

MCP integration

Use the Actor as a tool through Apify MCP Server.

Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/x-communities-members-posts"

Claude Desktop

Add this HTTP MCP server in Claude Desktop:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/x-communities-members-posts"
    }
  }
}

Cursor

Add the same apify HTTP server URL in Cursor Settings → MCP.

VS Code

Add the same apify HTTP server URL to your VS Code MCP configuration.

Example prompts:

• "Export the public members from this X Community."
• "Collect the latest 20 posts from these two X Communities."
• "Find the highest-engagement authors in this Community post export."

Integrations

Connect the output to:

• Google Sheets for review and tagging;
• Slack for a scheduled discussion digest;
• Make or Zapier for no-code automation;
• a warehouse for snapshot comparisons;
• an LLM workflow for topic clustering;
• a CRM only for responsible, lawful enrichment.

The default dataset works with standard Apify integrations.

Reliability and failure behavior

The Actor validates source page shapes and does not accept a challenge or login shell as an empty dataset.

Transient network and server failures receive bounded retries.

If one public data surface is unavailable, the Actor tries configured public fallbacks.

If every route is unavailable, the run fails non-zero with a concise message instead of silently succeeding with zero records.

Tips

• Test one active public Community first.
• Select only the record types needed for the workflow.
• Use latest for scheduled monitoring.
• Use top for content and engagement research.
• Increase requestDelayMs after transient source limits.
• Keep maxItems small during workflow setup.
• Store IDs and scrapedAt for snapshot comparisons.

Limitations

• Private, deleted, restricted, or members-only Communities may be unavailable.
• Public member surfaces may expose fewer profile fields than full profile pages.
• A Community can have more members than its public paginated surface returns at a given time.
• Deleted or unavailable posts are not returned.
• Engagement values can change after collection.
• X and public web surfaces can change without notice.
• The Actor does not log in, join Communities, post content, or access private data.
• The Actor does not compute historical changes; schedule and compare snapshots downstream.

Responsible use and legality

This Actor extracts publicly visible web data.

You are responsible for ensuring that your collection and use comply with applicable law, privacy rules, contractual obligations, and X's terms.

Collect only what you need.

Apply appropriate retention and security controls.

Do not use the output for harassment, discrimination, spam, or invasive profiling.

FAQ

Does the Actor require X cookies or login credentials?

No. It targets public Community surfaces and does not accept account credentials.

Can it extract private Community members or posts?

No. Private or unavailable data is outside the product scope.

Why did I receive fewer rows than maxItems?

The public source may have fewer available rows, stop pagination, remove posts, or expose a bounded member slice.

Why does a run fail instead of returning zero rows?

A challenge or unexpected page cannot be distinguished safely from real empty data. Failing clearly prevents misleading exports.

Can I extract only posts?

Yes. Set outputTypes to ["posts"] so metadata and members do not consume the global limit.

Can I monitor several Communities?

Yes. Supply multiple URLs or IDs and schedule the Task. Remember that maxItems applies globally.

Are member bios always present?

No. Bios are included only when the public member surface exposes them.

Related automation-lab Actors

• Twitter/X Lists Scraper for public X List metadata, members, and posts
• X Profiles Scraper for supplied public X profile enrichment
• Twitter/X Scraper for broader X post and search workflows

Support

For reproducible support, include:

• the public Community URL;
• the exact input JSON;
• the run ID;
• the expected record type;
• whether the failure affected top posts, latest posts, members, or metadata.

Do not include private account credentials or cookies.