X Popular Communities Scraper

Browse popular X Communities for a chosen topic—the same kind of spaces people discover when exploring Communities by category. Each dataset row is one Community with nested details such as name, description, rules, member counts, banners, themes, facepile members, and admin or creator profiles when returned.

Who it’s for

• Community managers & moderators researching active spaces in a theme.
• Marketers & analysts mapping topic ecosystems on X.
• Researchers who want structured exports instead of manual browsing.

What you can do with it

• Pick a topic id that matches a Community category (for example technology or hobby topics exposed in the product).
• Collect many Communities in one run until you reach the limit you set or there are no more results.
• Match every row to the topic you asked for via requestedTopicId, plus scrapedAt on each row.

How it works (in plain terms)

You enter the topic id (the same identifier used when X groups Communities by topic). The tool loads results in successive batches until your maximum row count is reached or the list ends—you do not need to manage paging yourself.

Input

Field	Required	What it means
Topic ID (topicId)	Yes	Which topic’s popular Communities to list.
Maximum results (maxResults)	No	Upper limit on how many community rows to collect (default 50).

Output

• One row per Community (until your cap). Rows are often large (hundreds of lines of JSON when fully expanded).
• Identity & basics: __typename (Community), name, description, access, numeric rest_id, created_at (epoch-style when returned), question, role (e.g. viewer vs member), is_nsfw, is_pinned.
• Scale & activity: member_count, moderator_count, new_tweet_count_since_last_viewed.
• Topic tag: primary_community_topic with topic_id and topic_name.
• Policies & invites: join_policy, join_requests_result, invites_policy, invites_result.
• Presentation: custom_theme, default_theme, custom_banner_media and default_banner_media (imagery, palettes, media_key).
• Discovery helpers: search_tags, trending_hashtags_slice (often an items array, sometimes empty).
• Rules: rules entries can include name, optional description, rest_id.
• Actions: actions (CommunityActions) may expose join/leave/spotlight-style results.
• People: admin_results and creator_results embed User objects with rich legacy profiles; members_facepile_results lists sample members the same way.
• Export metadata: requestedTopicId repeats the topic you entered; scrapedAt is when that row was saved.

Optional sections vary by Community; some fields appear only in certain contexts. Details may change as X updates its product.

Sample output (one dataset row, heavily trimmed)

The snippet below uses field names and values from a real run; nested user and media blocks are omitted or shortened.

{
  "__typename": "Community",
  "access": "Public",
  "actions": {
    "__typename": "CommunityActions",
    "community_spotlight_setup_action_result": {
      "__typename": "CommunitySpotlightSetupActionShow",
      "deeplink": "twitter://profile_spotlight"
    },
    "join_action_result": { "__typename": "CommunityJoinAction" },
    "leave_action_result": {
      "__typename": "CommunityLeaveActionUnavailable",
      "message": "Not a member.",
      "reason": "ViewerNotMember"
    }
  },
  "created_at": 1670743786266,
  "custom_theme": "Red",
  "default_theme": "Plum",
  "description": "Share AI-generated creations, news, tips and more. Sponsored by @higgsfield",
  "invites_policy": "MemberInvitesAllowed",
  "is_nsfw": false,
  "join_policy": "Open",
  "member_count": 245954,
  "moderator_count": 14,
  "name": "Generative AI",
  "new_tweet_count_since_last_viewed": 0,
  "primary_community_topic": {
    "topic_id": "302",
    "topic_name": "Artificial Intelligence"
  },
  "question": "",
  "rest_id": "1601841656147345410",
  "role": "NonMember",
  "rules": [
    {
      "description": "Harassment, bullying, or toxic behavior won’t be tolerated.",
      "name": "Be kind and respectful.",
      "rest_id": "1831180873569333471"
    }
  ],
  "search_tags": ["generative", "ai", "midjourney", "chatgpt"],
  "trending_hashtags_slice": { "items": [] },
  "requestedTopicId": "302",
  "scrapedAt": "2026-03-26T01:01:07.161Z"
}

admin_results, creator_results, members_facepile_results, and full banner media trees are omitted here but appear in full dataset rows.

Good to know

• The topic id must match a topic X recognizes; unknown ids may return few or no rows.
• Respect X’s rules and applicable laws when using exported data.