Circle Community Scraper

Extract publicly visible metadata from Circle.so community landing pages — community name, description, member count, pricing tiers, tags, and creator info.

How it works

Circle communities are hosted at <slug>.circle.so or on custom domains. The public landing page renders community metadata server-side in a mix of OG/meta tags, __NEXT_DATA__ JSON (where present), and visible HTML.

The actor:

1. Normalises each input URL.
2. Fetches the landing page using a full Chrome browser fingerprint (User-Agent + Sec-Fetch-* headers).
3. Detects Cloudflare challenge pages and retries up to 3× with a fresh proxy IP.
4. Parses community data from __NEXT_DATA__, OG tags, JSON-LD, and visible HTML — in that priority order.

Cloudflare bypass

Circle community subdomains are served behind Cloudflare bot management. Plain datacenter IPs will hit a Turnstile/JS-challenge page. Apify Residential proxies are required for reliable scraping. The actor detects CF challenge pages and rotates to a fresh residential IP on each retry.

Input

Field	Type	Default	Description
communityUrls	stringList	—	Public Circle community URLs (required)
maxConcurrency	integer	3	Parallel fetches
proxyConfiguration	proxy	Residential	Proxy — residential strongly recommended

Accepted URL formats:

• https://myslug.circle.so — Circle-hosted subdomain
• https://community.mysite.com — Custom domain pointing to Circle

The community must have a public-facing landing page. Private communities (login-only) will return a 403 or redirect to a login page.

Output schema (per community)

{
  "url": "https://ship30for30.circle.so",
  "slug": "ship30for30",
  "name": "Ship 30 for 30",
  "description": "A community for writers who want to publish every day for 30 days.",
  "cover_image_url": "https://...",
  "member_count": 4200,
  "pricing": [
    {
      "raw_price_text": "$29/month",
      "price_usd": 29.0,
      "interval": "month",
      "name": null
    },
    {
      "raw_price_text": "$199/year",
      "price_usd": 199.0,
      "interval": "year",
      "name": null
    }
  ],
  "is_free": false,
  "tags": ["Writing", "Accountability"],
  "creator_name": "Dickie Bush",
  "creator_url": null,
  "og_title": "Ship 30 for 30",
  "og_description": "...",
  "og_image": "https://..."
}

Communities that fail after all retries are still pushed with an "error" field:

{ "url": "https://private.circle.so", "error": "http_403" }
{ "url": "https://blocked.circle.so", "error": "max_retries_exceeded" }

Notes

• No explore/directory page: Circle.so does not provide a public community directory. You must supply individual community URLs. Known communities can be found via web search (site:*.circle.so).
• Private communities: Most Circle content is private. Only communities with a public landing page are scrapeable. If a community requires login to view any content, it will return error: "http_403".
• Cloudflare protection: Circle's CDN applies bot scoring to community subdomains. Apify Residential proxies bypass this; datacenter proxies may not.
• Pricing detection: Pricing is extracted from visible HTML via regex patterns. Circle does not consistently embed pricing in machine-readable form, so detection is best-effort. Always verify against the actual page.
• Member count: Available only if the community displays it publicly on the landing page.