Skool Group Community Infos Scraper By Keyword Search

Skool Scraper — Search Communities by Keyword With Discovery Rank

Skool Scraper (full name: Skool Group Community Infos Scraper By Keyword Search) searches Skool's own public community-discovery listing (skool.com/discovery?q=) by keyword, then fetches each matching community's /about page for a full profile — name, description, member and post counts, courses, owner details, and more. Every keyword-sourced row also carries the community's exact discoveryRank and the searchResultsTotal Skool itself reports for that keyword, so you see not just which communities matched but where they rank and how big the field is. Output is structured JSON, ready to pass to an LLM, load into a spreadsheet, or feed a monitoring pipeline. Paste niche keywords, paste direct community links, or both — rows land in Output the moment each community is collected.

What is Skool Scraper?

Skool Scraper is a keyword-driven discovery tool for Skool, the community and course-hosting platform. Give it one or more niche keywords and it runs each one through Skool's public discovery search, filters the hits by member count and paid/free status, and fetches a full /about-page snapshot of every survivor — no known community links required. It also accepts direct community links the classic way, so a run can mix keyword discovery and a hand-picked link list in the same job. No Skool account or login is used anywhere in the pipeline; every page it reads is publicly visible to a logged-out visitor.

• 🔍 Keyword discovery — searches skool.com/discovery?q=<keyword> and paginates through Skool's own ranked results
• 🧮 Pre-fetch hygiene filtering — drops hits below minMembers or (optionally) any paid community before spending an /about-page request on them
• 🔗 Direct link ingestion — paste /about URLs or bare slugs; runs standalone or alongside keyword search
• 🏁 Discovery rank per row — each keyword-sourced community carries its 1-based rank in Skool's own search order and the total match count for that keyword
• 👤 Owner profile extraction — name, bio, location, and profile picture of each community's creator, from the same /about snapshot
• 🛡️ Optional proxy ladder — direct connection first, then Apify datacenter, then residential, only when the site is slow to reach

What data can you get with Skool Scraper?

Skool Scraper returns one row per collected community, built from three data groups: the community profile itself, the owner's public profile, and the keyword search context that surfaced it.

Result Type	Extracted Fields	Primary Use Case
Community profile	slug, url, group_id, group_name, group_description, group_image, group_created_at, group_updated_at, total_members, total_online_members, total_posts, total_admins, privacy, membership, num_courses, num_modules, tabs, survey	Niche sizing, competitive research, course/community benchmarking
Owner profile	owner (id, name, email, location, bio, picture_profile, created_at, updated_at), owner_name	Outreach and lead research on community creators
Search context (discovery rank)	searchKeyword, discoveryRank, searchResultsTotal	Seeing exactly where a community ranks in Skool's own keyword search, and how large the matching field is

Discovery rank and search context

For every community sourced from searchKeywords, Skool Scraper keeps the exact position it held in Skool's own search results (discoveryRank, 1-based) and the total number of matches Skool reported for that keyword (searchResultsTotal, capped at 1000). Ranks are preserved across the hygiene filter, so a gap in the sequence (rank 1, 4, 7…) tells you real hits were filtered out for being too small or paid — not that the actor skipped rows. Rows sourced from urls instead carry searchKeyword: "" and discoveryRank/searchResultsTotal as null, so you can always tell which path found a given community.

{
  "slug": "ai-automation-society",
  "searchKeyword": "ai automation",
  "discoveryRank": 3,
  "searchResultsTotal": 842
}

Owner profile

Every successful /about fetch resolves the community's creator into a nested owner object — id, name, location, bio, picture_profile, and account timestamps — plus a flattened owner_name column for quick scanning in the dataset view. owner.email is set to "N/A" whenever Skool doesn't expose it to a logged-out visitor, which in practice is effectively always.

Why not build this yourself?

Skool has no published public API for community search or discovery — no documented /api/v1/... endpoints, no developer portal, no API key system, as of this writing. Anyone querying /discovery directly has to solve Skool's own protections, too: plain HTTP requests to /discovery come back HTTP 202 behind an AWS WAF JavaScript challenge from every Apify network path — direct container egress, datacenter proxy, and residential proxy alike. A script can't execute challenge.js, so no amount of retrying or proxy rotation over plain HTTP clears it; only a real browser that runs the challenge and receives the resulting cookie gets through. Building and maintaining that yourself means running and babysitting a headless browser, handling WAF cookie expiry mid-run, and layering a proxy ladder on top — all before you've written a single line of /about-page parsing. Skool Scraper does all of that already: a Playwright browser solves the discovery challenge (only spun up when searchKeywords is used), and the lighter /about fetch stays on a plain HTTP path with its own proxy fallback.

How to scrape Skool with Skool Scraper?

1. Open Skool Scraper on its Apify Store listing and click Try for free (or Run, if you already have it saved).
2. Enter one or more niche keywords in 🔍 Niche keywords to search (searchKeywords) — one per line, e.g. ai automation.
3. Set your query controls: 🔢 Max communities per keyword (maxCommunitiesPerKeyword), 👥 Minimum members (minMembers), and 🆓 Free-to-join communities only (publicOnly) to shape which hits survive to the /about fetch.
4. Click Start to launch the run.
5. Watch rows land live in the Output tab, then export as JSON, CSV, Excel, or any other format Apify's dataset export supports.

How to run multiple queries in one job

Add as many lines as you want to searchKeywords — each keyword is searched and paginated independently, and results accumulate in the same dataset with searchKeyword recording which line found each row. You can combine that with a urls list in the same run; direct links are processed after keyword discovery finishes, using the exact same /about-fetch logic.

⬇️ Input

At least one of searchKeywords or urls must be filled in — a run with both empty stops immediately with an error. Everything else is optional and defaults to a sensible value.

Parameter	Required	Type	Description	Example Value
searchKeywords	No	array (string list)	One topic/keyword per line, searched against Skool's public discovery listing. Runs alongside urls — use either or both.	["ai automation", "ecommerce"]
maxCommunitiesPerKeyword	No	integer (min 1, max 1000, default 30)	How many matching communities to keep per keyword, ranked by Skool's own search order.	30
minMembers	No	integer (min 0, default 100)	Skip search hits below this member count, using the count Skool's own search result already reports — applied before the /about fetch. 0 disables the filter.	100
publicOnly	No	boolean (default true)	When true, skip search hits carrying a paid-membership price tag in Skool's search result, keeping free-to-join communities only.	true
urls	No	array (string list)	Skool community/group links (https://www.skool.com/{slug}/about) or bare slugs, one per line. Works standalone or alongside searchKeywords.	["https://www.skool.com/your-group/about"]
proxyConfiguration	No	object (proxy editor, default {"useApifyProxy": false})	Optional Apify Proxy for the /about page fetch (direct → datacenter → residential, sticky after the first residential success).	{"useApifyProxy": true}
maxRetries	No	integer (min 0, max 10, default 3)	Retries per /about-fetch HTTP attempt on connection errors (not when data is simply missing).	3

{
  "searchKeywords": ["ai automation", "ecommerce"],
  "maxCommunitiesPerKeyword": 30,
  "minMembers": 100,
  "publicOnly": true,
  "urls": [],
  "maxRetries": 3
}

⚠️ proxyConfiguration does not cover the keyword-discovery step. The optional proxy ladder applies only to /about-page fetches — the Playwright browser that solves Skool's discovery WAF challenge always runs from the Actor's own container network, with no proxy attached. If /discovery search itself is unreachable, a proxy toggle won't fix it; only /about-fetch reachability responds to it.

⬆️ Output

Every collected community is written to the Apify dataset as one JSON row, the instant it's fetched — no waiting for the whole run to finish. Export the dataset as JSON, CSV, Excel, or any other format Apify supports.

Scraped results

[
  {
    "slug": "ai-automation-society",
    "url": "https://www.skool.com/ai-automation-society/about",
    "group_id": "8f2c1a90-...",
    "group_name": "AI Automation Society",
    "group_description": "A community for founders automating their business with AI.",
    "group_image": "https://cdn.skool.com/...jpg",
    "group_created_at": "2023-02-11T00:00:00.000Z",
    "group_updated_at": "2026-08-01T00:00:00.000Z",
    "total_members": 4820,
    "total_online_members": 112,
    "total_posts": 3105,
    "total_admins": 3,
    "privacy": 1,
    "map_enabled": "N/A",
    "membership": 1,
    "num_courses": 4,
    "num_modules": 22,
    "tabs": {},
    "survey": {},
    "owner": {
      "id": "usr_1a2b3c",
      "name": "Jordan Lee",
      "email": "N/A",
      "location": "Austin, TX",
      "bio": "Building AI workflows for solopreneurs.",
      "picture_profile": "https://cdn.skool.com/...jpg",
      "created_at": "2022-11-04T00:00:00.000Z",
      "updated_at": "2026-07-20T00:00:00.000Z"
    },
    "owner_name": "Jordan Lee",
    "searchKeyword": "ai automation",
    "discoveryRank": 1,
    "searchResultsTotal": 842
  },
  {
    "slug": "ecom-scaling-lab",
    "url": "https://www.skool.com/ecom-scaling-lab/about",
    "group_id": "9d4e2b71-...",
    "group_name": "Ecom Scaling Lab",
    "group_description": "Free community for 7-figure ecommerce operators.",
    "group_image": "https://cdn.skool.com/...jpg",
    "group_created_at": "2022-06-30T00:00:00.000Z",
    "group_updated_at": "2026-08-10T00:00:00.000Z",
    "total_members": 15230,
    "total_online_members": 340,
    "total_posts": 9871,
    "total_admins": 5,
    "privacy": 1,
    "map_enabled": "N/A",
    "membership": 1,
    "num_courses": 6,
    "num_modules": 41,
    "tabs": {},
    "survey": {},
    "owner": {
      "id": "usr_7f9a2c",
      "name": "Priya Anand",
      "email": "N/A",
      "location": "",
      "bio": "",
      "picture_profile": "https://cdn.skool.com/...jpg",
      "created_at": "2021-09-18T00:00:00.000Z",
      "updated_at": "2026-06-02T00:00:00.000Z"
    },
    "owner_name": "Priya Anand",
    "searchKeyword": "ecommerce",
    "discoveryRank": 7,
    "searchResultsTotal": 1000
  },
  {
    "slug": "your-group",
    "url": "https://www.skool.com/your-group/about",
    "group_name": "",
    "owner_name": "",
    "searchKeyword": "",
    "discoveryRank": null,
    "searchResultsTotal": null,
    "error": "page not reachable — try optional proxy in input"
  }
]

The third row shows a failed fetch: on any error (unreachable page, blocked response, malformed /about HTML, or an invalid entry in urls), the Actor still writes a row with error set, group_name/owner_name empty, and none of the community-profile fields — instead of silently dropping the input.

How can I use the data extracted with Skool Scraper?

• 🎯 Course creators and community founders: size a niche before launching — run your topic keywords, sort by total_members and discoveryRank, and see how crowded (or open) a space really is before committing to it.
• 🤖 AI engineers and LLM developers: wire the Actor's JSON output straight into an agent's context — a keyword goes in, structured community and owner data comes out, ready to ground an answer about a niche.
• 📊 Market researchers: track how many total matches (searchResultsTotal) a niche keyword produces on Skool over time, and how the top-ranked (discoveryRank) communities in it shift.
• 🧭 Agencies and growth teams: build outreach lists from owner_name and owner.bio, filtered by total_members and publicOnly, to find community owners worth partnering with or pitching.

How do you monitor discovery rank over time?

Discovery-rank monitoring means re-running the same keyword set on a schedule and diffing each community's discoveryRank and searchResultsTotal between runs, instead of treating a single run as a one-off snapshot. Because Skool's search order shifts as communities grow, launch, or go quiet, a community that ranked #3 for "ai automation" last month and #11 this month is a real, checkable signal — not noise. The fields to diff are discoveryRank (did this community move up or down for the keyword?), searchResultsTotal (is the whole niche getting more crowded?), and total_members/total_posts (is a specific community actually growing, or just holding its rank). A simple workflow: schedule a run across your keyword set, store each run's dataset, and alert whenever a tracked community's discoveryRank crosses a threshold you care about — newly breaking into the top 10, or falling out of it. Apify's built-in Schedule feature (Console → Actor → Schedules) can trigger the same input on a recurring cron, so you don't have to start runs by hand.

Integrate Skool Scraper and automate your workflow

Skool Scraper works with any language or tool that can send an HTTP request to the Apify API.

REST API with Python

import requests

run = requests.post(
    "https://api.apify.com/v2/acts/scrapier~skool-group-community-infos-scrape-by-keyword-search/run-sync-get-dataset-items",
    params={"token": "<YOUR_APIFY_TOKEN>"},
    json={
        "searchKeywords": ["ai automation"],
        "maxCommunitiesPerKeyword": 30,
        "minMembers": 100,
        "publicOnly": True,
    },
)

for community in run.json():
    print(community["group_name"], community["total_members"], community.get("discoveryRank"))

MCP for query-grounded AI agents

Skool Scraper is reachable through Apify's MCP server like any Apify Actor: point an MCP-compatible client (Claude, Claude Code, or another agent framework) at npx @apify/actors-mcp-server --actors scrapier/skool-group-community-infos-scrape-by-keyword-search (with APIFY_TOKEN set), or the remote endpoint https://mcp.apify.com/?actors=scrapier/skool-group-community-infos-scrape-by-keyword-search. Once registered, an agent can call the Actor as a tool — a user asks about a niche, the agent runs a keyword search, gets structured JSON back, and grounds its answer in real communities instead of guessing.

Scheduled monitoring and delivery

Use Apify's Schedule feature to re-run the same keyword set on a cron interval, and Apify's Integrations tab to push each run's dataset to storage, a webhook, or a downstream automation tool (Zapier, Make, Slack, and others) — no code changes to the Actor required.

Is it legal to scrape Skool community data?

Yes — Skool Scraper only reads pages Skool already serves to any logged-out visitor, the same /discovery search results and /about pages a human browsing Skool would see. In the U.S., accessing publicly available web data has been found not to violate the Computer Fraud and Abuse Act (hiQ Labs v. LinkedIn Corp, 9th Cir., 2019). Community name, description, member/post counts, and pricing are business data about a public group, governed by Skool's Terms of Service rather than data-protection law. The owner object is different: it carries a real person's name, bio, and location, which is personal data — treat it under the applicable privacy regime (GDPR/CCPA) if you store or reuse it, particularly for outreach. Scraping for one-off research and scraping for ongoing operational monitoring carry different risk profiles. Consult your legal team for commercial use cases involving bulk data storage.

Frequently asked questions

Can I use searchKeywords and urls in the same run?

Yes. Both fields are processed independently and merged into one dataset — keyword discovery runs first, then any urls entries are fetched the same way the base link-only workflow always has. Leave either one empty if you only need the other.

How does maxCommunitiesPerKeyword control result count?

It caps how many communities are kept per keyword after the minMembers/publicOnly hygiene filter — not how many raw search hits are scanned. Skool serves 30 hits per discovery page and hard-caps every keyword at 1000 total matches (searchResultsTotal), so the Actor paginates until it has maxCommunitiesPerKeyword survivors or runs out of hits, whichever comes first.

How does Skool Scraper handle Skool's anti-bot measures?

Skool's /discovery search sits behind an AWS WAF JavaScript challenge that returns HTTP 202 to plain HTTP requests from any network path. Skool Scraper runs discovery through a real headless Playwright browser that executes the challenge script and reuses the resulting session cookie across pages, falling back to a fresh page load if the cookie expires mid-run. The lighter /about fetch has no such gate and uses a proxy ladder (direct → Apify datacenter → Apify residential, sticky after success) when proxyConfiguration is enabled.

Does Skool Scraper extract discovery rank?

Yes. Every community sourced from searchKeywords carries discoveryRank (its 1-based position in Skool's own search order for that keyword) and searchResultsTotal (Skool's total match count, capped at 1000). Both are null for communities sourced from urls, since there's no keyword search behind them.

How many results does Skool Scraper return per query?

Up to maxCommunitiesPerKeyword per keyword (1–1000, default 30), limited by however many communities actually pass the minMembers/publicOnly filter and by Skool's own 1000-match cap per keyword.

How do I use Skool Scraper to monitor a niche over time?

Schedule the same searchKeywords list to run on a recurring basis (Apify's Schedule feature), then diff each run's discoveryRank, searchResultsTotal, and total_members against the previous run to see which communities are climbing, which are shrinking, and whether the whole niche is getting more crowded on Skool.

Does Skool Scraper work with Claude, ChatGPT, and AI agent frameworks?

Yes. It's callable as a plain HTTP endpoint by any agent framework, and it's also reachable through Apify's MCP server (@apify/actors-mcp-server or mcp.apify.com) for MCP-compatible clients — letting an agent run a keyword search and ground its answer in live Skool community data.

How does Skool Scraper compare to other Skool scrapers?

As observed on their Apify Store listings on 2026-08-16: Skool Community Scraper and SKOOL Members/Groups Discovery Scraper both support keyword-based discovery search, but neither lists a search-rank or total-match-count field in its output; Skool Scraper exposes both (discoveryRank, searchResultsTotal) on every keyword-sourced row. Skool Scraper by louisdeconinck also supports discovery search but is billed as a monthly rental plus usage, rather than pay-per-result.

Can I use Skool Scraper without managing proxies or Skool credentials?

Yes. No Skool account, login, or cookie is used anywhere in the pipeline — every page fetched is public. proxyConfiguration is entirely optional and only affects the /about-page fetch step; leave it off for the simplest setup, or turn it on if /about pages are slow or blocked from your run's IP.

Your feedback

Found a bug, a missing field, or a Skool page layout that broke the extractor? Open an issue from this Actor's page in Apify Console, or message Scrapier directly through the Store listing — reports get read and acted on.