Class Central Courses Scraper

Searches Class Central — the largest cross-platform MOOC discovery/aggregator site — and returns real course records: rating, provider, subject, level, certificate and free/paid status, spanning Coursera, edX, Udemy, YouTube, university OpenCourseWare and hundreds of other providers in one search.

Public data only. No login, no cookies, no browser, no bot challenge of any kind — 7/7 TLS profiles tested return 200, cold, on both the search and course-detail surfaces.

How this differs from edx-courses-scraper

edx-courses-scraper covers edX's own catalogue only. Class Central is a cross-platform aggregator: one search returns matches from dozens of providers at once, plus Class Central's own community rating layer that individual provider sites don't have.

What you get

Two record types share one dataset, told apart by recordType.

COURSE — one row per course

Raw passthrough of Class Central's own per-course analytics record: course_id, course_name, course_slug, course_provider, course_institution, course_subject, course_level, course_certificate, course_language, course_is_university, course_is_classroom, course_is_free, course_avg_rating, course_num_rating. Flattened course* convenience fields are added for the table view. With Fetch full course details on, four more fields are added that the search result does not carry at all: detailDescription (full text), detailReviews (dated reviews with body text), detailOffers (exact price/currency/seller), detailAggregateRating.

SEARCH_SUMMARY — one row per search term

The site's own reported total, how many rows this run pulled, and how many pages it fetched — plus totalMayBeCapped, explained below.

Input

Field	What it does
Search terms	one search per entry — e.g. machine learning, general relativity
Max courses per search term	0 = unlimited, up to maxPagesPerQuery
Max pages per search term	15 courses/page; caps how deep an unlimited crawl goes
Fetch full course details	one extra request per course for description/reviews/price (off by default)

Example

{
  "searchTerms": ["machine learning", "general relativity"],
  "maxItemsPerTerm": 30,
  "includeCourseDetails": true
}

A known ceiling in the reported total — totalMayBeCapped

Class Central's own "Load the next 15 courses of N" text reports exactly 19,985 for every broad subject tested — "machine learning", "data science", "programming", "business", "engineering", "art", "health" all land on the identical figure, while genuinely narrower queries ("python" 17,536, "history" 15,243, "quantum computing" 3,846) each report a different, plausible number. Seven unrelated subjects landing on one exact number rules out coincidence: this is a display ceiling on the reported total, not necessarily the true match count.

The actual course data is not affected — pagination was verified genuinely deep and honest even for a capped query (checked to page 1000 on "machine learning": zero id overlap between any two pages tested, all the way out). Only the headline number in SEARCH_SUMMARY.totalMatches can be an undercount; the courses this actor returns are real and correctly matched either way. totalMayBeCapped: true fires whenever totalMatches equals exactly 19,985, so you know when to treat that number as "at least this many" rather than "exactly this many."

Notes on reliability

• No WAF. chrome124, chrome131, chrome120, safari17_0, firefox133, chrome99_android and edge101 all return 200, cold.
• robots.txt has no ClaudeBot/anthropic-ai/Claude-User/ Claude-SearchBot/CCBot group at all (only GPTBot is blocked, and this actor does not identify as GPTBot) — policy-open. /search is not disallowed; /api/* and /reviews/* are, and this actor never touches either — see CRAWLING_METHOD.md.
• Search is fuzzy relevance, not exact-phrase matching. A narrow-looking multi-word query can still match thousands of courses on individual words — documented, not a bug (same as this portfolio's app-store-keyword-rank-tracker and efinancialcareers-jobs-scraper). A genuinely nonsense query does return a real, verified zero.
• Residential proxy, no country pin is the cloud default.

Known limits

• The 19,985 total ceiling above — see the dedicated section.
• Course detail URLs are only ever built from search results, never accepted as raw user input. The detail page's slug segment is cosmetic (only the trailing numeric id is authoritative) — a hand-typed wrong slug with a real id silently resolves to a different, unrelated course. This actor cannot hit that trap because every (slug, id) pair it uses comes straight from the site's own search result, never from a user-guessed string.