Coursera Courses Scraper (Ratings & Syllabus)

Coursera courses from the site's own published sitemap and course pages: title, partner universities and companies, the rating the site displays with its count, difficulty and duration band, skills, instructors with their learner reach, the full week-by-week syllabus with per-item durations, the parent specialization, translated languages and a review sample.

HTTP only, no login, no key, no browser. One request for the whole catalogue (21,000+ course URLs), then one request per course page.

What it is for

• Catalogue snapshots by keyword — every python or finance course with rating, enrolment-facing numbers and syllabus.
• Course monitoring — a list of slugs on a schedule, diff the ratings.
• Curriculum research — weeks, lessons and minutes per course.

Input

field	what it does
mode	catalog (default): read the sitemap, filter by keyword, fetch course pages. courses: fetch the slugs/URLs you give.
keyword	Catalog mode. Matched against the course slug (machine-learning, python, finance). Empty = the catalogue in sitemap order.
courses	Courses mode. Slugs, /learn/... paths or full URLs.
fetchDetails	On by default. The sitemap carries URLs only; everything else is on the course page.
maxResults, maxConcurrency, minRequestInterval, proxyConfiguration	Limits. Pages are ~950 KB each; keep runs modest.

Three things about this site worth knowing before you trust a run

1. Discovery uses the sitemap, not the search — by policy

Coursera's robots.txt disallows /search and /api/. This Actor never touches them. It reads the sitemap Coursera itself links from robots.txt, which lists the entire catalogue under the allowed /learn/ prefix, and filters that list by keyword. (robots.txt also names ClaudeBot with a single rule, Disallow: /lecture/; nothing here requests /lecture/.)

2. Most of the sitemap no longer serves a course

Measured 2026-09-16 on random slugs from the live sitemap: 9 of 14 answered HTTP 200 with intact markup and no course record — withdrawn, region-gated or renamed courses Coursera still lists. That is a real state of the catalogue, not a parser fault, so it has its own outcome (course_unavailable) and every summary carries coursesUnavailable, coursesUnavailableShare and sitemapListsMoreThanIsServed. A run that selects 8 slugs and returns 2 courses is telling you about Coursera, and says so.

3. The course JSON-LD is gone; the data is in the Apollo cache, and it lies twice

Course pages carry no Course JSON-LD any more (0 of 20 checked). The data lives in window.__APOLLO_STATE__, richer than the JSON-LD ever was — but:

• a course page carries three course nodes (the course and its siblings in the specialization); the Actor takes the one whose slug matches the URL requested, never the first;
• the rating on the course node is the instructor rating (4.95); the rating the site displays (ratingValue, 4.89 with ratingCount 32,911) lives on a different node. Both are emitted, named for what they are.

Difficulty and duration are enum strings (BEGINNER, ONE_TO_FOUR_WEEKS) — there is no numeric week count on the site; weekCount and lessonCount are counted from the syllabus itself. Per-item durations are ISO-8601 and are converted to minutes with the original kept.

Output

• COURSE — slug, courseId, name, courseUrl, description, ratingValue, ratingCount, instructorRating, contentSatisfactionRatingsCount, difficultyLevel, durationBand, partnerNames, partnerWebsites, instructorNames, instructors (with learner reach), skills, specializationName, specializationSlug, weekCount, lessonCount, totalLessonMinutes, weeks (items with durations), translatedLanguages, reviewSampleCount, reviewSampleAverage, catalogRank, discovery.
• SEARCH_SUMMARY — one per run: keyword, catalogueSize, matchedCourses, coursesReturned, detailsFetched, coursesUnavailable, coursesUnavailableShare, stoppedBeforeExhausted, requestsMade, sitemapListsMoreThanIsServed.
• ERROR — course_unavailable (per slug), not_found (a clean 404), unexpected_shape, fetch_failed, with detail.

Known limits

• Keyword matching is on the slug, because that is what the sitemap carries; a course whose slug does not contain the word is not found by it. Use courses mode for exact lists.
• Enrolment counts, prices and reviews beyond the page's sample are not on the course page and are not fetched.
• The catalogue is a ceiling: expect roughly a third to two thirds of any slice to be course_unavailable until Coursera prunes its sitemap.