edX Courses Scraper

Export individual courses from edX's rendered public catalogue. Search results include course titles, provider names, course URLs and the summary text displayed on each result card. Topic links and unrelated recommendations are excluded.

The September 11 update renders the page in Chromium. The initial HTTP response can contain generic “Popular courses” while the requested search is still loading, so those embedded recommendations are not used as substitutes for search results.

Try a Python course search

Open the existing public example, or run:

{
  "searchUrls": ["https://www.edx.org/search?q=python"],
  "maxResults": 3
}

The Actor selects the Courses tab by adding tab=course to a search URL. It preserves the other query parameters. It waits for the actual result cards or edX's explicit no-course message before exporting. The heading can briefly display zero during loading; that intermediate state is not treated as a completed empty search.

Inputs and page boundaries

Input	Meaning
searchUrls	edx.org /search pages or /learn/<topic> catalogue pages.
maxResults	Maximum exported courses per supplied page, from 1 to 1,000.

This Actor reads the displayed page only. It does not automatically advance pagination. A result cap of 100 can therefore return fewer than 100 courses even when the search reports hundreds of matches. For search pages, only cards tied to the current search response are accepted; recommendation cards outside those results are excluded.

Degree programmes, professional certificates and executive education are outside the course export. Individual course detail URLs, authenticated learning pages, video lessons and enrolment flows are not supported inputs. This run does not enrol you in a course or download protected learning material.

Real output example

Observed on September 11, 2026 in the Python course results:

{
  "source": "edx-courses-scraper",
  "kind": "courses",
  "index": 1,
  "title": "CS50's Introduction to Artificial Intelligence with Python",
  "company": "Harvard University",
  "url": "https://www.edx.org/learn/artificial-intelligence/harvard-university-cs50-s-introduction-to-artificial-intelligence-with-python",
  "description": "Course | CS50's Introduction to Artificial Intelligence with Python | Harvard University | 7 weeks to complete | Introductory level",
  "input_url": "https://www.edx.org/search?q=python",
  "source_url": "https://www.edx.org/search?q=python&tab=course&page=1"
}

Field	Meaning
title	The course title displayed on its result card.
company	The provider shown on that card, or null if absent. The existing field name is retained for compatibility.
url	Individual edX course URL with search-tracking parameters removed.
description	Text from that card, joined with separators and capped at 1,500 characters. This is not a full syllabus.
input_url, source_url	Requested URL and final browser URL, including the selected Courses tab.
source, kind, index	Source identifier, record type and one-based position within the exported page.

The Actor does not normalize price, certificate availability, language, duration or difficulty into separate fields. Values that happen to appear in the summary retain their displayed wording. Missing values are not filled with assumptions.

Build a useful course inventory

For a shortlist, keep the canonical course URL as the source key and retain the query that found it. A course can appear in several supplied searches; deduplicate across the combined export only if you no longer need those separate search observations. Within one page, repeated course URLs are exported once.

To compare course offerings over time, save each successful dataset with an observation date. Treat title, provider and card-text changes as review candidates. Absence from a bounded search page does not prove that enrolment closed or the course was withdrawn. Search ranking and pagination can change between observations.

Use Apify's dataset export for JSON, CSV or Excel, or read the default dataset through its API. A saved Task retains your input for later runs. Scheduling and downstream integrations are available through Apify; this Actor does not create schedules or send messages on its own.

Empty searches and failures

An explicit edX message saying that no courses match the query produces a successful empty dataset. Popular courses displayed below that message are not exported. A denied request, browser timeout or page without identifiable results produces a failed run instead of a fabricated result or silent success.

If one of several input pages fails, already saved records remain in the dataset and the final run is marked failed. Check that status before replacing an existing inventory. Partial data can still be useful, but it should retain its incomplete-run context.

Pricing and runtime

The Free-tier result price remains $2.50 per 1,000 saved courses, plus the existing Actor start event. Progressive discounts are unchanged. The Pricing tab shows your current tier and memory-dependent start charge.

Chromium is required to see the actual search results. Media loading is disabled, and the existing public Task's 4,096 MB setting is retained. This release does not establish a lower-memory operating limit. Use a small result cap and maximum total charge for an initial run; each input URL has its own result limit. A failed or empty run can still incur the start event.

Support

If a current query returns unexpected courses, share the public URL, run ID and affected fields through an Apify issue or debugging share. Keep account credentials and private learning content out of public issues. The source can change its search layout; a successful build alone does not verify extraction.