Edx Courses Scraper

edX Courses Scraper — Course Catalog, Providers, and Enrollment Details

Collect publicly available edX course catalog information into a structured dataset. Discover courses by keyword and gather course titles, provider names, course numbers, schedules, pacing, descriptions, and available media information. Export your results in JSON, CSV, Excel, and other supported formats.

Overview

edX Courses Scraper helps learners, researchers, and analysts explore course catalogs and organize educational opportunities for research and planning.

Enter an optional search query and set a maximum number of results. Leave the query empty to browse the available catalog without a keyword filter.

Features

• Course Discovery: Find courses matching a search term.
• Catalog Browsing: Collect available courses without a search filter.
• Course Metadata: Gather course names, identifiers, numbers, and provider organizations.
• Schedule Information: Capture available start dates, end dates, and enrollment windows.
• Learning Details: Collect effort estimates, pacing, and mobile availability when provided.
• Descriptions and Media: Retrieve available course descriptions, image information, and video references.
• Course Pages: Include links to course information pages when available.
• Duplicate Prevention: Avoid duplicate records based on course URLs.
• Configurable Collection: Set a maximum item count and request timing options.
• Structured Exports: Export results in JSON, CSV, Excel, and other supported formats.

Use Cases

• Course Research: Discover courses related to programming, data science, business, and other subjects.
• Learning Plans: Organize course options for personal or professional development.
• Provider Analysis: Compare course catalogs across educational organizations.
• Catalog Management: Maintain structured records of course titles, schedules, and descriptions.
• Educational Research: Analyze available course formats and pacing options.
• Course Discovery Tools: Prepare datasets for educational directories and search applications.

Input Parameters

Parameter	Type	Description
query	String	Optional keyword used to search the course catalog. Leave empty to browse without a keyword filter.
maxItems	Integer	Maximum number of course records to collect.
requestDelaySecs	Number	Delay between requests, in seconds.
userAgent	String	Optional client identification string.
timeoutSecs	Number	Maximum overall actor runtime, in seconds.
requestTimeoutSecs	Number	Maximum wait time for an individual request, in seconds.

Example Input

{
  "query": "python",
  "maxItems": 50,
  "requestDelaySecs": 1,
  "userAgent": "ActorResearch/1.0",
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}

To browse without a keyword filter, set query to an empty string.

What Data Does This Actor Extract?

Available fields depend on the information provided for each course.

Field	Description
courseKey	Course's unique catalog identifier.
courseName	Course title.
courseNumber	Course number assigned by the provider.
org	Provider organization.
shortDescription	Short course description.
effort	Estimated weekly workload or effort, when available.
start	Course start date or timestamp.
end	Course end date or timestamp, when available.
enrollmentStart	Enrollment opening date, when available.
enrollmentEnd	Enrollment closing date, when available.
startDisplay	Human-readable start date.
startType	Type of start-date information.
pacing	Course pacing, such as self-paced.
mobileAvailable	Indicates whether mobile access is available.
hidden	Indicates whether the course is marked as hidden.
invitationOnly	Indicates whether enrollment is invitation-only.
courseImageUri	Course image reference.
courseImageRaw	Available course image location.
courseVideoUri	Course video reference, when available.
courseUrl	Link to the course information page.
scraped_at	Timestamp when the record was collected.

Example Output

{
  "courseKey": "course-v1:MITx+6.00.1x+2T2017",
  "courseName": "Introduction to Computer Science and Programming Using Python",
  "courseNumber": "6.00.1x",
  "org": "MITx",
  "shortDescription": "",
  "effort": "6:00",
  "start": "2017-05-30T00:00:00Z",
  "end": null,
  "enrollmentStart": null,
  "enrollmentEnd": null,
  "startDisplay": "May 30, 2017",
  "startType": "timestamp",
  "pacing": "self",
  "mobileAvailable": true,
  "hidden": false,
  "invitationOnly": false,
  "courseImageUri": "/asset-v1:MITx+6.00.1x+2T2017+type@asset+block@images_course_image.jpg",
  "courseVideoUri": null,
  "courseUrl": "https://courses.edx.org/courses/course-v1%3AMITx%2B6.00.1x%2B2T2017/about",
  "scraped_at": "2026-10-26T00:00:00+00:00"
}

This example illustrates the output structure. Actual course information and field availability depend on the catalog records returned.

Working with Your Dataset

After the actor finishes, open the dataset to review and export your course records.

You can:

• Export results as JSON, CSV, Excel, or another supported format.
• Search and sort courses by title, provider, or pacing.
• Filter records by course start date or organization.
• Build course directories and learning-resource catalogs.
• Analyze course information using Python, Pandas, or spreadsheet tools.
• Save periodic snapshots for educational research and reporting.

FAQ

Who is this actor for?

It is designed for learners, educators, researchers, developers, and analysts who want structured edX course catalog information.

Can I search for a specific subject?

Yes. Enter a keyword such as python in the query field.

Can I browse the catalog without a search term?

Yes. Leave query empty to collect available catalog records without a keyword filter.

What export formats are supported?

You can export the dataset in JSON, CSV, Excel, and other formats supported by Apify.

Can I schedule recurring collections?

Yes. Configure an Apify schedule to refresh the dataset periodically.

Will every course include enrollment dates and media details?

No. Some fields may be empty or unavailable, depending on the information published for each course.

Disclaimer

This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with edX or its owners. Users are responsible for complying with applicable terms of use and laws when collecting and using data.

Is Web Scraping Legal?

Collecting information that is already public is generally legal. There is no single law that bans all forms of web scraping. The limits are the usual ones: do not break into accounts, do not take private or confidential data, and be careful with personal data and copyrighted works. A site's terms can also restrict how you access it. Facts such as a public name, price, or rating are not the same thing as republishing someone's article or photo.

This actor collects information from public pages. It does not log in and it does not bypass access controls.

For a longer explanation, see Apify's article Is web scraping legal?.