edX Course Scraper

Pull the full edX catalog into clean, structured rows — courses, programs, executive education, and online degrees, each with its school, skills taught, difficulty level, pacing, weekly effort, language, availability, and recent enrollment counts. Search by keyword or paste edX links, and get one flat row per product. Built for edtech market researchers, course-directory builders, and learning & development teams who need edX catalog data at scale without copy-pasting product pages one at a time.

Why This Scraper?

• All four edX content types in one run — Courses, Programs (Professional Certificate, MicroMasters, XSeries), Executive Education, and Online Degrees, each tagged with a recordType discriminator so you can split them downstream.
• 15 course languages and 3 difficulty levels — filter to English, Spanish, French, Chinese, Arabic, Japanese, Portuguese, Korean, and 7 more, at Introductory, Intermediate, or Advanced.
• Recent enrollment counts on every product — a live demand signal (recentEnrollmentCount) showing how many learners recently signed up, captured on every course, program, and degree.
• Two-level skills taxonomy on every product — each course and program carries both a flat skills list and a structured skillsData tree (skill → category → subcategory), ready for taxonomy mapping, skills-gap analysis, or curriculum matching.
• Partner-school name and logo — the offering institution plus its logo image URL for Harvard, MIT, Berkeley, and 200+ other edX partners.
• Effort and pacing on a single row — self-paced vs. instructor-led, estimated weeks to complete, and minimum/maximum weekly hours, so you can rank offerings by time commitment.
• Beat the 1,000-per-query ceiling — leave the content-type filter empty and each of the four types is searched separately, returning up to 4,000 products for one broad keyword.
• Flat, one-row-per-product output — every field sits at the top level with no nested blobs, so results drop straight into Google Sheets, Excel, or a BI table.
• Short, full, and overview descriptions — a clean plain-text summary alongside the complete description and course overview for every product.

Use Cases

Edtech Market Research

• Size the online-learning market by subject, school, or content type
• Track how many programs each partner institution offers on edX
• Rank subjects by learner demand using recent enrollment counts
• Map language coverage across the catalog to spot localization gaps

Competitive Intelligence

• Monitor a competing institution's new courses, programs, and degrees
• Benchmark pricing and pacing against comparable offerings
• Detect emerging topics from a burst of newly listed products
• Compare executive-education portfolios across business schools

Course Aggregators & Directories

• Populate a course-comparison site with fresh, structured catalog data
• Keep listings current with weekly re-runs on the same keywords
• Enrich existing directory entries with skills, effort hours, and levels
• Build subject-specific landing pages from a single keyword search

Lead Generation for Edtech

• Identify partner schools and programs for affiliate or content partnerships
• Build outreach lists of institutions active in a given subject area
• Surface high-enrollment programs worth featuring or promoting

Learning & Development / Content Curation

• Curate approved course lists for employee upskilling by level and language
• Match internal skills frameworks to courses that teach those skills
• Compare weekly time commitment across candidate programs
• Assemble localized catalogs for regional teams using the language filter

Getting Started

Simple Keyword Search

The simplest run — one topic, up to 100 products:

{
    "searchQueries": ["data science"],
    "maxResults": 100
}

Filtered Search (Content Type + Level + Language)

Narrow to introductory English courses and programs on a topic:

{
    "searchQueries": ["business analytics"],
    "contentType": ["course", "program"],
    "level": ["Introductory"],
    "language": ["English"],
    "maxResults": 200
}

Search Page + Direct Product Link

Paste edX URLs directly — a search results page and a single course page in one run:

{
    "startUrls": [
        "https://www.edx.org/search?q=artificial+intelligence",
        "https://www.edx.org/learn/computer-science/harvard-university-cs50-s-introduction-to-computer-science"
    ],
    "maxResults": 150
}

Input Reference

What to Scrape

Parameter	Type	Default	Description
searchQueries	string[]	["data science"]	Keyword searches such as "data science", "python", or a school name like "Harvard". Each query is searched separately. Leave empty if you are only using Start URLs below.
startUrls	string[]	[]	Paste edX URLs directly — search result pages, subject/category pages, or individual course, program, or degree pages. Leave empty if you are only using Search Queries.

Filters

Filters apply to keyword searches. Leave a filter empty to include everything.

Parameter	Type	Default	Description
contentType	select[]	["course"]	Which product types to include: Courses; Programs (Professional Certificate, MicroMasters, XSeries); Executive Education; Online Degrees. Leave empty to include all four types.
level	select[]	[]	Only include products at these difficulty levels: Introductory, Intermediate, Advanced. Empty means all levels.
language	select[]	[]	Only include products taught in these languages — 15 options including English, Spanish, French, Chinese, Arabic, and Japanese. Empty means all languages.

Limits

Parameter	Type	Default	Description
maxResults	integer	100	Maximum products to return in total across all queries and URLs. Set to 0 for unlimited. edX returns up to 1,000 products per keyword for a single content type; leaving the content-type filter empty searches all four types separately, so a broad keyword can return up to 4,000. Start with 50–100 to test.

Output

Every row carries a recordType field — course, program, executive_education, or degree — over one shared, flat schema. Program rows add a few program-specific fields, and executive-education and degree rows add an externalUrl.

Course (recordType: "course")

{
    "recordType": "course",
    "id": "a1b2c3d4-5678-90ab-cdef-1234567890ab",
    "title": "CS50's Introduction to Computer Science",
    "url": "https://www.edx.org/learn/computer-science/harvard-university-cs50-s-introduction-to-computer-science",
    "productType": "Course",
    "partnerName": "Harvard University",
    "partnerLogoUrl": "https://prod-discovery.edx-cdn.org/organization/logos/harvard.png",
    "subjects": ["Computer Science"],
    "skills": ["Algorithms", "Data Structures", "SQL", "Web Development"],
    "skillsData": [
        { "skill": "Algorithms", "category": "Information Technology", "subcategory": "Software Development" },
        { "skill": "SQL (Programming Language)", "category": "Information Technology", "subcategory": "Query Languages" }
    ],
    "level": ["Introductory"],
    "pacing": ["self_paced"],
    "weeksToComplete": 11,
    "minHoursPerWeek": 6,
    "maxHoursPerWeek": 18,
    "languages": ["English"],
    "availability": ["Available now"],
    "isActive": true,
    "isPartOfProgram": false,
    "recentEnrollmentCount": 154236,
    "staff": ["david-j-malan", "doug-lloyd"],
    "imageUrl": "https://prod-discovery.edx-cdn.org/media/course/image/cs50.jpg",
    "shortDescription": "An introduction to the intellectual enterprises of computer science and the art of programming.",
    "fullDescription": "<p>This is CS50, Harvard University's introduction to the intellectual enterprises of computer science...</p>",
    "overview": "<p>Learn how to think algorithmically and solve problems efficiently.</p>",
    "translationLanguages": ["Spanish", "French"],
    "price": 0,
    "currency": "USD",
    "sourceUrl": "https://www.edx.org/search?q=computer science",
    "seedType": "search",
    "seedValue": "computer science"
}

Program (recordType: "program")

Program rows share the core fields above and add course-membership details:

{
    "recordType": "program",
    "id": "9f8e7d6c-1234-5678-9abc-def012345678",
    "title": "Professional Certificate in Data Science",
    "productType": "Professional Certificate",
    "partnerName": "Harvard University",
    "level": ["Introductory", "Intermediate"],
    "courseCount": 9,
    "courseUuids": ["11111111-...", "22222222-...", "33333333-..."],
    "weeksToCompleteMin": 8,
    "weeksToCompleteMax": 16,
    "tags": ["Data Science", "Statistics"],
    "recentEnrollmentCount": 48210,
    "seedType": "search",
    "seedValue": "data science"
}

Executive-education and online-degree rows carry the same core fields plus an externalUrl pointing to the program's landing page.

Core Product Fields

Field	Type	Description
recordType	string	course, program, executive_education, or degree
id	string	edX product UUID (dedup key is recordType:id)
title	string	Product title
url	string	edX product page URL
productType	string	edX's own label (e.g. "Course", "Professional Certificate")
partnerName	string	Offering school or institution
partnerLogoUrl	string	Partner logo image URL
subjects	string[]	Subject / category tags
skills	string[]	Skills taught (flat list)
skillsData	object[]	Skills taxonomy — one { skill, category, subcategory } object per skill
level	string[]	Difficulty level(s)
languages	string[]	Language(s) of instruction
availability	string[]	e.g. "Available now", "Starting soon", "Archived"
isActive	boolean	Whether the product is currently active
isPartOfProgram	boolean	Whether the course belongs to a program

Effort, Pacing & Pricing

Field	Type	Description
pacing	string[]	self_paced or instructor_paced
weeksToComplete	number	Estimated weeks to complete
minHoursPerWeek	number	Minimum effort hours per week
maxHoursPerWeek	number	Maximum effort hours per week
recentEnrollmentCount	number	Recent enrollment count (demand signal)
price	number	Verified/paid track price when present
currency	string	Currency code for price

Content & Media

Field	Type	Description
imageUrl	string	Product card / hero image URL
staff	string[]	Instructor / staff identifiers
shortDescription	string	Plain-text short description
fullDescription	string	Full description (HTML)
overview	string	Product overview (HTML)
translationLanguages	string[]	AI-translation languages offered

Program & Provenance Fields

Field	Type	Description
courseCount	number	Programs only — number of courses in the program
courseUuids	string[]	Programs only — UUIDs of the included courses
weeksToCompleteMin	number	Programs only — minimum weeks across courses
weeksToCompleteMax	number	Programs only — maximum weeks across courses
tags	string[]	Programs only — tag labels
externalUrl	string	Executive Education / Online Degrees only — external landing-page URL
sourceUrl	string	Search or discovery page the record was found on
seedType	string	search or url — how the record was reached
seedValue	string	The query text or URL that produced this record

Tips for Best Results

• Start small. Set maxResults to 50–100 on your first run to confirm the data matches your needs, then scale up.
• Leave the content type empty to break the 1,000-per-query ceiling. With no contentType filter, all four product types are searched separately, so a broad keyword can return up to 4,000 products instead of 1,000.
• Build localized catalogs with the language filter. The 15 language options map to edX's own catalog facets — set language to ["Spanish"] to pull a Spanish-only catalog in one pass.
• Narrow broad keywords. A term like "business" returns the maximum window; "business analytics for managers" gives a tighter, more relevant set.
• Rank by demand. Sort your dataset by recentEnrollmentCount (descending) to surface the most popular offerings first — ideal for featuring or curation.
• Combine searches and links. Mix searchQueries and startUrls in a single run to cover keyword discovery and specific product pages at once.
• Filter by level for curriculum planning. Use level to separate Introductory offerings from Advanced ones when assembling role-based learning tracks.

Pricing

From $2.50 per 1,000 results — a flat pay-per-result rate that undercuts comparable edX extractors. Bronze, Silver, and Gold subscribers pay progressively less; the table below shows the total cost at each discount tier.

Results	No discount	Bronze	Silver	Gold
100	$0.30	$0.28	$0.27	$0.25
1,000	$3.00	$2.80	$2.65	$2.50
10,000	$30.00	$28.00	$26.50	$25.00
100,000	$300.00	$280.00	$265.00	$250.00

A "result" is any product row in the output dataset — a course, program, executive-education course, or online degree. No compute or time-based charges — you pay per result, plus a small fixed per-run start fee.

Integrations

Export data in JSON, CSV, Excel, XML, or RSS. Connect to 1,500+ apps via:

• Zapier / Make / n8n — Workflow automation
• Google Sheets — Direct spreadsheet export
• Slack / Email — Notifications on new results
• Webhooks — Trigger custom APIs on run completion
• Apify API — Full programmatic access

Legal & Ethical Use

This actor is designed for legitimate market research, competitive analysis, course-directory building, and learning & development use. Users are responsible for complying with applicable laws and edX's terms of service, including making reasonable-rate requests and respecting content usage rules for any material linked from edX. Do not use extracted data for spam, harassment, or any illegal purpose.