# edX Scraper | University Courses and Programs (`parseforge/edx-scraper`) Actor Extract edX course catalog data including title, university, instructors, level, duration, price, language, subject, prerequisites, and full description. Track MicroMasters, professional certificates, and degree programs for education analytics, lead generation, and market research. - **URL**: https://apify.com/parseforge/edx-scraper.md - **Developed by:** [ParseForge](https://apify.com/parseforge) (community) - **Categories:** Education, For creators, Lead generation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $19.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![ParseForge Banner](https://github.com/ParseForge/apify-assets/blob/ad35ccc13ddd068b9d6cba33f323962e39aed5b2/banner.jpg?raw=true) ## ๐ŸŽ“ edX Scraper > ๐Ÿš€ **Export edX course data in seconds.** Search by keyword or subject and collect course titles, partners, prices, durations, enrollment counts, and direct links in one clean dataset. > ๐Ÿ•’ **Last updated:** 2026-05-22 ยท **๐Ÿ“Š 13 fields** per record ยท **Up to 1,000,000 courses** ยท **Global course catalog coverage** The edX Scraper extracts course and program listings from edX's public course catalog using Algolia's search index - the same fast search engine edX uses for its own website. Each record includes the course title, subtitle, offering partner (MIT, Harvard, Microsoft, Google, etc.), subject category, difficulty level, language, duration, price, recent enrollment count, course type, and a direct link to the course page. edX hosts thousands of courses and professional programs from over 160 partner universities and institutions worldwide, covering subjects from data science and machine learning to business, humanities, and language learning. This scraper gives you structured access to that entire catalog in one run. ### Target Audience | Who | Why | |-----|-----| | EdTech researchers | Analyze course catalog composition, pricing, and partner distribution | | Curriculum developers | Benchmark course structures and durations across providers | | HR and L&D teams | Discover upskilling programs for specific skills and subjects | | Competitive intelligence teams | Track which skills edX partners are prioritizing | | Content creators and educators | Research what already exists before building new courses | ### ๐Ÿ“‹ What the edX Scraper does - Queries edX's Algolia search index by keyword and optional subject filter - Returns courses, programs, executive education, and degree products - Captures partner name and logo, pricing, enrollment count, language, and duration - Formats duration as a human-readable string (e.g. "6 weeks (5-8 hrs/wk)") - Strips HTML from descriptions to return clean plain text - Paginates automatically until `maxItems` is reached or all results are exhausted > ๐Ÿ’ก **Why it matters:** edX's course catalog spans thousands of offerings across 160+ partner institutions - far too many to browse manually. This actor lets researchers, product teams, and L&D professionals access the full catalog in minutes, with structured data ready for analysis or import. ### ๐ŸŽฌ Full Demo ๐Ÿšง Coming soon ### โš™๏ธ Input | Field | Type | Default | Description | |-------|------|---------|-------------| | `query` | string | `python` | Search keyword (e.g. "machine learning", "data science", "project management") | | `subject` | string | *(none)* | Subject filter (e.g. "Computer Science", "Business & Management") | | `maxItems` | integer | `10` | Maximum number of courses to collect (free: 10, paid: up to 1,000,000) | **Example 1 - Search for data science courses:** ```json { "query": "data science", "maxItems": 100 } ```` **Example 2 - Browse computer science courses:** ```json { "query": "", "subject": "Computer Science", "maxItems": 200 } ``` > โš ๏ธ **Good to Know:** The `subject` field must match an edX subject category exactly (e.g. "Computer Science", "Business & Management", "Data Analysis & Statistics"). Free users are limited to 10 courses per run. All course types are included: individual courses, programs, executive education, and degrees. ### ๐Ÿ“Š Output | Field | Type | Description | |-------|------|-------------| | ๐Ÿ–ผ๏ธ `imageUrl` | string | Course cover image URL | | ๐Ÿ“ `title` | string | Full course or program name | | ๐Ÿ“„ `subtitle` | string | Short course description (HTML-stripped) | | ๐Ÿ›๏ธ `partner` | string | Offering institution (e.g. MIT, Harvard, Google) | | ๐Ÿ–ผ๏ธ `partnerLogo` | string | Partner institution logo URL | | ๐Ÿ—‚๏ธ `subject` | string | Subject category | | ๐Ÿ“Š `level` | string | Difficulty level (Introductory, Intermediate, Advanced) | | ๐ŸŒ `language` | string | Language of instruction | | โฑ๏ธ `duration` | string | Course length (e.g. "6 weeks (5-8 hrs/wk)") | | ๐Ÿ’ฐ `price` | string | Price or "Free" if no cost | | ๐Ÿ‘ฅ `enrollmentCount` | number | Recent enrollment count | | ๐Ÿท๏ธ `courseType` | string | Product type (Course, Program, Executive Education, 2U Degree) | | ๐Ÿ”— `url` | string | Direct link to the edX course page | | ๐Ÿ•’ `scrapedAt` | string | ISO timestamp of when the record was collected | **Sample record:** ```json { "imageUrl": "https://prod-discovery.edx-cdn.org/media/programs/card_images/abc123.jpg", "title": "Python for Data Science", "subtitle": "Learn Python programming fundamentals and apply them to real-world data analysis problems.", "partner": "MIT", "partnerLogo": "https://prod-discovery.edx-cdn.org/organization/logos/mit.png", "subject": "Computer Science", "level": "Introductory", "language": "English", "duration": "4 weeks (5-7 hrs/wk)", "price": "USD 149", "enrollmentCount": 48200, "courseType": "Course", "url": "https://www.edx.org/learn/python/massachusetts-institute-of-technology-python-for-data-science", "scrapedAt": "2026-05-22T09:15:00.000Z" } ``` ### โœจ Why choose this Actor - **Algolia-powered** - uses edX's own fast search index, not brittle HTML scraping - **All product types** - courses, programs, executive education, and degree programs in one dataset - **Human-readable duration** - combines weeks and hours/week into a single readable string - **HTML-stripped descriptions** - plain text subtitles, no markup to clean up - **Partner logo included** - institution logos for display in apps or reports - **Pay-per-item pricing** - only pay for the course records you collect ### ๐Ÿ“ˆ How it compares to alternatives | Feature | ParseForge edX Scraper | Manual catalog browsing | Coursera scraper | |---------|------------------------|------------------------|-----------------| | Full catalog export | Yes | No | Different platform | | Enrollment counts | Yes | Sometimes | Sometimes | | Partner logos | Yes | Yes | Varies | | Duration data | Yes | Yes | Varies | | Price extraction | Yes | Yes | Yes | | Free tier | 10 courses | Unlimited (slow) | N/A | ### ๐Ÿš€ How to use 1. [Create a free Apify account](https://console.apify.com/sign-up?fpr=vmoqkp) (includes $5 credit) 2. Open the edX Scraper actor page and click **Try for free** 3. Enter a search keyword (e.g. `machine learning`, `finance`, `UX design`) 4. Optionally add a subject filter 5. Set `maxItems` to the number of courses you want 6. Click **Start** - results are typically ready in under 30 seconds 7. Download as CSV, JSON, Excel, or connect via API ### ๐Ÿ’ผ Business use cases #### Learning and Development (L\&D) Planning HR and L\&D teams can pull all courses in a given subject area, compare partner quality and pricing, and build a curated list of upskilling resources for employees. #### EdTech Market Research EdTech companies and investors can analyze the full edX catalog to understand market saturation, pricing norms, popular topics, and which institutions are most active. #### Curriculum Gap Analysis Educators building new courses can scan existing edX offerings in their subject to identify what already exists and where gaps remain in the curriculum landscape. #### Enrollment Trend Analysis The `enrollmentCount` field reveals which topics are most in-demand. Researchers and product teams can track which subjects are growing in popularity over time by running periodic scans. ### ๐Ÿ”Œ Automating edX Scraper - **Make (formerly Integromat)** - Schedule monthly catalog scans and update an Airtable course database automatically - **Zapier** - Notify your team when new courses matching your keyword are added to the catalog - **Google Sheets** - Export course lists and share with your L\&D team for review and curation - **REST API** - Embed course search into internal learning portals or recommendation tools ### ๐ŸŒŸ Beyond business use cases #### Personal Learning Discovery Students and self-learners can scan the full edX catalog for a topic they want to master, comparing multiple courses from different institutions side by side before enrolling. #### Academic Research on Online Education Researchers studying MOOCs and the online learning landscape can collect systematic data on course offerings, pricing models, and institutional participation. #### Journalism and Reporting Education journalists can track how the edX catalog evolves over time - which subjects are growing, which partners are expanding, and how pricing changes. #### Open Data Projects Developers and data enthusiasts can contribute edX course data to open educational resource directories or build recommendation engines. ### ๐Ÿค– Ask an AI assistant about this scraper Not sure how to work with your results? Ask an AI: > "I have JSON data from the edX Scraper with fields like title, partner, subject, level, price, and enrollmentCount. How do I identify the 10 most popular free courses and group them by subject?" The structured output is designed to be immediately usable with spreadsheets, databases, or AI-powered analysis tools. ### โ“ Frequently Asked Questions **Is this scraper affiliated with edX or 2U?** No. This is an independent tool that accesses edX's public Algolia search index - the same index powering edX's own website search. **What course types are included?** All edX product types: individual Courses, Programs (MicroMasters, MicroBachelors, Professional Certificate), Executive Education, and 2U Degrees. **What does `price: "Free"` mean?** The course is available to audit for free. Some courses offer a paid verified certificate option but can be taken without charge. **What does `enrollmentCount` represent?** The recent enrollment count as reported by edX's Algolia index. This is not a total all-time enrollment - it reflects recent activity. **Can I filter by language?** Not at the input level. The `language` field in the output lets you filter after export. **How current is the data?** Data is fetched live from Algolia at run time. It reflects the edX catalog as it was at that moment. **Can I collect all courses on edX?** Yes - leave `query` empty and set a large `maxItems`. The catalog has several thousand courses and programs. **What is the `subject` filter?** It maps to edX's subject taxonomy. Examples: "Computer Science", "Data Analysis & Statistics", "Business & Management", "Language", "Engineering". **Does it include course syllabi or module breakdowns?** No. The scraper extracts catalog-level metadata. For detailed course content, visit the course URL directly. **How many courses does edX have?** edX hosts several thousand courses and programs from 160+ partner institutions. **Does it capture discounts or promotional pricing?** The `price` field reflects the list price from the catalog. Promotional pricing is not captured. **Can I run this on a schedule?** Yes. Use Apify's built-in scheduler to run monthly or quarterly catalog snapshots. ### ๐Ÿ”Œ Integrate with any app Export your dataset to: **Spreadsheets:** Google Sheets, Microsoft Excel, Airtable **Databases:** PostgreSQL, MySQL, MongoDB, Supabase **LMS and HR Tools:** Workday Learning, Cornerstone, Degreed **Automation:** Make, Zapier, n8n, Pipedream **Analytics:** Tableau, Power BI, Metabase, Google Looker Studio ### ๐Ÿ”— Recommended Actors | Actor | Description | |-------|-------------| | [Coursera Scraper](https://apify.com/parseforge) | Extract course listings and reviews from Coursera | | [Udemy Scraper](https://apify.com/parseforge) | Scrape Udemy courses with pricing and ratings | | [LinkedIn Learning Scraper](https://apify.com/parseforge) | Collect professional courses from LinkedIn Learning | > ๐Ÿ’ก **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge) for 50+ ready-to-use data extractors covering EdTech platforms, job boards, marketplaces, and more. *** **๐Ÿ†˜ Need Help?** [Open our contact form](https://tally.so/r/BzdKgA) and we will get back to you within one business day. *** > **โš ๏ธ Disclaimer:** This actor is an independent tool not affiliated with, endorsed by, or connected to edX Inc. or 2U Inc. It accesses only publicly available course catalog data. Use responsibly and in accordance with edX's Terms of Service. ParseForge is not responsible for how collected data is used. # Actor input Schema ## `query` (type: `string`): Search keyword to find courses (e.g. python, machine learning, data science) ## `maxItems` (type: `integer`): Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000 ## `subject` (type: `string`): Filter by subject area (e.g. Computer Science, Business, Data Science). Leave empty for all subjects. ## Actor input object example ```json { "query": "python", "maxItems": 10 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "python", "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("parseforge/edx-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "python", "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("parseforge/edx-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "python", "maxItems": 10 }' | apify call parseforge/edx-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=parseforge/edx-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "edX Scraper | University Courses and Programs", "description": "Extract edX course catalog data including title, university, instructors, level, duration, price, language, subject, prerequisites, and full description. Track MicroMasters, professional certificates, and degree programs for education analytics, lead generation, and market research.", "version": "0.1", "x-build-id": "qTlCjjhQaFa8bN7fK" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/parseforge~edx-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-parseforge-edx-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/parseforge~edx-scraper/runs": { "post": { "operationId": "runs-sync-parseforge-edx-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/parseforge~edx-scraper/run-sync": { "post": { "operationId": "run-sync-parseforge-edx-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "query": { "title": "Search Query", "type": "string", "description": "Search keyword to find courses (e.g. python, machine learning, data science)" }, "maxItems": { "title": "Max Items", "minimum": 1, "maximum": 1000000, "type": "integer", "description": "Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000" }, "subject": { "title": "Subject Filter", "type": "string", "description": "Filter by subject area (e.g. Computer Science, Business, Data Science). Leave empty for all subjects." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```