# Online Course Scraper — edX + Coursera, Multi-Platform (`commanding_hotdog/online-course-scraper`) Actor Scrape online courses from edX & Coursera — prices, reviews, skills, duration. 20+ fields. Course comparison, MOOC research, education market analysis. Free API. - **URL**: https://apify.com/commanding\_hotdog/online-course-scraper.md - **Developed by:** [qingwa](https://apify.com/commanding_hotdog) (community) - **Categories:** Education - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Online Course Scraper Search and extract online courses from **edX** (primary) and **Coursera** (beta) — **20+ fields** per course including price, level, skills, enrollment count, and duration. ### Features - 🎓 **edX**: Full Algolia API — 4,000+ courses, 163+ filtered results per query - 📚 **Coursera**: Beta HTML parsing (limited) - 🔍 **Keyword search** — search across all platforms - 🎯 **Level filter** — Introductory, Intermediate, Advanced - 💰 **Free-only filter** — find free courses - 📊 **20+ fields** — title, partner, price, level, skills, enrollment, duration, pace, language ### Output Fields | Field | Type | Description | |---|---|---| | `platform` | string | Source platform (edX, Coursera) | | `title` | string | Course title | | `url` | string | Course URL | | `partner` | string | Institution/partner name | | `description` | string | Course description | | `price` | number | Price in USD | | `is_free` | boolean | Whether course is free | | `level` | string | Difficulty level | | `product_type` | string | Type (Course, Professional Certificate, etc.) | | `language` | string | Course language | | `pace` | string | Self-paced or instructor-paced | | `duration_weeks` | number | Estimated duration in weeks | | `effort_hours_min` | number | Min hours per week | | `effort_hours_max` | number | Max hours per week | | `enrollment_count` | number | Recent enrollment count | | `skills` | string | Skills you'll gain | | `availability` | string | Current, Upcoming, etc. | | `image_url` | string | Course thumbnail URL | | `source` | string | Data source identifier | ### Input Parameters | Parameter | Type | Default | Description | |---|---|---|---| | `query` | Text | python | Search keyword | | `platform` | Select | all | Platform: all/edx/coursera | | `maxResults` | Select | 100 | Max results: 50/100/200/500 | | `levelFilter` | Select | — | Level: Introductory/Intermediate/Advanced | | `freeOnly` | Checkbox | false | Only free courses | ### Example Usage **All Python courses:** ```json {"query": "python", "platform": "all", "maxResults": "100"} ```` **Free intermediate machine learning courses:** ```json {"query": "machine learning", "platform": "edx", "levelFilter": "Intermediate", "freeOnly": true} ``` ### Data Source - **edX**: Algolia Search API (public, no auth needed) - **Coursera**: HTML parsing (beta, limited) ### Pricing $5 per 1,000 results. # Actor input Schema ## `query` (type: `string`): Search keyword for courses (e.g., 'python', 'machine learning', 'data science'). ## `platformEdx` (type: `boolean`): Search edX courses (Algolia API, 4,000+ courses). ## `platformCoursera` (type: `boolean`): Search Coursera courses (SSR extraction, 3,000+ courses). ## `maxResults` (type: `string`): Maximum number of courses to return. ## `levelFilter` (type: `string`): Filter courses by difficulty level. ## `freeOnly` (type: `boolean`): Only return free courses. ## Actor input object example ```json { "query": "python", "platformEdx": true, "platformCoursera": true, "maxResults": "100", "levelFilter": "", "freeOnly": false } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "python", "platformEdx": true, "platformCoursera": true, "maxResults": "100", "levelFilter": "", "freeOnly": false }; // Run the Actor and wait for it to finish const run = await client.actor("commanding_hotdog/online-course-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "python", "platformEdx": True, "platformCoursera": True, "maxResults": "100", "levelFilter": "", "freeOnly": False, } # Run the Actor and wait for it to finish run = client.actor("commanding_hotdog/online-course-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "python", "platformEdx": true, "platformCoursera": true, "maxResults": "100", "levelFilter": "", "freeOnly": false }' | apify call commanding_hotdog/online-course-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=commanding_hotdog/online-course-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Online Course Scraper — edX + Coursera, Multi-Platform", "description": "Scrape online courses from edX & Coursera — prices, reviews, skills, duration. 20+ fields. Course comparison, MOOC research, education market analysis. Free API.", "version": "1.0", "x-build-id": "t9IatkOiUiasNehKR" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/commanding_hotdog~online-course-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-commanding_hotdog-online-course-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/commanding_hotdog~online-course-scraper/runs": { "post": { "operationId": "runs-sync-commanding_hotdog-online-course-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/commanding_hotdog~online-course-scraper/run-sync": { "post": { "operationId": "run-sync-commanding_hotdog-online-course-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "query": { "title": "Search Query", "type": "string", "description": "Search keyword for courses (e.g., 'python', 'machine learning', 'data science').", "default": "python" }, "platformEdx": { "title": "edX", "type": "boolean", "description": "Search edX courses (Algolia API, 4,000+ courses).", "default": true }, "platformCoursera": { "title": "Coursera", "type": "boolean", "description": "Search Coursera courses (SSR extraction, 3,000+ courses).", "default": true }, "maxResults": { "title": "Maximum Results", "enum": [ "50", "100", "200", "500" ], "type": "string", "description": "Maximum number of courses to return.", "default": "100" }, "levelFilter": { "title": "Filter by Level", "enum": [ "", "Introductory", "Intermediate", "Advanced", "Beginner" ], "type": "string", "description": "Filter courses by difficulty level.", "default": "" }, "freeOnly": { "title": "Free Courses Only", "type": "boolean", "description": "Only return free courses.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```