# Edx Courses Scraper (`b2b_lead_generation/edx-courses-scraper`) Actor

edX Courses Scraper is a Python tool designed to extract course titles, partner universities, instructor details, enrollment numbers, subject tags, and syllabi from the edX platform. It exports structured online learning datasets into clean CSV or JSON format

- **URL**: https://apify.com/b2b_lead_generation/edx-courses-scraper.md
- **Developed by:** [B2B Lead Generation](https://apify.com/b2b_lead_generation) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### edX Courses Scraper — Course Catalog, Providers, and Enrollment Details

![edX Courses Scraper cover](https://images.bigibyte.com/cdn-cgi/image/width=1260,height=540,fit=cover,format=webp/apify-actors-covers/edx-courses-scraper-cover.png)

Collect publicly available edX course catalog information into a structured dataset. Discover courses by keyword and gather course titles, provider names, course numbers, schedules, pacing, descriptions, and available media information. Export your results in JSON, CSV, Excel, and other supported formats.

### Overview

edX Courses Scraper helps learners, researchers, and analysts explore course catalogs and organize educational opportunities for research and planning.

Enter an optional search query and set a maximum number of results. Leave the query empty to browse the available catalog without a keyword filter.

### Features

- **Course Discovery:** Find courses matching a search term.
- **Catalog Browsing:** Collect available courses without a search filter.
- **Course Metadata:** Gather course names, identifiers, numbers, and provider organizations.
- **Schedule Information:** Capture available start dates, end dates, and enrollment windows.
- **Learning Details:** Collect effort estimates, pacing, and mobile availability when provided.
- **Descriptions and Media:** Retrieve available course descriptions, image information, and video references.
- **Course Pages:** Include links to course information pages when available.
- **Duplicate Prevention:** Avoid duplicate records based on course URLs.
- **Configurable Collection:** Set a maximum item count and request timing options.
- **Structured Exports:** Export results in JSON, CSV, Excel, and other supported formats.

### Use Cases

- **Course Research:** Discover courses related to programming, data science, business, and other subjects.
- **Learning Plans:** Organize course options for personal or professional development.
- **Provider Analysis:** Compare course catalogs across educational organizations.
- **Catalog Management:** Maintain structured records of course titles, schedules, and descriptions.
- **Educational Research:** Analyze available course formats and pacing options.
- **Course Discovery Tools:** Prepare datasets for educational directories and search applications.

### Input Parameters

| Parameter | Type | Description |
|---|---|---|
| `query` | String | Optional keyword used to search the course catalog. Leave empty to browse without a keyword filter. |
| `maxItems` | Integer | Maximum number of course records to collect. |
| `requestDelaySecs` | Number | Delay between requests, in seconds. |
| `userAgent` | String | Optional client identification string. |
| `timeoutSecs` | Number | Maximum overall actor runtime, in seconds. |
| `requestTimeoutSecs` | Number | Maximum wait time for an individual request, in seconds. |

### Example Input

```json id="p7h4mx"
{
  "query": "python",
  "maxItems": 50,
  "requestDelaySecs": 1,
  "userAgent": "ActorResearch/1.0",
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}
```

To browse without a keyword filter, set `query` to an empty string.

### What Data Does This Actor Extract?

Available fields depend on the information provided for each course.

| Field | Description |
|---|---|
| `courseKey` | Course's unique catalog identifier. |
| `courseName` | Course title. |
| `courseNumber` | Course number assigned by the provider. |
| `org` | Provider organization. |
| `shortDescription` | Short course description. |
| `effort` | Estimated weekly workload or effort, when available. |
| `start` | Course start date or timestamp. |
| `end` | Course end date or timestamp, when available. |
| `enrollmentStart` | Enrollment opening date, when available. |
| `enrollmentEnd` | Enrollment closing date, when available. |
| `startDisplay` | Human-readable start date. |
| `startType` | Type of start-date information. |
| `pacing` | Course pacing, such as self-paced. |
| `mobileAvailable` | Indicates whether mobile access is available. |
| `hidden` | Indicates whether the course is marked as hidden. |
| `invitationOnly` | Indicates whether enrollment is invitation-only. |
| `courseImageUri` | Course image reference. |
| `courseImageRaw` | Available course image location. |
| `courseVideoUri` | Course video reference, when available. |
| `courseUrl` | Link to the course information page. |
| `scraped_at` | Timestamp when the record was collected. |

### Example Output

```json id="m3k8vd"
{
  "courseKey": "course-v1:MITx+6.00.1x+2T2017",
  "courseName": "Introduction to Computer Science and Programming Using Python",
  "courseNumber": "6.00.1x",
  "org": "MITx",
  "shortDescription": "",
  "effort": "6:00",
  "start": "2017-05-30T00:00:00Z",
  "end": null,
  "enrollmentStart": null,
  "enrollmentEnd": null,
  "startDisplay": "May 30, 2017",
  "startType": "timestamp",
  "pacing": "self",
  "mobileAvailable": true,
  "hidden": false,
  "invitationOnly": false,
  "courseImageUri": "/asset-v1:MITx+6.00.1x+2T2017+type@asset+block@images_course_image.jpg",
  "courseVideoUri": null,
  "courseUrl": "https://courses.edx.org/courses/course-v1%3AMITx%2B6.00.1x%2B2T2017/about",
  "scraped_at": "2026-10-26T00:00:00+00:00"
}
```

This example illustrates the output structure. Actual course information and field availability depend on the catalog records returned.

### Working with Your Dataset

After the actor finishes, open the dataset to review and export your course records.

You can:

- Export results as JSON, CSV, Excel, or another supported format.
- Search and sort courses by title, provider, or pacing.
- Filter records by course start date or organization.
- Build course directories and learning-resource catalogs.
- Analyze course information using Python, Pandas, or spreadsheet tools.
- Save periodic snapshots for educational research and reporting.

### FAQ

**Who is this actor for?**

It is designed for learners, educators, researchers, developers, and analysts who want structured edX course catalog information.

**Can I search for a specific subject?**

Yes. Enter a keyword such as `python` in the `query` field.

**Can I browse the catalog without a search term?**

Yes. Leave `query` empty to collect available catalog records without a keyword filter.

**What export formats are supported?**

You can export the dataset in JSON, CSV, Excel, and other formats supported by Apify.

**Can I schedule recurring collections?**

Yes. Configure an Apify schedule to refresh the dataset periodically.

**Will every course include enrollment dates and media details?**

No. Some fields may be empty or unavailable, depending on the information published for each course.

### Disclaimer

This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with edX or its owners. Users are responsible for complying with applicable terms of use and laws when collecting and using data.

### Is Web Scraping Legal?

Collecting information that is already public is generally legal. There is no single law that bans all forms of web scraping. The limits are the usual ones: do not break into accounts, do not take private or confidential data, and be careful with personal data and copyrighted works. A site's terms can also restrict how you access it. Facts such as a public name, price, or rating are not the same thing as republishing someone's article or photo.

This actor collects information from public pages. It does not log in and it does not bypass access controls.

For a longer explanation, see Apify's article [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/).

# Actor input Schema

## `query` (type: `string`):

Search term passed to the Open edX courses API as search_term. Leave empty to list courses without a search filter.

## `maxItems` (type: `integer`):

Stop after this many successful rows. Hard max is 1000.

## `requestDelaySecs` (type: `number`):

Seconds to wait between requests. Keep at least 1 for polite pacing.

## `userAgent` (type: `string`):

User-Agent header sent with each request.

## `timeoutSecs` (type: `integer`):

Overall run budget in seconds.

## `requestTimeoutSecs` (type: `integer`):

Per-request timeout in seconds.

## `proxyConfiguration` (type: `object`):

Optional proxy. Leave off unless you need it.

## Actor input object example

```json
{
  "query": "python",
  "maxItems": 100,
  "requestDelaySecs": 1,
  "userAgent": "ActorResearch/1.0",
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

Dataset rows in the overview table.

## `dataset` (type: `string`):

Full dataset for this run.

## `items` (type: `string`):

Dataset items as JSON.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "python"
};

// Run the Actor and wait for it to finish
const run = await client.actor("b2b_lead_generation/edx-courses-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "python" }

# Run the Actor and wait for it to finish
run = client.actor("b2b_lead_generation/edx-courses-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "python"
}' |
apify call b2b_lead_generation/edx-courses-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,b2b_lead_generation/edx-courses-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/NGWaMNkmG0LU6AAKK/builds/dy0225IN6Hv8J7fEZ/openapi.json