MIT OpenCourseWare Scraper

Scrape course metadata from MIT OpenCourseWare — the free, open publication of virtually all MIT course content. The actor extracts structured data on 2,500+ courses across all MIT departments, with no authentication or proxy required.

What You Get

Each course record contains:

Field	Description
courseId	MIT course number (e.g. 6-006, 18-900)
title	Full course title
department	Department name (e.g. Mathematics)
departmentUrl	OCW department search URL
level	Undergraduate or Graduate
term	When taught (e.g. Spring 2020)
instructors	List of instructor names
description	Course description
topics	List of topic tags (e.g. Algorithms and Data Structures)
imageUrl	Course thumbnail URL
courseUrl	Canonical OCW URL
hasLectureVideos	true if lecture videos are available
hasLectureNotes	true if lecture notes are available
resourceTypes	List of available material types
sourceUrl	Same as courseUrl
scrapedAt	UTC ISO timestamp
recordType	Always course

Modes

search (default)

Full-text search across all OCW courses by keyword.

Input:

{
  "mode": "search",
  "searchQuery": "machine learning",
  "maxItems": 20
}

byDepartment

Browse all courses in a specific MIT department.

Input:

{
  "mode": "byDepartment",
  "department": "Mathematics",
  "maxItems": 50
}

byLevel

Browse all Undergraduate or Graduate courses.

Input:

{
  "mode": "byLevel",
  "level": "Undergraduate",
  "maxItems": 100
}

byUrl

Fetch a single course by its OCW URL.

Input:

{
  "mode": "byUrl",
  "courseUrl": "https://ocw.mit.edu/courses/6-006-introduction-to-algorithms-spring-2020/"
}

Filters (all modes)

Filter	Type	Description
filterDepartment	string	Substring match on department name
filterLevel	select	Undergraduate or Graduate
minYear	integer	Only courses taught in or after this year
maxYear	integer	Only courses taught in or before this year
containsKeyword	string	Keyword must appear in title or description
maxItems	integer	Maximum records to output (default 50, max 3000)

Example Output

{
  "courseId": "6-867",
  "title": "Machine Learning",
  "department": "Electrical Engineering and Computer Science",
  "level": "Graduate",
  "term": "Fall 2006",
  "instructors": ["Prof. Tommi Jaakkola", "Ali Mohammad", "Rohit Singh"],
  "description": "An introductory course on machine learning covering classification, regression, boosting, SVMs, and Bayesian networks.",
  "topics": ["Computer Science", "Artificial Intelligence", "Mathematics"],
  "imageUrl": "https://ocw.mit.edu/courses/6-867-machine-learning-fall-2006/image.jpg",
  "courseUrl": "https://ocw.mit.edu/courses/6-867-machine-learning-fall-2006/",
  "hasLectureVideos": false,
  "hasLectureNotes": true,
  "resourceTypes": ["Lecture Notes", "Problem Sets", "Exams"],
  "sourceUrl": "https://ocw.mit.edu/courses/6-867-machine-learning-fall-2006/",
  "scrapedAt": "2026-06-07T10:00:00+00:00",
  "recordType": "course"
}

Data Source

MIT OpenCourseWare (ocw.mit.edu) is a free and open MIT activity that publishes course materials. All data is publicly available with no authentication. The scraper uses the internal Elasticsearch API at open.mit.edu/api/v0/search/ — the same API the OCW website uses to power its search page.

FAQs

Q: Does this require login or API keys? A: No. All OCW content is freely public. No credentials needed.

Q: How many courses are available? A: MIT OCW has approximately 2,500 published courses spanning 37 departments.

Q: Can I filter by topic? A: Use mode=search with a topic keyword in searchQuery, or combine with containsKeyword for post-filter matching.

Q: What departments are available? A: All 37 MIT departments including Electrical Engineering & Computer Science, Mathematics, Physics, Economics, Mechanical Engineering, and more. See the department input dropdown for the full list.

Q: Are lecture videos included? A: The hasLectureVideos field indicates if a course includes lecture videos. resourceTypes lists all available material types (lecture notes, problem sets, exams, etc.).

Q: How fresh is the data? A: Data is fetched live from the OCW search API on each run. MIT typically publishes new courses each semester.

Q: What is the daily test prefill? A: {"mode": "search", "searchQuery": "machine learning", "maxItems": 5} — always returns results as ML courses are among OCW's most popular.