# Futurelearn Course Spider (`getdataforme/futurelearn-course-spider`) Actor The FutureLearn Course Spider - Run is an Apify Actor that scrapes course details from FutureLearn.com across specified pages. Extract titles, providers, durations, ratings, and more in structured JSON. Ideal for educational research, content curation, or market analysis. Flexible and reliable - **URL**: https://apify.com/getdataforme/futurelearn-course-spider.md - **Developed by:** [GetDataForMe](https://apify.com/getdataforme) (community) - **Categories:** AI, Automation, E-commerce - **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $9.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## FutureLearn Course Spider ### Introduction The FutureLearn Course Spider - Run is an Apify Actor designed to scrape and collect detailed course information from FutureLearn.com. It navigates through specified pages to extract data such as course titles, introductions, providers, durations, ratings, and more, delivering structured results for easy analysis or integration. ### Input Parameters The Actor accepts a JSON input with the following parameters: | Parameter | Type | Required | Description | |------------|---------|----------|-------------| | StartPage | Integer | Yes | The starting page number for scraping courses. | | MaxPages | Integer | Yes | The maximum number of pages to scrape. | Example input: ```` { "StartPage": 1, "MaxPages": 100 } ``` ### Features - **Paginated Scraping**: Efficiently crawls multiple pages on FutureLearn, starting from a specified page up to a maximum limit. - **Comprehensive Data Extraction**: Gathers rich details including course ID, title, URL, provider, duration, ratings, category, and metadata like Unlimited inclusion. - **Structured JSON Output**: Returns results in a clean, parsable JSON array for straightforward use in applications or databases. - **Flexible Configuration**: Allows users to control the scope of scraping with start and max page parameters for targeted data collection. - **Reliable and Up-to-Date**: Ensures fresh course data from FutureLearn without manual effort. ### Possible Use Cases - **Educational Research**: Analyze course trends, such as popular categories, average ratings, or provider distributions across pages. - **Content Curation**: Build custom course catalogs or recommendation systems by aggregating FutureLearn data. - **Market Insights**: Evaluate online learning offerings, including durations and review metrics, for competitive analysis. - **Personal Development Tools**: Integrate scraped data into apps for users to discover and compare courses efficiently. - **Data Enrichment**: Enhance datasets for reporting, AI models, or dashboards with structured FutureLearn course information. ### Output The Actor outputs an array of JSON objects, each representing a scraped course. Example output: ``` \[ { "course\_id": 1187, "course\_slug": "bookkeeping-financial-accounting", "course\_url": "https://www.futurelearn.com/courses/bookkeeping-financial-accounting", "title": "Bookkeeping for Personal and Business Accounting", "introduction": "Learn how to balance your books and develop the skills to manage finances in personal and business environments.", "image\_url": "https://ugc.futurelearn.com/uploads/images/5d/cb/thumbnail\_5dcb270b-8e7c-4582-b9c6-3c3bc9b7888a.jpg", "image\_alt": "man and woman at desk with laptop, calculator and printed paper with table of numbers", "provider": "The Open University", "duration\_weeks": "4 weeks", "hours\_per\_week": "3 hrs per week", "total\_reviews": 155, "average\_rating": 4.45806, "category": "Business & Management", "course\_type\_metadata": \[ { "icon": "unlimited", "label": "Included in Unlimited" } ], "is\_study\_uk": false }, { "course\_id": 928, "course\_slug": "perioperative-medicine", "course\_url": "https://www.futurelearn.com/courses/perioperative-medicine", "title": "Perioperative Medicine in Action", "introduction": "Explore perioperative medicine and how you can improve care for the high risk surgical patient. Join this RCoA endorsed course.", "image\_url": "https://ugc.futurelearn.com/uploads/images/42/83/thumbnail\_4283e6ec-7adb-48a0-b7df-5327551c1988.jpg", "image\_alt": "", "provider": "UCL (University College London)", "duration\_weeks": "4 weeks", "hours\_per\_week": "3 hrs per week", "total\_reviews": 236, "average\_rating": 4.73729, "category": "Healthcare & Medicine", "course\_type\_metadata": \[ { "icon": "unlimited", "label": "Included in Unlimited" } ], "is\_study\_uk": false } ] ```` ### Support For bug reports or custom needs, contact support@getdataforme.com Or use our contact form: https://getdataforme.com/contact/ (Make sure to include a clear subject line in your message) # Actor input Schema ## `StartPage` (type: `integer`): The start page for the spider. ## `MaxPages` (type: `integer`): The max pages for the spider. ## Actor input object example ```json { "StartPage": 1, "MaxPages": 100 } ```` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("getdataforme/futurelearn-course-spider").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("getdataforme/futurelearn-course-spider").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call getdataforme/futurelearn-course-spider --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=getdataforme/futurelearn-course-spider", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Futurelearn Course Spider", "description": "The FutureLearn Course Spider - Run is an Apify Actor that scrapes course details from FutureLearn.com across specified pages. Extract titles, providers, durations, ratings, and more in structured JSON. Ideal for educational research, content curation, or market analysis. Flexible and reliable", "version": "0.0", "x-build-id": "9szfYQEbV3Hx8Tt0w" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/getdataforme~futurelearn-course-spider/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-getdataforme-futurelearn-course-spider", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/getdataforme~futurelearn-course-spider/runs": { "post": { "operationId": "runs-sync-getdataforme-futurelearn-course-spider", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/getdataforme~futurelearn-course-spider/run-sync": { "post": { "operationId": "run-sync-getdataforme-futurelearn-course-spider", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "StartPage": { "title": "Start Page", "type": "integer", "description": "The start page for the spider.", "default": 1 }, "MaxPages": { "title": "Max Pages", "type": "integer", "description": "The max pages for the spider.", "default": 100 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```