# SourceForge Scraper (`crawlerbros/sourceforge-scraper`) Actor Scrape SourceForge software listings from direct product URLs, SourceForge search queries, or SourceForge category pages. Extract ratings, pricing, company information, features, and alternatives. - **URL**: https://apify.com/crawlerbros/sourceforge-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Developer tools, Automation, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 17 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing from $3.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## SourceForge Scraper Scrape SourceForge business-software listings from direct product URLs, search queries, curated popular categories, or arbitrary category URLs. ### What it extracts - Product title and description - Overall rating and review count - Starting price and pricing details - Company information (name, founded year, website, country) - Supported platforms, training, and support options - API availability - Feature groups - Alternative software links ### Modes - **Product URLs** — direct `/software/product//` URLs. - **Search queries** — SourceForge software search. - **Popular category** — curated dropdown of 34 top SourceForge categories. - **Category URLs** — free-text any SourceForge category URL. ### Sort options `featured`, `rating_avg`, `rating_count`, `rating`. ### Inputs | Input | Type | Notes | |---|---|---| | `productUrls` | array | Direct product URLs | | `searchQueries` | array | Free-text search | | `categorySlug` | enum | Curated popular category | | `categoryUrls` | array | Free-text category URLs | | `sortBy` | enum | Sort order (default: featured) | | `searchPages` | integer | Pages per query/category (1–20) | | `maxItems` | integer | Cap on emitted products (1–500) | | `useProxy` | boolean | Apify proxy (default: off) | | `autoEscalateOnBlock` | boolean | Rotate sessions on block (default: on) | | `proxyConfiguration` | object | Apify proxy configuration (used when `useProxy` is true) | ### Use cases - Competitive software research - B2B software category mapping - Pricing intelligence - Alternatives and comparison datasets ### Limitations - SourceForge does not expose a stable per-product review-list pagination feed; we surface aggregate review count and overall rating from the product page. - The deployment-axis filters (SaaS / Windows / Mac / Linux) are URL-driven on SourceForge and can be supplied via free-text `categoryUrls` (e.g. `/software/saas/`). ### Notes - This actor is HTTP-first. - Empty values are omitted from output. # Actor input Schema ## `productUrls` (type: `array`): Direct SourceForge product URLs, such as https://sourceforge.net/software/product/Asana/. ## `searchQueries` (type: `array`): Keywords to run against SourceForge software search. ## `categorySlug` (type: `string`): (Optional) Browse a SourceForge top-level category. Use `categoryUrls` for any non-listed category. ## `categoryUrls` (type: `array`): Free-text SourceForge category URLs (e.g. https://sourceforge.net/software/marketing/). ## `sortBy` (type: `string`): Sort order applied on category and search browse pages. ## `searchPages` (type: `integer`): How many SourceForge pages to scan per search query / category. ## `maxItems` (type: `integer`): Hard cap on emitted SourceForge products. ## `useProxy` (type: `boolean`): Route requests through Apify proxy. SourceForge tolerates datacenter IPs so proxy is off by default. ## `autoEscalateOnBlock` (type: `boolean`): If SourceForge starts returning blocks, retry with rotating sessions. ## `proxyConfiguration` (type: `object`): Proxy configuration used when `useProxy` is true. ## Actor input object example ```json { "productUrls": [ "https://sourceforge.net/software/product/Asana/" ], "searchQueries": [], "categorySlug": "", "categoryUrls": [], "sortBy": "featured", "searchPages": 1, "maxItems": 25, "useProxy": false, "autoEscalateOnBlock": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ```` # Actor output Schema ## `software` (type: `string`): Dataset containing SourceForge software listing records. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "productUrls": [ "https://sourceforge.net/software/product/Asana/" ], "searchQueries": [], "categoryUrls": [], "searchPages": 1, "maxItems": 25, "useProxy": false, "autoEscalateOnBlock": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/sourceforge-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "productUrls": ["https://sourceforge.net/software/product/Asana/"], "searchQueries": [], "categoryUrls": [], "searchPages": 1, "maxItems": 25, "useProxy": False, "autoEscalateOnBlock": True, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], }, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/sourceforge-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "productUrls": [ "https://sourceforge.net/software/product/Asana/" ], "searchQueries": [], "categoryUrls": [], "searchPages": 1, "maxItems": 25, "useProxy": false, "autoEscalateOnBlock": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }' | apify call crawlerbros/sourceforge-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/sourceforge-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "SourceForge Scraper", "description": "Scrape SourceForge software listings from direct product URLs, SourceForge search queries, or SourceForge category pages. Extract ratings, pricing, company information, features, and alternatives.", "version": "1.0", "x-build-id": "Gnax7cV83iAzYgYrx" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~sourceforge-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-sourceforge-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~sourceforge-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-sourceforge-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~sourceforge-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-sourceforge-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "productUrls": { "title": "Product URLs", "type": "array", "description": "Direct SourceForge product URLs, such as https://sourceforge.net/software/product/Asana/.", "default": [ "https://sourceforge.net/software/product/Asana/" ], "items": { "type": "string" } }, "searchQueries": { "title": "Search queries", "type": "array", "description": "Keywords to run against SourceForge software search.", "default": [], "items": { "type": "string" } }, "categorySlug": { "title": "Popular category", "enum": [ "", "artificial-intelligence", "business-intelligence", "collaboration", "communications", "compliance", "construction-management", "content-management", "crm", "customer-service", "data-management", "document-management", "ecommerce", "education", "engineering", "event-management", "field-service-management", "finance", "hardware", "healthcare", "human-resources", "insurance", "it-management", "it-security", "legal", "logistics", "manufacturing", "marketing", "operations", "project-management", "real-estate", "retail", "sales", "supply-chain", "transportation" ], "type": "string", "description": "(Optional) Browse a SourceForge top-level category. Use `categoryUrls` for any non-listed category.", "default": "" }, "categoryUrls": { "title": "Category URLs", "type": "array", "description": "Free-text SourceForge category URLs (e.g. https://sourceforge.net/software/marketing/).", "default": [], "items": { "type": "string" } }, "sortBy": { "title": "Sort order", "enum": [ "featured", "rating_avg", "rating_count", "rating" ], "type": "string", "description": "Sort order applied on category and search browse pages.", "default": "featured" }, "searchPages": { "title": "Search pages per query", "minimum": 1, "maximum": 20, "type": "integer", "description": "How many SourceForge pages to scan per search query / category.", "default": 1 }, "maxItems": { "title": "Max items", "minimum": 1, "maximum": 500, "type": "integer", "description": "Hard cap on emitted SourceForge products.", "default": 25 }, "useProxy": { "title": "Use Apify proxy", "type": "boolean", "description": "Route requests through Apify proxy. SourceForge tolerates datacenter IPs so proxy is off by default.", "default": false }, "autoEscalateOnBlock": { "title": "Auto-escalate on block", "type": "boolean", "description": "If SourceForge starts returning blocks, retry with rotating sessions.", "default": true }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy configuration used when `useProxy` is true.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```