# Matrix Message Scraper (`jungle_synthesizer/matrix-message-scraper`) Actor Scrape public rooms and messages from any Matrix homeserver (matrix.org, Element, or self-hosted). Discover public rooms by keyword or scrape message history from specific rooms using a Matrix access token. - **URL**: https://apify.com/jungle\_synthesizer/matrix-message-scraper.md - **Developed by:** [BowTiedRaccoon](https://apify.com/jungle_synthesizer) (community) - **Categories:** Social media, Automation - **Stats:** 2 total users, 1 monthly users, 66.7% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Matrix Message Scraper Scrape public rooms and message history from any [Matrix homeserver](https://matrix.org) — matrix.org, Element, or self-hosted. Returns room metadata and message events in clean JSON. Matrix is a federated open protocol with 80 million accounts across 100,000+ homeservers. It's the backend for Element, the chat platform used by Mozilla, KDE, the French government, and a long list of open-source projects. Most public rooms are unencrypted and fully readable. This scraper gets you into that data. ### Matrix Message Scraper Features - **Discovers public rooms** across any Matrix homeserver without an account — name, topic, member count, join rules, aliases, and avatar URL - **Filters by keyword** — narrow public room results to a specific topic before collecting - **Scrapes message history** from one or more rooms using your Matrix access token — works on any public or joined room - **Extracts full event metadata** — sender ID, display name, timestamp, message type, reply threading, edits, and media URLs - **Handles cursor pagination** — fetches complete message history across thousands of events without manual cursor management - **Federation-aware** — query a different homeserver's public room list by pointing the homeserver URL at any Matrix server - **No proxies required.** Matrix REST API is accessible from standard IPs. - **Two distinct modes:** room discovery (no auth) and message scraping (access token required) ### What Can You Do With Matrix Data? - **Open-source community researchers** — map the Matrix ecosystem, identify active projects, track migration from Slack/Discord - **Academic linguists** — large-scale multilingual conversation corpora without Twitter's data access restrictions - **Compliance teams** — archive message history for organizations that adopted Element as their primary chat platform - **Privacy advocates** — analyze the federated network structure and federation patterns across homeservers - **Content moderation tooling** — build training datasets from public room conversations - **Developer tools** — monitor Matrix rooms for mentions, keywords, or bot triggers ### How It Works 1. **Pick a mode.** Discover Public Rooms lists rooms on the homeserver by keyword or returns all rooms. Scrape Room Messages fetches message history for specific room IDs. 2. **Configure the homeserver.** Defaults to matrix.org. Point it at any Matrix homeserver URL for federated queries. 3. **Provide a token for messages.** The Discover mode needs no credentials. For messages, get your access token from Element: Settings → Help & About → Access Token. 4. **Run.** The scraper handles cursor pagination — the Matrix API returns results in batches with next-page tokens, and this actor follows them until it hits your item limit. ### Matrix Message Scraper Input ```json { "action": "Discover Public Rooms", "homeserver": "https://matrix.org", "searchTerm": "linux", "maxItems": 100 } ```` ```json { "action": "Scrape Room Messages", "homeserver": "https://matrix.org", "accessToken": "syt_...", "roomIds": ["!abcdef:matrix.org", "!ghijkl:kde.org"], "maxMessagesPerRoom": 500, "maxItems": 1000, "messageDirection": "b" } ``` | Field | Type | Default | Description | |-------|------|---------|-------------| | `action` | string | `Discover Public Rooms` | What to scrape. Options: `Discover Public Rooms`, `Scrape Room Messages` | | `homeserver` | string | `https://matrix.org` | Matrix homeserver base URL | | `accessToken` | string | — | Matrix access token. Required for Scrape Room Messages. | | `roomIds` | array | — | List of Matrix room IDs (e.g. `!abcdef:matrix.org`). Used with Scrape Room Messages. | | `searchTerm` | string | — | Keyword filter for Discover Public Rooms. Leave blank to return all rooms. | | `maxItems` | integer | `10` | Maximum number of records to return across all rooms | | `maxMessagesPerRoom` | integer | `100` | Maximum messages per room. Used with Scrape Room Messages. | | `messageDirection` | string | `b` | Pagination direction. `b` = backward (older first), `f` = forward (newest first) | ### Matrix Message Scraper Output #### Mode 1: Discover Public Rooms ```json { "record_type": "room", "room_id": "!L58ME6ufiP49v97UIOBIpvWKEgj4912JmECPuDzlvCI", "room_name": "Matrix HQ", "room_topic": "The Official Matrix HQ — chat about Matrix here! | https://matrix.org", "room_canonical_alias": "#matrix:matrix.org", "room_member_count": 5451, "room_is_encrypted": false, "room_join_rule": "public", "room_world_readable": true, "room_guest_can_join": true, "room_avatar_url": "mxc://matrix.org/DRevoaEiuzbkOznknySKuMmE", "room_type": null, "homeserver": "https://matrix.org" } ``` | Field | Type | Description | |-------|------|-------------| | `record_type` | string | `room` for this mode | | `room_id` | string | Unique Matrix room ID (e.g. `!abcdef:matrix.org`) | | `room_name` | string | Display name of the room | | `room_topic` | string | Room topic or description | | `room_canonical_alias` | string | Canonical room alias (e.g. `#matrix:matrix.org`) | | `room_member_count` | integer | Number of joined members | | `room_is_encrypted` | boolean | Whether end-to-end encryption is enabled | | `room_join_rule` | string | Join rule: `public`, `invite`, `knock`, or `restricted` | | `room_world_readable` | boolean | Whether history is visible without joining | | `room_guest_can_join` | boolean | Whether guests can join | | `room_avatar_url` | string | `mxc://` URL for the room avatar | | `room_type` | string | Room type (`m.space` for Matrix Spaces, null for regular rooms) | | `homeserver` | string | Homeserver URL used for the query | #### Mode 2: Scrape Room Messages ```json { "record_type": "message", "event_id": "$example_event_id:matrix.org", "event_type": "m.room.message", "event_sender": "@alice:matrix.org", "event_sender_display_name": "Alice", "event_origin_server_ts": 1715000000000, "event_content_msgtype": "m.text", "event_content_body": "Hey everyone, anyone know the status of the new spec?", "event_content_formatted_body": null, "event_content_url": null, "event_reply_to": null, "event_thread_id": null, "event_edits": null, "event_room_id": "!L58ME6ufiP49v97UIOBIpvWKEgj4912JmECPuDzlvCI" } ``` | Field | Type | Description | |-------|------|-------------| | `record_type` | string | `message` for this mode | | `event_id` | string | Unique Matrix event ID (e.g. `$eventid:matrix.org`) | | `event_type` | string | Event type (only `m.room.message` events are saved) | | `event_sender` | string | Matrix user ID of the sender (e.g. `@user:matrix.org`) | | `event_sender_display_name` | string | Display name at time of event (may be null) | | `event_origin_server_ts` | integer | Event timestamp in milliseconds since Unix epoch | | `event_content_msgtype` | string | Message subtype: `m.text`, `m.image`, `m.file`, `m.video`, `m.audio`, `m.notice` | | `event_content_body` | string | Plain-text body or filename for media messages | | `event_content_formatted_body` | string | HTML-formatted body (when present) | | `event_content_url` | string | `mxc://` media URL for image/file/video/audio messages | | `event_reply_to` | string | Event ID being replied to (`m.in_reply_to`), if any | | `event_thread_id` | string | Thread root event ID (`m.thread` relation), if any | | `event_edits` | string | Event ID being edited by this event (`m.replace` relation), if any | | `event_room_id` | string | Room ID the event belongs to | ### 🔍 FAQ **How do I scrape Matrix rooms?** Matrix Message Scraper connects to the Matrix Client-Server API at your chosen homeserver. For public room discovery, no credentials are needed — just set `action` to `Discover Public Rooms` and run. For message history, get your access token from Element (Settings → Help & About → Access Token) and supply a list of room IDs. **How much does Matrix Message Scraper cost to run?** Matrix Message Scraper charges $0.10 per run start plus $0.001 per record. Scraping 1,000 messages from a set of rooms costs roughly $1.10. Public room discovery is inexpensive — 100 rooms is about $0.20 total. **What data can I get from matrix.org without logging in?** The public room directory. Matrix Message Scraper returns room ID, name, topic, member count, canonical alias, join rule, and avatar URL for all publicly listed rooms — no access token needed. Message history requires authentication. **Can Matrix Message Scraper scrape rooms on homeservers other than matrix.org?** Yes. Set `homeserver` to any Matrix server URL (e.g. `https://matrix.mozilla.org`, `https://matrix.kde.org`, or a self-hosted instance). Public room discovery and authenticated message scraping work on any standard Matrix homeserver. **Does it scrape encrypted rooms?** No. Encrypted room messages are stored as opaque ciphertext — without the private keys they're unreadable. Matrix Message Scraper skips encrypted event payloads. Most public rooms don't use encryption. ### Need More Features? Need custom filters, incremental scraping across runs, or support for a different homeserver configuration? [File a request](https://console.apify.com/actors/issues) or get in touch. ### Why Use Matrix Message Scraper? - **First Matrix scraper on Apify** — zero competing actors means no stale or abandoned alternatives - **No proxies, no browser overhead** — pure REST API access, which keeps costs low and runs fast - **Federation-aware** — one actor queries any homeserver, not just matrix.org # Actor input Schema ## `sp_intended_usage` (type: `string`): Please describe how you plan to use the data extracted by this crawler. ## `sp_improvement_suggestions` (type: `string`): Provide any feedback or suggestions for improvements. ## `sp_contact` (type: `string`): Provide your email address so we can get in touch with you. ## `action` (type: `string`): What to scrape: - **Discover Public Rooms** – list public rooms on the homeserver (no token required) - **Scrape Room Messages** – fetch message history for one or more rooms (access token required) ## `homeserver` (type: `string`): Matrix homeserver base URL. Defaults to matrix.org. ## `accessToken` (type: `string`): Matrix access token. Required for 'Scrape Room Messages'. Get it from Element: Settings → Help & About → Access Token. ## `roomIds` (type: `array`): List of Matrix room IDs to scrape messages from (e.g. !abcdef:matrix.org). Used by 'Scrape Room Messages' action. ## `searchTerm` (type: `string`): Filter public rooms by name/topic keyword. Used by 'Discover Public Rooms' action. Leave blank to return all public rooms. ## `maxItems` (type: `integer`): Maximum number of records to return across all rooms. ## `maxMessagesPerRoom` (type: `integer`): Maximum number of messages to fetch per room. 0 = unlimited. ## `messageDirection` (type: `string`): Pagination direction. 'b' (backward) fetches oldest messages first; 'f' (forward) fetches newest first. ## Actor input object example ```json { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "action": "Discover Public Rooms", "homeserver": "https://matrix.org", "maxItems": 10, "maxMessagesPerRoom": 100, "messageDirection": "b" } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "action": "Discover Public Rooms", "homeserver": "https://matrix.org", "maxItems": 10, "maxMessagesPerRoom": 100 }; // Run the Actor and wait for it to finish const run = await client.actor("jungle_synthesizer/matrix-message-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "action": "Discover Public Rooms", "homeserver": "https://matrix.org", "maxItems": 10, "maxMessagesPerRoom": 100, } # Run the Actor and wait for it to finish run = client.actor("jungle_synthesizer/matrix-message-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "sp_intended_usage": "Describe your intended use...", "sp_improvement_suggestions": "Share your suggestions here...", "sp_contact": "Share your email here...", "action": "Discover Public Rooms", "homeserver": "https://matrix.org", "maxItems": 10, "maxMessagesPerRoom": 100 }' | apify call jungle_synthesizer/matrix-message-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=jungle_synthesizer/matrix-message-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Matrix Message Scraper", "description": "Scrape public rooms and messages from any Matrix homeserver (matrix.org, Element, or self-hosted). Discover public rooms by keyword or scrape message history from specific rooms using a Matrix access token.", "version": "0.1", "x-build-id": "ennhogm0KARv3a0g9" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/jungle_synthesizer~matrix-message-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-jungle_synthesizer-matrix-message-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/jungle_synthesizer~matrix-message-scraper/runs": { "post": { "operationId": "runs-sync-jungle_synthesizer-matrix-message-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/jungle_synthesizer~matrix-message-scraper/run-sync": { "post": { "operationId": "run-sync-jungle_synthesizer-matrix-message-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "action" ], "properties": { "sp_intended_usage": { "title": "What is the intended usage of this data?", "minLength": 1, "type": "string", "description": "Please describe how you plan to use the data extracted by this crawler." }, "sp_improvement_suggestions": { "title": "How can we improve this crawler for you?", "minLength": 1, "type": "string", "description": "Provide any feedback or suggestions for improvements." }, "sp_contact": { "title": "Contact Email", "minLength": 1, "type": "string", "description": "Provide your email address so we can get in touch with you." }, "action": { "title": "Action", "enum": [ "Discover Public Rooms", "Scrape Room Messages" ], "type": "string", "description": "What to scrape:\n- **Discover Public Rooms** – list public rooms on the homeserver (no token required)\n- **Scrape Room Messages** – fetch message history for one or more rooms (access token required)\n", "default": "Discover Public Rooms" }, "homeserver": { "title": "Homeserver URL", "type": "string", "description": "Matrix homeserver base URL. Defaults to matrix.org.", "default": "https://matrix.org" }, "accessToken": { "title": "Access Token", "type": "string", "description": "Matrix access token. Required for 'Scrape Room Messages'. Get it from Element: Settings → Help & About → Access Token." }, "roomIds": { "title": "Room IDs", "type": "array", "description": "List of Matrix room IDs to scrape messages from (e.g. !abcdef:matrix.org). Used by 'Scrape Room Messages' action.", "items": { "type": "string" } }, "searchTerm": { "title": "Search Term (Public Room Filter)", "type": "string", "description": "Filter public rooms by name/topic keyword. Used by 'Discover Public Rooms' action. Leave blank to return all public rooms." }, "maxItems": { "title": "Max Items", "type": "integer", "description": "Maximum number of records to return across all rooms.", "default": 10 }, "maxMessagesPerRoom": { "title": "Max Messages Per Room", "type": "integer", "description": "Maximum number of messages to fetch per room. 0 = unlimited.", "default": 100 }, "messageDirection": { "title": "Message Direction", "enum": [ "b", "f" ], "type": "string", "description": "Pagination direction. 'b' (backward) fetches oldest messages first; 'f' (forward) fetches newest first.", "default": "b" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```