Discord Message Scraper

An Apify actor designed to scrape messages and channel data from Discord. This tool allows you to extract message history, guild channel lists, and other structured data from Discord servers and channels using a user or bot token.

---

Features

• Scrape Messages: Fetch up to a specified limit of messages from a Discord channel with support for cursor-based pagination.
• Guild Channel List: Retrieve all channels within a guild, including metadata.
• Guild Message Scraping: Collect messages from all text channels in a guild, with customizable limits.
• Cursor Management: Automatically saves and reloads cursors to ensure continuity between scrapes.
• Error Handling: Handles permission errors gracefully and provides detailed error logs.

---

Use Cases

• Data Analysis: Gather historical data for text analysis, sentiment tracking, or other insights.
• Community Management: Monitor conversations or archive messages for compliance.
• Content Backup: Maintain a backup of important conversations and shared media.
• Guild Mapping: Extract the structure of a Discord server, including its channels and metadata.

---

How It Works

1. Input Configuration: The actor takes input parameters like channel ID, Discord token, action type, and optional pagination cursor.
2. Action Selection: Based on the action parameter, the scraper performs one of the following:
   • Fetch channel messages.
   • Retrieve all guild channels.
   • Collect messages from all guild channels.
3. Cursor Management: Saves and retrieves cursors for seamless pagination.
4. Data Processing: Formats raw Discord data into structured JSON with metadata, attachments, and timestamps.
5. Output: Pushes the processed data to the Apify dataset for further use.

---

Input Schema

The scraper accepts a JSON object with the following parameters:

Field	Type	Required	Default	Description
channelId	string	Yes	""	The ID of the Discord channel to scrape.
token	string	Yes	""	Discord user or bot token. Use carefully (may violate TOS).
limit	number	No	100	Maximum number of messages to fetch (per channel in guild mode).
action	string	Yes	""	Action to perform: "Scrape Messages", "Scrape Channel List for Guild", "Scrape Messages for Guild".
cursorType	string	No	"after"	Cursor direction: "before" or "after".
cursor	string	No	null	Optional user-specified cursor for pagination.

---

Example Input

1{
2  "channelId": "123456789012345678",
3  "token": "YOUR_DISCORD_TOKEN",
4  "limit": 50,
5  "action": "Scrape Messages",
6  "cursorType": "after",
7  "cursor": null
8}

---

Output

The actor outputs structured JSON data to the Apify dataset, formatted as:

Scrape Messages

1[
2    {
3        "type": 0,
4        "content": "Give me a fun picture",
5        "mentions": [],
6        "mention_roles": [],
7        "attachments": [],
8        "embeds": [],
9        "timestamp": "2023-10-29T02:03:46.247000+00:00",
10        "edited_timestamp": null,
11        "flags": 0,
12        "components": [],
13        "resolved": { "users": {}, "members": {}, "channels": {}, "roles": {} },
14        "id": "1168007238766559311",
15        "channel_id": "1143396487057899611",
16        "author": {
17            "id": "936929561302675456",
18            "username": "Midjourney Bot",
19            "avatar": "f6ce562a6b4979c4b1cbc5b436d3be76",
20            "discriminator": "9282",
21            "public_flags": 589824,
22            "flags": 589824,
23            "bot": true,
24            "banner": null,
25            "accent_color": null,
26            "global_name": null,
27            "avatar_decoration_data": null,
28            "banner_color": null,
29            "clan": null,
30            "primary_guild": null
31        },
32        "pinned": false,
33        "mention_everyone": false,
34        "tts": false
35  }
36]

Scrape Channel List for Guild

1[
2    {
3        "id": "1143396487057899613",
4        "type": 2,
5        "last_message_id": null,
6        "flags": 0,
7        "guild_id": "1143396486411984942",
8        "name": "Best Channel Ever",
9        "parent_id": "1143396487057899611",
10        "rate_limit_per_user": 0,
11        "bitrate": 64000,
12        "user_limit": 0,
13        "rtc_region": null,
14        "position": 0,
15        "permission_overwrites": [],
16        "nsfw": false
17    },
18    {
19        "id": "1143396487057899613",
20        "type": 2,
21        "last_message_id": null,
22        "flags": 0,
23        "guild_id": "1143396486411984943",
24        "name": "Even Better Channel",
25        "parent_id": "1143396487057899613",
26        "rate_limit_per_user": 0,
27        "bitrate": 64000,
28        "user_limit": 0,
29        "rtc_region": null,
30        "position": 0,
31        "permission_overwrites": [],
32        "nsfw": false
33    }
34]

---

Example Output

• Messages: List of all scraped messages with metadata and attachments.
• Channels: Array of guild channels with IDs, names, and types.

---

Limitations

• Discord TOS: Using a user token may violate Discord’s Terms of Service. Use responsibly.
• Rate Limits: The scraper is subject to Discord API rate limits and permissions.
• Data Size: Large guilds or channels may require multiple runs due to API limitations.

---

Notes

• Customization: The scraper can be extended to support additional data processing or Discord endpoints. Please reach out.

---

Disclaimer

This project is not affiliated with or endorsed by Discord. Use responsibly and adhere to Discord’s API Terms of Service.