🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🚜

🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽🌽

Harvest Edge / Discord Message Scraper

🌱🌱🌱🌱🌱🌱🌱🌱🌱🌱🌱🌱🌱🌱🌱🌱

🌽🌽🌽🌽🍆🌽🌽🌽🌽🚜🌱🌱🌱🌱🌱🌱

Harvest Edge makes business information available to everyone!

Overview

Easily scrape Discord messages with the Discord Message Scraper, a powerful Apify actor designed for developers and data analysts. This actor extracts messages, attachments, embeds, reactions, and mentions from any Discord channel or guild, using the Discord API. With support for scraping in both “newer” and “older” directions and a customizable 60-second timeout, it ensures reliable data collection for analytics, research, or archiving. Feel free to contact us via Apify for feature requests or bug reports.

How to Get Your Token

• Open developer tools on your browser, by pressing F12, and go to discord.com/channels/@me
• Click the 'Network' tab in the developer tools, and find the file named @me. You can filter for Fetch/XHR requests to help find it. Refresh the page if you do not see it.
• Be sure you are looking at the 'Headers' of this file and scroll down to 'Request Headers'
• Your token is the code that is listed as 'Authorization'

Features

• Fetches up to 10,000 messages per run from a specified Discord channel.
• Supports scraping in batches, with the option of going either direction from a starting message.
• Outputs detailed message data (content, author, timestamp, attachments, embeds, etc.) to an Apify dataset.
• Configurable batch delay to manage Discord API rate limits.
• Robust error handling with clear logs for troubleshooting.

Performance

• Execution Time: Fetching 1000 messages takes ~10 seconds (with default 1.0-second batch delay). Fetching the maximum 10,000 messages takes ~100 seconds (~1.7 minutes).
• Resource Usage: Optimized for low compute unit consumption, with a maximum limit of 10,000 messages to prevent excessive resource use.

Input

The Actor accepts the following input parameters via the Apify platform:

Key	Type	Description	Default	Required
guild_id	String	The ID of the Discord guild (server) containing the channel.	-	Yes
channel_id	String	The ID of the Discord channel to fetch messages from.	-	Yes
token	String	Your Discord user account token for authentication. Ensure compliance with Discord's Terms of Service.	-	Yes
max_messages	Integer	Limit the number of messages to fetch (1 to 10,000). Use 100 for quick tests or up to 10,000 for large channels.	1000	No
start_message_id	String	Message ID to start scraping from. For "newer", scrapes messages before this ID; for "older", scrapes messages after this ID.	-	No
scrape_direction	String	Direction to scrape: "newer" (towards past) or "older" (towards future).	Newer	No
batch_delay_seconds	String	Delay between batches of 100 messages (non-negative number, in seconds). Increase to 2.0 if rate limit errors occur.	1.0	No

Output

The Actor outputs a dataset in JSON format with the following fields for each product:

Field	Description
guild_id	Unique ID of the message.
channel_id	ID of the channel.
author	Object with author details (ID, username, discriminator, avatar, public flags).
content	Message text content.
timestamp	Creation timestamp (YYYY-MM-DD HH:MM:SS).
edited_timestamp	Edit timestamp, if edited (YYYY-MM-DD HH:MM:SS).
type	Message type (e.g., default, reply).
attachments	Array of attachment objects (ID, filename, size, URL, etc.).
embeds	Array of embed objects (type, URL, title, description, etc.).
pinned	Boolean indicating if the message is pinned.
mention_everyone	Boolean indicating if the message mentions everyone.
tts	Boolean indicating if the message is text-to-speech.
flags	Message flags (integer).
components	Array of message components (e.g., buttons).
reactions	Array of reaction objects (emoji, count, etc.).
mentions	Array of mentioned user objects (ID, username, discriminator).
message_reference	Object with reply details (message ID, channel ID, guild ID), if a reply.

Example Output:

{
  "message_id": "123456789012345678",
  "channel_id": "987654321098765432",
  "author": {
    "id": "111222333444555666",
    "username": "user",
    "discriminator": "1234",
    "avatar": "https://cdn.discordapp.com/avatars/...",
    "public_flags": 0
  },
  "content": "Hello, world!",
  "timestamp": "2025-10-08 14:30:00",
  "edited_timestamp": null,
  "type": 0,
  "attachments": [],
  "embeds": [],
  "pinned": false,
  "mention_everyone": false,
  "tts": false,
  "flags": 0,
  "components": [],
  "reactions": [],
  "mentions": [],
  "message_reference": null
}

Troubleshooting

• Invalid Token: Ensure the token is correct and not expired. Obtain a new token if needed.
• Message ID Not Found: Verify the start_message_id exists in the specified channel and is accessible.
• Rate Limit Errors: Increase batch_delay_seconds to 2.0 or higher if you see HTTPException errors.
• Timeout Errors: If the Actor times out after 60 seconds while fetching the starting message, check the start_message_id and network connection, or contact support for assistance.
• No Messages Fetched: Ensure the channel has messages and the token has permission to read the channel’s history.