ChatGPT Conversation Scraper

Extract complete conversations from ChatGPT shared links with smart Pay-Per-Event pricing. Get full dialogues with code blocks and metadata - perfect for training datasets, conversation analysis, and knowledge management.

What is ChatGPT Conversation Scraper?

ChatGPT Conversation Scraper is a powerful data extraction tool that automatically captures complete conversations from publicly shared ChatGPT links. Built for reliability with smart pricing that scales with conversation size.

Why use ChatGPT Conversation Scraper?

For AI Developers & Researchers

• Build training datasets from real ChatGPT interactions
• Analyze conversation patterns and user behavior
• Create knowledge bases from expert conversations

For Businesses & Content Teams

• Archive important ChatGPT consultations
• Convert conversations into documentation
• Analyze customer support interactions

For Educators & Students

• Collect educational conversations for study materials
• Archive coding tutorials and explanations
• Build reference libraries from expert discussions

✨ Features

• 🎯 Complete Conversation Extraction: Captures full conversations from ChatGPT shared links
• 👥 Role Detection: Identifies user vs assistant messages
• 💻 Code Block Extraction: Automatically extracts code blocks with language detection
• 🔄 Fallback Strategies: Handles different ChatGPT page layouts
• 🛡️ Anti-Blocking: Uses residential proxies to avoid detection
• ⚙️ Configurable Settings: Adjust timeouts and extraction parameters
• 📊 Structured Output: Clean JSON/CSV format with metadata

💰 Smart Pricing Model

Our Pay-Per-Event pricing automatically adjusts based on conversation size, ensuring you only pay for what you actually extract.

Conversation-Based Pricing

Conversation Size	Messages	Typical Length	Price per Conversation
Small	1-10	< 10K chars	$0.05
Medium	11-30	< 50K chars	$0.10
Large	31-100	< 200K chars	$0.20
Extra Large	100+	200K+ chars	$0.35

Additional Charges

Feature	Description	Additional Cost
HTML Content	Preserve original HTML of messages (optional)	+$0.02
Base Processing Fee	One-time fee per run	$0.003

Note: Code block extraction is included automatically at no extra charge

Volume Discounts (Automatic)

Your Apify Plan	Discount	Effective Price (Medium)
Free	Standard pricing	$0.10
Starter	10%	$0.09
Scale	20%	$0.08
Business	30%	$0.07

Cost Examples

• Research Dataset (100 small conversations): ~$5.00
• Training Data (50 medium, 20 large): ~$9.00
• Full Archive (500 mixed conversations): ~$35-50

Important Note on Pricing

• You're charged even for failed attempts (invalid/expired URLs) as the scraper still uses resources
• The base processing fee applies once per run, regardless of URL count
• Prices adjust automatically based on your Apify subscription tier

🚀 How to Use

1. Create a free Apify account (Sign up here)
2. Open ChatGPT Conversation Scraper in Apify Store
3. Add conversation URLs - Paste one or more shared ChatGPT links
4. Configure settings (optional) - Adjust extraction parameters
5. Click "Start" - The scraper will extract all conversations
6. Download your data - Get results in JSON, CSV, Excel, or via API

⏱️ Average extraction time: 15-30 seconds per conversation (varies by size)

📥 Input Configuration

Required Fields

• startUrls: List of ChatGPT shared conversation URLs
  • Format: https://chatgpt.com/share/[conversation-id]
  • Example: https://chatgpt.com/share/68c85745-bda4-800c-8922-7398158c52f3

Optional Settings

• proxyConfiguration: Proxy settings (residential recommended for best results)
• maxRequestsPerCrawl: Maximum conversations to process (default: 100)
• includeHtmlContent: Store original HTML of messages (+$0.02 per conversation)
• enableCostTracking: Show detailed pricing breakdown in results
• debugMode: Enable detailed extraction logs for troubleshooting
• maxScrollAttempts: Number of scroll attempts to load full conversation (default: 15)
• navigationTimeout: Maximum time to wait for page load in milliseconds (default: 30000)

📤 Output Format

The scraper outputs structured JSON data with extracted conversations:

{
  "url": "https://chatgpt.com/share/conversation-id",
  "conversationId": "conversation-id", 
  "success": true,
  "messages": [
    {
      "role": "user",
      "content": "Full message text content",
      "messageIndex": 1,
      "timestamp": "2024-01-15T10:30:00.000Z",
      "codeBlocks": [
        {
          "type": "code",
          "language": "python",
          "content": "def hello():\n    print('Hello World')"
        }
      ]
    },
    {
      "role": "assistant",
      "content": "Response message text",
      "messageIndex": 2,
      "timestamp": "2024-01-15T10:30:15.000Z"
    }
  ],
  "metadata": {
    "messageCount": 2,
    "extractedAt": "2024-01-15T10:30:30.000Z"
  },
  "classification": "small",
  "pricing": {
    "conversationSize": "small",
    "charged": {
      "total": 0.053
    },
    "userTier": "Free"
  }
}

What You Get

• Message content: Complete text of each message
• Role identification: Whether each message is from user or assistant
• Code blocks: Automatically extracted with language identification (included by default)
• Message order: Sequential index for each message
• Metadata: Extraction timestamp and message count
• Pricing info: Breakdown of charges (if cost tracking enabled)

🔧 Use Cases

🎓 Education & Training

• Create study materials from expert ChatGPT sessions
• Build FAQ databases from support conversations
• Archive coding tutorials and explanations

🤖 AI Development

• Generate training data for custom chatbots
• Analyze prompt engineering techniques
• Build conversation datasets for fine-tuning

📚 Knowledge Management

• Convert conversations into searchable documentation
• Archive important consultations and decisions
• Create knowledge bases from expert discussions

📊 Research & Analytics

• Study conversation patterns and user behavior
• Analyze response quality and length
• Track topic trends across conversations

🔌 Integrations

ChatGPT Conversation Scraper can be integrated with:

• 📊 Data Processing: Export to JSON, CSV, Excel
• 🔄 Automation: Zapier, Make, n8n webhooks
• 💾 Storage: Direct dataset access via Apify API
• 📝 Analysis: Import into any data analysis tool

API Access

Access your scraped data programmatically:

JavaScript

const response = await fetch('https://api.apify.com/v2/datasets/[DATASET_ID]/items');
const data = await response.json();

Python

from apify_client import ApifyClient
client = ApifyClient('[YOUR_API_TOKEN]')
items = client.dataset('[DATASET_ID]').list_items().items

⚠️ Limitations & Known Issues

• Only works with shared links: Cannot access private conversations or your ChatGPT account
• Layout changes: ChatGPT UI updates may temporarily affect extraction
• Rate limiting: High-volume scraping may trigger ChatGPT's rate limits
• Expired links: Some shared links may expire or be deleted
• Large conversations: Very long conversations (500+ messages) may timeout

❓ FAQ

Is it legal to scrape ChatGPT conversations?

We only extract publicly shared conversations that users have explicitly made available via share links. Always respect copyright and privacy regulations.

What's the actual success rate?

Success depends on valid URLs and ChatGPT's current layout. Invalid or expired links will fail but still incur charges.

Why am I charged for failed extractions?

Even failed attempts consume resources (browser instance, proxy, compute time). The charge ensures fair usage.

Can I test before paying?

Yes! Apify's free tier includes credits to test the scraper before upgrading.

How accurate is role detection?

The scraper uses multiple methods to identify roles, but may occasionally misclassify in complex conversations.

What if ChatGPT changes their layout?

We monitor for changes and update the scraper, but temporary disruptions may occur.

Are code blocks extracted automatically?

Yes, code blocks are automatically detected and extracted with language identification at no extra charge.

🆘 Support

For issues or questions:

• Check the run log in Apify Console for error details
• Review failed URLs to ensure they're valid shared links
• Contact support through Apify Console

🔐 Privacy & Data Handling

• We don't store your conversations - data goes directly to your Apify dataset
• Only public shared links are processed
• Your API credentials remain secure in your Apify account
• Extracted data is retained according to your Apify plan

⚖️ Terms of Use

By using this scraper, you agree to:

• Only scrape publicly shared conversations
• Respect ChatGPT's terms of service
• Handle extracted data responsibly
• Not use the tool for any illegal purposes

---

Start extracting ChatGPT conversations with transparent, size-based pricing!