CoinDesk News Scraper

Scrape CoinDesk articles and crypto news with full article text, author information, publication dates, tags, and categories.

Features

• Full article extraction — title, subtitle, body text, author, date, tags, image
• Section filtering — browse Markets, Policy, Tech, or Consensus Magazine
• Search — find articles by keyword (e.g. "bitcoin ETF", "ethereum merge")
• JSON-LD extraction — uses CoinDesk's structured data for reliable parsing
• Batch processing — scrape up to 500 articles per run

Input

Field	Type	Description
searchQuery	string	Search term for articles. Leave empty to browse by section.
section	select	Filter by section: all, markets, policy, tech, consensus-magazine
maxResults	integer	Maximum articles to scrape (1-500, default 50)
proxyConfiguration	object	Optional proxy configuration

Output

Each article in the dataset contains:

{
    "title": "Bitcoin Falls After Trump Tariff Announcement",
    "subtitle": "The crypto market reacted sharply to new trade tensions",
    "author": "Will Canny",
    "authors": [
        {
            "name": "Will Canny",
            "jobTitle": "Finance Reporter",
            "url": "https://www.coindesk.com/author/will-canny"
        }
    ],
    "section": "markets",
    "publishedAt": "2026-04-25T18:53:48.734Z",
    "modifiedAt": "2026-04-25T19:00:36.313Z",
    "url": "https://www.coindesk.com/markets/2026/04/25/bitcoin-falls-...",
    "body": "Full article text here...",
    "bodyLength": 2847,
    "tags": ["Bitcoin", "Donald Trump", "Markets"],
    "image": "https://www.coindesk.com/resizer/...",
    "scrapedAt": "2026-04-25T20:00:00.000Z"
}

Output fields

Field	Description
title	Article headline
subtitle	Article subtitle/abstract
author	Author name(s), comma-separated
authors	Array of author objects with name, jobTitle, url
section	CoinDesk section (markets, policy, tech, etc.)
publishedAt	ISO 8601 publication timestamp
modifiedAt	ISO 8601 last modified timestamp
url	Full article URL
body	Full article body text
bodyLength	Character count of body text
tags	Array of article tags/topics
image	Featured image URL
scrapedAt	When the scrape was performed

Use cases

• Crypto market research — track news sentiment across Bitcoin, Ethereum, altcoins
• Regulatory tracking — monitor crypto policy and regulation news
• Content aggregation — build crypto news feeds and dashboards
• Sentiment analysis — analyze article text for market sentiment signals
• Historical research — collect article archives by topic or section

Example usage

Browse latest market news

{
    "section": "markets",
    "maxResults": 20
}

Search for Bitcoin ETF articles

{
    "searchQuery": "bitcoin ETF",
    "maxResults": 100
}

Scrape all sections

{
    "section": "all",
    "maxResults": 200
}

Notes

• CoinDesk uses Next.js with server-side rendering. The scraper extracts JSON-LD structured data from article pages for reliable data extraction.
• Rate limiting is built in to avoid overwhelming the server.
• Body text extraction uses multiple fallback strategies for maximum coverage.
• Articles are deduplicated by URL before scraping.