X Articles Scraper

Extract public long-form X Articles as clean Markdown and structured JSON. Give the Actor x.com status URLs or post IDs and receive the full rich-text body, author details, publication timestamps, links, inline media, and current engagement metadata—without supplying an X account or developer API key.

Unlike an ordinary tweet scraper, this Actor reads the Article rich-text record behind a post. It preserves headings, lists, quotes, emphasis, hyperlinks, cover images, and inline images instead of returning only the short Article preview.

What does X Articles Scraper do?

The Actor accepts public X/Twitter post URLs and numeric post IDs. For every post that contains a public long-form Article, it:

1. obtains an anonymous X guest session;
2. retrieves the Article's structured rich-text payload;
3. converts rich-text blocks and entities into Markdown;
4. normalizes author, publication, media, link, and engagement data;
5. saves one typed row in the default Apify dataset.

Ordinary posts, deleted posts, private posts, and Articles unavailable to logged-out visitors do not produce result rows and do not incur the per-Article event charge.

Who is this X Articles extractor for?

• Researchers archiving cited long-form social content with a stable source URL.
• Content analysts comparing topics, structure, links, and engagement across Articles.
• RAG and AI teams building Markdown corpora from selected public X Articles.
• Knowledge managers importing long-form posts into Notion, Obsidian, or a CMS.
• Journalists and fact-checkers preserving a readable copy alongside source metadata.
• Developers who need a JSON API without maintaining X guest-token and GraphQL logic.

Why use this Actor?

• Full Article bodies: not just tweet text or a preview snippet.
• Useful Markdown: headings, lists, quotes, emphasis, links, and images are preserved.
• Structured metadata: author, dates, media, links, language, and engagement travel with the text.
• No X login required: the default workflow uses only data exposed to logged-out visitors.
• Success-only item billing: ordinary or unavailable posts are skipped without an item charge.
• Batch ready: combine URLs and IDs, automatically deduplicate them, and process up to 100 per run.
• Low overhead: direct HTTP is the default; there is no automatic browser or residential-proxy fallback.

What data can I extract?

Field	Meaning
tweetId	ID of the X post containing the Article
articleId	Stable Article identifier exposed by X
url	Canonical public post URL
title	Full Article title
markdown	Complete body converted to Markdown
plainText	Complete body without Markdown syntax
previewText	Short preview supplied by X
language	Language code reported for the containing post
author	Public user ID, username, display name, profile URL, image, follower count, and verification when available
publishedAt	First publication timestamp when available
modifiedAt	Last Article modification timestamp when available
coverImageUrl	Original-resolution cover image
media	Cover and inline media URLs with dimensions
links	Link text and resolved destinations from the body
engagement	Likes, replies, reposts, quotes, bookmarks, and views when exposed
scrapedAt	Time the row was extracted

Upstream fields can disappear or be withheld. Optional fields may therefore be absent even when the Article body is available.

How to extract X Articles in 3 steps

1. Open the Actor input page.
2. Paste one or more public x.com status URLs into X Article or post URLs, or add numeric IDs under Post IDs.
3. Click Start and open the default dataset when the run finishes.

A working input is:

{
  "startUrls": [
    { "url": "https://x.com/thedankoe/status/2010751592346030461" }
  ],
  "maxItems": 1
}

Input parameters

startUrls

Public x.com or twitter.com status URLs. Both forms are accepted:

https://x.com/username/status/2010751592346030461
https://twitter.com/username/status/2010751592346030461

tweetIds

Numeric post IDs for systems that already store X identifiers. URLs and IDs can be used together. Duplicate IDs are removed before requests begin.

maxItems

Maximum number of unique posts inspected and Article rows saved. The allowed range is 1–100 and the default is 20. If the input has more unique IDs, later IDs are not requested.

requestDelayMs

Delay between X requests, from 0 to 10,000 milliseconds. The default is 800 ms. Keep the default for normal use; increase it if a larger run encounters rate limits.

proxyConfiguration

Optional Apify Proxy settings. Direct access is used when no proxy is enabled. The Actor does not silently switch to residential traffic. If you explicitly enable a proxy, its transfer cost is part of your platform usage.

Output example

A representative row looks like this (long text and arrays are shortened):

{
  "tweetId": "2010751592346030461",
  "articleId": "2010742786430021632",
  "url": "https://x.com/thedankoe/status/2010751592346030461",
  "title": "How to fix your entire life in 1 day",
  "markdown": "If you're anything like me...\n\n## I – The Default Path...",
  "plainText": "If you're anything like me...",
  "previewText": "If you're anything like me, you think new years resolutions are stupid...",
  "author": {
    "id": "256523056",
    "username": "thedankoe",
    "name": "DAN KOE",
    "profileUrl": "https://x.com/thedankoe",
    "verified": true
  },
  "publishedAt": "2026-01-12T16:31:52.000Z",
  "coverImageUrl": "https://pbs.twimg.com/media/G-efypCbQAEEi8a.jpg",
  "media": [
    { "url": "https://pbs.twimg.com/media/G-efypCbQAEEi8a.jpg", "isCover": true }
  ],
  "links": [
    { "text": "my newsletter", "url": "https://letters.thedankoe.com" }
  ],
  "engagement": {
    "likes": 321422,
    "replies": 8907,
    "reposts": 49977,
    "quotes": 15038,
    "bookmarks": 832720
  },
  "scrapedAt": "2026-07-30T07:10:00.000Z"
}

Engagement values are a snapshot and can change after extraction.

How is rich text converted to Markdown?

The converter maps X Article blocks into familiar Markdown:

• Article headers become #, ##, ###, or #### headings.
• Ordered and unordered list items retain their list form and depth.
• Quotes become Markdown blockquotes.
• Bold, italic, strikethrough, underline, and monospace styles are preserved.
• Rich-text links become Markdown links with resolved destinations.
• Article media entities become inline Markdown images using original media URLs.
• Code blocks are fenced.

The separate plainText field omits Markdown syntax for search indexing, embeddings, or NLP pipelines.

How much does it cost to extract X Articles?

The Actor uses pay-per-event pricing: a $0.0005 start fee plus one X Article event for each successfully saved row. The current per-Article tiers are:

Tier	Price per successful Article
Free	$0.0055108
Bronze	$0.004792
Silver	$0.0037378
Gold	$0.0028752
Platinum	$0.0019168
Diamond	$0.0013418

At the Free-tier event rate, one Article costs about $0.0060, 10 cost about $0.0556, and 100 cost about $0.5516, excluding any optional proxy and ordinary platform usage shown by Apify. Posts that do not return a complete public Article have no per-Article event charge.

Batch archiving and content-analysis workflows

Archive selected sources

Store X Article URLs from a reading list, run this Actor on a schedule, and save the dataset to cloud storage. Use articleId, modifiedAt, and the Markdown body to detect later changes in your own downstream workflow.

Build a multilingual RAG corpus

Submit selected public Article IDs in one batch, ingest markdown or plainText, and retain url, author, and publication fields as citation metadata.

Analyze outbound sources

Flatten the links arrays to identify domains and sources cited across an Article collection. Media URLs are returned separately for systems that need image-aware analysis.

Compare content and engagement

Join Article text with engagement snapshots to study how structure, topic, or link use relates to public interactions. Do not treat engagement as immutable.

Tips for reliable runs

• Test URLs in a logged-out browser; private or subscriber-only content may not be public.
• Prefer status URLs or the ID of the post that shares the Article, not an unrelated repost.
• Keep the default delay for ordinary batches.
• Increase requestDelayMs before adding a proxy when rate limiting is temporary.
• Use smaller scheduled batches rather than repeatedly requesting the same large list.
• Store source IDs because display names and usernames can change.
• Treat media URLs as external references; X controls their lifetime and availability.

Limits and failure behavior

• The Actor extracts selected public Articles; it does not search X or discover an author's Articles.
• A run accepts at most 100 unique post IDs.
• Deleted, protected, login-gated, geographic-restricted, and ordinary non-Article posts are skipped.
• If X changes its public guest API, a run can fail until the integration is updated.
• X may omit follower counts, views, language, dates, or other optional metadata.
• The Actor does not download media files or create permanent media copies.
• The Actor does not bypass a login, paywall, CAPTCHA, or access restriction.

Malformed inputs and terminal upstream errors fail the run with a non-zero status instead of returning a misleading successful empty result.

Use the Apify API with cURL

Start a synchronous run and receive dataset rows:

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~x-articles-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "tweetIds": ["2010751592346030461"],
    "maxItems": 1
  }'

Keep tokens in environment variables or a secrets manager rather than source code.

JavaScript API example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/x-articles-scraper').call({
  startUrls: [{ url: 'https://x.com/thedankoe/status/2010751592346030461' }],
  maxItems: 1,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0].title, items[0].markdown);

Python API example

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/x-articles-scraper').call(run_input={
    'tweetIds': ['2010751592346030461'],
    'maxItems': 1,
})

for article in client.dataset(run['defaultDatasetId']).iterate_items():
    print(article['title'])
    print(article['markdown'][:500])

Use X Articles Scraper with Apify MCP

Add this Actor to Claude Code through Apify's hosted MCP server:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/x-articles-scraper"

Claude Desktop

Add the following HTTP MCP server in Claude Desktop's MCP configuration.

Cursor

Use the same server object in Cursor's MCP settings.

VS Code

Add the server URL to your VS Code MCP configuration. All three clients use this equivalent JSON:

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/x-articles-scraper"
    }
  }
}

Example prompts:

• “Extract this public X Article and return its Markdown with citations.”
• “Convert these two X post IDs into a multilingual research dataset.”
• “List the outbound links and media referenced in this X Article.”
• “Summarize this Article, but preserve its author and source URL.”

Responsible and legal use

This Actor accesses data that X exposes to logged-out visitors. Public availability does not automatically grant every reuse right. You are responsible for complying with applicable law, X's terms, copyright, privacy, database rights, and the rules of the jurisdiction where you operate.

Use reasonable request volumes. Do not use the Actor for harassment, surveillance, unauthorized profiling, spam, or attempts to access protected content. When republishing or quoting an Article, preserve attribution and check whether your use requires permission from the author or rights holder.

Troubleshooting

Why did a successful run return no rows?

The supplied posts were available but did not expose complete public long-form Articles. Confirm that each status opens an Article—not an ordinary tweet, thread, Note, link card, or repost—in a logged-out browser.

Why was one URL skipped while others succeeded?

The skipped post may be deleted, protected, region-limited, ordinary, or temporarily missing its rich-content payload. The Actor saves and charges only complete Articles.

What should I do after an X rate-limit error?

Wait before retrying, keep the default delay, or increase requestDelayMs. For a persistent network-specific block, explicitly configure Apify Proxy. The Actor does not automatically select residential traffic.

Why are follower counts or views missing?

X does not always expose every optional profile and engagement field to anonymous sessions. The Article text can still be complete.

Does this Actor scrape all Articles from a profile?

No. It processes selected status URLs or post IDs. It does not perform profile discovery or X search.

Can it extract ordinary tweets or threads?

No. Use Twitter/X Scraper for ordinary posts, profiles, timelines, and supported tweet workflows.

Related automation-lab Actors

• Twitter/X Scraper — ordinary tweets, profiles, timelines, and supported search workflows.
• News & Article Extractor — discover and normalize articles from websites, RSS feeds, and sitemaps.
• Webpage Text Extractor — extract clean text from general public web pages.

FAQ

Do I need an X developer API key?

No. The Actor uses X's public logged-out guest flow.

Do I need to provide X cookies?

No. User cookies are not accepted or required.

Is each media image charged separately?

No. Media metadata and URLs are included in the successful X Article result and have no separate event charge.

Are duplicate IDs charged twice?

No. IDs are deduplicated before requests begin.

Does maxItems count skipped ordinary posts?

It limits unique posts inspected and successful rows saved. A skipped post consumes an input position but does not incur an item event.

Can I export to CSV, Excel, or JSON?

Yes. Use the default dataset's export controls or API endpoints. Markdown remains a string field in each exported record.

Does the Actor store my input outside Apify?

The Actor uses normal Apify run storage. It does not send your input to a separate third-party scraping service.

Is Markdown guaranteed to render identically everywhere?

No. Markdown renderers vary, especially for underline and nested rich-text styles. The Actor preserves semantic structure and also provides plainText for consumers that do not want Markdown syntax.