# X Articles Scraper (`automation-lab/x-articles-scraper`) Actor

Extract public long-form X Articles as clean Markdown with author, media, links, publication, and engagement metadata.

- **URL**: https://apify.com/automation-lab/x-articles-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Social media, AI
- **Stats:** 4 total users, 3 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.88 / 1,000 x articles

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## X Articles Scraper

Extract public long-form **X Articles** as clean Markdown and structured JSON.
Give the Actor x.com status URLs or post IDs and receive the full rich-text body,
author details, publication timestamps, links, inline media, and current engagement
metadata—without supplying an X account or developer API key.

Unlike an ordinary tweet scraper, this Actor reads the Article rich-text record
behind a post. It preserves headings, lists, quotes, emphasis, hyperlinks, cover
images, and inline images instead of returning only the short Article preview.

### What does X Articles Scraper do?

The Actor accepts public X/Twitter post URLs and numeric post IDs. For every post
that contains a public long-form Article, it:

1. obtains an anonymous X guest session;
2. retrieves the Article's structured rich-text payload;
3. converts rich-text blocks and entities into Markdown;
4. normalizes author, publication, media, link, and engagement data;
5. saves one typed row in the default Apify dataset.

Ordinary posts, deleted posts, private posts, and Articles unavailable to logged-out
visitors do not produce result rows and do not incur the per-Article event charge.

### Who is this X Articles extractor for?

- **Researchers** archiving cited long-form social content with a stable source URL.
- **Content analysts** comparing topics, structure, links, and engagement across Articles.
- **RAG and AI teams** building Markdown corpora from selected public X Articles.
- **Knowledge managers** importing long-form posts into Notion, Obsidian, or a CMS.
- **Journalists and fact-checkers** preserving a readable copy alongside source metadata.
- **Developers** who need a JSON API without maintaining X guest-token and GraphQL logic.

### Why use this Actor?

- **Full Article bodies:** not just tweet text or a preview snippet.
- **Useful Markdown:** headings, lists, quotes, emphasis, links, and images are preserved.
- **Structured metadata:** author, dates, media, links, language, and engagement travel with the text.
- **No X login required:** the default workflow uses only data exposed to logged-out visitors.
- **Success-only item billing:** ordinary or unavailable posts are skipped without an item charge.
- **Batch ready:** combine URLs and IDs, automatically deduplicate them, and process up to 100 per run.
- **Low overhead:** direct HTTP is the default; there is no automatic browser or residential-proxy fallback.

### What data can I extract?

| Field | Meaning |
| --- | --- |
| `tweetId` | ID of the X post containing the Article |
| `articleId` | Stable Article identifier exposed by X |
| `url` | Canonical public post URL |
| `title` | Full Article title |
| `markdown` | Complete body converted to Markdown |
| `plainText` | Complete body without Markdown syntax |
| `previewText` | Short preview supplied by X |
| `language` | Language code reported for the containing post |
| `author` | Public user ID, username, display name, profile URL, image, follower count, and verification when available |
| `publishedAt` | First publication timestamp when available |
| `modifiedAt` | Last Article modification timestamp when available |
| `coverImageUrl` | Original-resolution cover image |
| `media` | Cover and inline media URLs with dimensions |
| `links` | Link text and resolved destinations from the body |
| `engagement` | Likes, replies, reposts, quotes, bookmarks, and views when exposed |
| `scrapedAt` | Time the row was extracted |

Upstream fields can disappear or be withheld. Optional fields may therefore be
absent even when the Article body is available.

### How to extract X Articles in 3 steps

1. Open the Actor input page.
2. Paste one or more public x.com status URLs into **X Article or post URLs**, or add numeric IDs under **Post IDs**.
3. Click **Start** and open the default dataset when the run finishes.

A working input is:

```json
{
  "startUrls": [
    { "url": "https://x.com/thedankoe/status/2010751592346030461" }
  ],
  "maxItems": 1
}
```

### Input parameters

#### `startUrls`

Public `x.com` or `twitter.com` status URLs. Both forms are accepted:

```text
https://x.com/username/status/2010751592346030461
https://twitter.com/username/status/2010751592346030461
```

#### `tweetIds`

Numeric post IDs for systems that already store X identifiers. URLs and IDs can
be used together. Duplicate IDs are removed before requests begin.

#### `maxItems`

Maximum number of unique posts inspected and Article rows saved. The allowed
range is 1–100 and the default is 20. If the input has more unique IDs, later IDs
are not requested.

#### `requestDelayMs`

Delay between X requests, from 0 to 10,000 milliseconds. The default is 800 ms.
Keep the default for normal use; increase it if a larger run encounters rate limits.

#### `proxyConfiguration`

Optional Apify Proxy settings. Direct access is used when no proxy is enabled.
The Actor does not silently switch to residential traffic. If you explicitly
enable a proxy, its transfer cost is part of your platform usage.

### Output example

A representative row looks like this (long text and arrays are shortened):

```json
{
  "tweetId": "2010751592346030461",
  "articleId": "2010742786430021632",
  "url": "https://x.com/thedankoe/status/2010751592346030461",
  "title": "How to fix your entire life in 1 day",
  "markdown": "If you're anything like me...\n\n## I – The Default Path...",
  "plainText": "If you're anything like me...",
  "previewText": "If you're anything like me, you think new years resolutions are stupid...",
  "author": {
    "id": "256523056",
    "username": "thedankoe",
    "name": "DAN KOE",
    "profileUrl": "https://x.com/thedankoe",
    "verified": true
  },
  "publishedAt": "2026-01-12T16:31:52.000Z",
  "coverImageUrl": "https://pbs.twimg.com/media/G-efypCbQAEEi8a.jpg",
  "media": [
    { "url": "https://pbs.twimg.com/media/G-efypCbQAEEi8a.jpg", "isCover": true }
  ],
  "links": [
    { "text": "my newsletter", "url": "https://letters.thedankoe.com" }
  ],
  "engagement": {
    "likes": 321422,
    "replies": 8907,
    "reposts": 49977,
    "quotes": 15038,
    "bookmarks": 832720
  },
  "scrapedAt": "2026-07-30T07:10:00.000Z"
}
```

Engagement values are a snapshot and can change after extraction.

### How is rich text converted to Markdown?

The converter maps X Article blocks into familiar Markdown:

- Article headers become `#`, `##`, `###`, or `####` headings.
- Ordered and unordered list items retain their list form and depth.
- Quotes become Markdown blockquotes.
- Bold, italic, strikethrough, underline, and monospace styles are preserved.
- Rich-text links become Markdown links with resolved destinations.
- Article media entities become inline Markdown images using original media URLs.
- Code blocks are fenced.

The separate `plainText` field omits Markdown syntax for search indexing,
embeddings, or NLP pipelines.

### How much does it cost to extract X Articles?

The Actor uses pay-per-event pricing: a **$0.0005 start fee** plus one `X Article`
event for each successfully saved row. The current per-Article tiers are:

| Tier | Price per successful Article |
| --- | ---: |
| Free | $0.0055108 |
| Bronze | $0.004792 |
| Silver | $0.0037378 |
| Gold | $0.0028752 |
| Platinum | $0.0019168 |
| Diamond | $0.0013418 |

At the Free-tier event rate, one Article costs about **$0.0060**, 10 cost about
**$0.0556**, and 100 cost about **$0.5516**, excluding any optional proxy and
ordinary platform usage shown by Apify. Posts that do not return a complete
public Article have no per-Article event charge.

### Batch archiving and content-analysis workflows

#### Archive selected sources

Store X Article URLs from a reading list, run this Actor on a schedule, and save
the dataset to cloud storage. Use `articleId`, `modifiedAt`, and the Markdown body
to detect later changes in your own downstream workflow.

#### Build a multilingual RAG corpus

Submit selected public Article IDs in one batch, ingest `markdown` or `plainText`,
and retain `url`, author, and publication fields as citation metadata.

#### Analyze outbound sources

Flatten the `links` arrays to identify domains and sources cited across an Article
collection. Media URLs are returned separately for systems that need image-aware
analysis.

#### Compare content and engagement

Join Article text with `engagement` snapshots to study how structure, topic, or
link use relates to public interactions. Do not treat engagement as immutable.

### Tips for reliable runs

- Test URLs in a logged-out browser; private or subscriber-only content may not be public.
- Prefer status URLs or the ID of the post that shares the Article, not an unrelated repost.
- Keep the default delay for ordinary batches.
- Increase `requestDelayMs` before adding a proxy when rate limiting is temporary.
- Use smaller scheduled batches rather than repeatedly requesting the same large list.
- Store source IDs because display names and usernames can change.
- Treat media URLs as external references; X controls their lifetime and availability.

### Limits and failure behavior

- The Actor extracts selected public Articles; it does not search X or discover an author's Articles.
- A run accepts at most 100 unique post IDs.
- Deleted, protected, login-gated, geographic-restricted, and ordinary non-Article posts are skipped.
- If X changes its public guest API, a run can fail until the integration is updated.
- X may omit follower counts, views, language, dates, or other optional metadata.
- The Actor does not download media files or create permanent media copies.
- The Actor does not bypass a login, paywall, CAPTCHA, or access restriction.

Malformed inputs and terminal upstream errors fail the run with a non-zero status
instead of returning a misleading successful empty result.

### Use the Apify API with cURL

Start a synchronous run and receive dataset rows:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~x-articles-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "tweetIds": ["2010751592346030461"],
    "maxItems": 1
  }'
```

Keep tokens in environment variables or a secrets manager rather than source code.

### JavaScript API example

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/x-articles-scraper').call({
  startUrls: [{ url: 'https://x.com/thedankoe/status/2010751592346030461' }],
  maxItems: 1,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0].title, items[0].markdown);
```

### Python API example

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/x-articles-scraper').call(run_input={
    'tweetIds': ['2010751592346030461'],
    'maxItems': 1,
})

for article in client.dataset(run['defaultDatasetId']).iterate_items():
    print(article['title'])
    print(article['markdown'][:500])
```

### Use X Articles Scraper with Apify MCP

Add this Actor to Claude Code through Apify's hosted MCP server:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/x-articles-scraper"
```

#### Claude Desktop

Add the following HTTP MCP server in Claude Desktop's MCP configuration.

#### Cursor

Use the same server object in Cursor's MCP settings.

#### VS Code

Add the server URL to your VS Code MCP configuration. All three clients use this equivalent JSON:

```json
{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=automation-lab/x-articles-scraper"
    }
  }
}
```

Example prompts:

- “Extract this public X Article and return its Markdown with citations.”
- “Convert these two X post IDs into a multilingual research dataset.”
- “List the outbound links and media referenced in this X Article.”
- “Summarize this Article, but preserve its author and source URL.”

### Responsible and legal use

This Actor accesses data that X exposes to logged-out visitors. Public availability
does not automatically grant every reuse right. You are responsible for complying
with applicable law, X's terms, copyright, privacy, database rights, and the rules
of the jurisdiction where you operate.

Use reasonable request volumes. Do not use the Actor for harassment, surveillance,
unauthorized profiling, spam, or attempts to access protected content. When
republishing or quoting an Article, preserve attribution and check whether your use
requires permission from the author or rights holder.

### Troubleshooting

#### Why did a successful run return no rows?

The supplied posts were available but did not expose complete public long-form
Articles. Confirm that each status opens an Article—not an ordinary tweet, thread,
Note, link card, or repost—in a logged-out browser.

#### Why was one URL skipped while others succeeded?

The skipped post may be deleted, protected, region-limited, ordinary, or temporarily
missing its rich-content payload. The Actor saves and charges only complete Articles.

#### What should I do after an X rate-limit error?

Wait before retrying, keep the default delay, or increase `requestDelayMs`. For a
persistent network-specific block, explicitly configure Apify Proxy. The Actor does
not automatically select residential traffic.

#### Why are follower counts or views missing?

X does not always expose every optional profile and engagement field to anonymous
sessions. The Article text can still be complete.

#### Does this Actor scrape all Articles from a profile?

No. It processes selected status URLs or post IDs. It does not perform profile
discovery or X search.

#### Can it extract ordinary tweets or threads?

No. Use [Twitter/X Scraper](https://apify.com/automation-lab/twitter-scraper) for
ordinary posts, profiles, timelines, and supported tweet workflows.

### Related automation-lab Actors

- [Twitter/X Scraper](https://apify.com/automation-lab/twitter-scraper) — ordinary tweets, profiles, timelines, and supported search workflows.
- [News & Article Extractor](https://apify.com/automation-lab/news-article-extractor) — discover and normalize articles from websites, RSS feeds, and sitemaps.
- [Webpage Text Extractor](https://apify.com/automation-lab/webpage-text-extractor) — extract clean text from general public web pages.

### FAQ

#### Do I need an X developer API key?

No. The Actor uses X's public logged-out guest flow.

#### Do I need to provide X cookies?

No. User cookies are not accepted or required.

#### Is each media image charged separately?

No. Media metadata and URLs are included in the successful `X Article` result and
have no separate event charge.

#### Are duplicate IDs charged twice?

No. IDs are deduplicated before requests begin.

#### Does `maxItems` count skipped ordinary posts?

It limits unique posts inspected and successful rows saved. A skipped post consumes
an input position but does not incur an item event.

#### Can I export to CSV, Excel, or JSON?

Yes. Use the default dataset's export controls or API endpoints. Markdown remains a
string field in each exported record.

#### Does the Actor store my input outside Apify?

The Actor uses normal Apify run storage. It does not send your input to a separate
third-party scraping service.

#### Is Markdown guaranteed to render identically everywhere?

No. Markdown renderers vary, especially for underline and nested rich-text styles.
The Actor preserves semantic structure and also provides `plainText` for consumers
that do not want Markdown syntax.

# Actor input Schema

## `startUrls` (type: `array`):

Public x.com or twitter.com status URLs whose posts contain long-form Articles.

## `tweetIds` (type: `array`):

Optional numeric X post IDs. You can combine these with URLs; duplicates are removed.

## `maxItems` (type: `integer`):

Maximum number of unique posts to inspect and Article records to save.

## `requestDelayMs` (type: `integer`):

A conservative delay reduces X rate limits. Increase it for larger batches.

## `proxyConfiguration` (type: `object`):

Optional Apify Proxy settings. Direct access is used when this is disabled.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://x.com/thedankoe/status/2010751592346030461"
    }
  ],
  "tweetIds": [],
  "maxItems": 20,
  "requestDelayMs": 800,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `articles` (type: `string`):

Default dataset with one row per successfully converted Article.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://x.com/thedankoe/status/2010751592346030461"
        }
    ],
    "tweetIds": [],
    "maxItems": 20,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/x-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://x.com/thedankoe/status/2010751592346030461" }],
    "tweetIds": [],
    "maxItems": 20,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/x-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://x.com/thedankoe/status/2010751592346030461"
    }
  ],
  "tweetIds": [],
  "maxItems": 20,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call automation-lab/x-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=automation-lab/x-articles-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/CHKsxediii7qy8dJT/builds/hS217dC7SOhxcYIPD/openapi.json