# Twitter Article Scraper (`maximedupre/twitter-article-scraper`) Actor Extract public long-form X Articles from post URLs or numeric post IDs. Get Markdown, plain text, author details, dates, links, media references, content blocks, and public engagement counts in a structured dataset. - **URL**: https://apify.com/maximedupre/twitter-article-scraper.md - **Developed by:** [Maxime DuprΓ©](https://apify.com/maximedupre) (community) - **Categories:** Social media, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $0.10 / 1,000 x articles This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ### πŸ“° Turn selected X Articles into usable text Researchers, journalists, and AI teams can submit public X post URLs or numeric post IDs and get the public Article body as Markdown and plain text. Each dataset row keeps the source post, author, publication dates, links, media references, content blocks, and public engagement counts when X exposes them, so you can cite, search, archive, or analyze the text. - Convert a public X post into Markdown with **[X Long Form Article Scraper](https://apify.com/maximedupre/twitter-article-scraper/examples/x-long-form-article-scraper)**. - Read a selected long-form post for a research file with **[Twitter Long Form Article Scraper](https://apify.com/maximedupre/twitter-article-scraper/examples/twitter-long-form-article-scraper)**. - Save Article text and source details with **[Twitter Article Extractor](https://apify.com/maximedupre/twitter-article-scraper/examples/twitter-article-extractor)**. - Pull one public Article from its post URL with **[X Article Scraper](https://apify.com/maximedupre/twitter-article-scraper/examples/x-article-scraper)**. - Build a structured dataset from submitted Article sources with **[Twitter Article Scraper](https://apify.com/maximedupre/twitter-article-scraper/examples/twitter-article-scraper)**. #### πŸ“„ See the Article, source post, and author A dataset row groups three top-level objects: `article`, `sourcePost`, and `author`. Optional source fields can be absent when X does not expose them. **Returned data** The `article` object holds the Article body, structure, media, and links. `sourcePost` keeps the containing post and its public engagement snapshot. `author` keeps the public profile fields that X exposes. #### ▢️ Run the X Article scraper 1. Choose `Public post URLs` or `Numeric post IDs` in **Find Articles by**. 2. Add the matching public sources. The other source section is not used for that run. 3. Start the run and open the default dataset to review the nested Article rows. **What the Actor reads** The Actor reads selected public long-form Articles that are accessible without an X login. It does not search X or discover Articles from a profile, timeline, keyword, or feed. It does not scrape ordinary posts, threads, replies, quote posts, or profiles. **Access limits** Private, deleted, paywalled, geo-restricted, login-gated, and CAPTCHA-protected content is outside the supported scope. Media is returned as source references; the Actor does not download or permanently store media files. #### βš™οΈ Input Choose one discovery method, then add sources in the matching section. The other section is not used for that run. **Input fields** | Field | Type | What it does | | --- | --- | --- | | `discoveryMethod` | choice | Required. Choose `postUrls` to use public post URLs or `postIds` to use numeric post IDs. | | `postUrls` | array of objects | When `discoveryMethod` is `postUrls`, add one or more full X or Twitter status URLs for posts that contain public long-form Articles. | | `postUrls[].url` | string | Full public X or Twitter post URL with a numeric status ID. | | `postIds` | array of strings | When `discoveryMethod` is `postIds`, add one or more numeric IDs for public X posts that contain long-form Articles. | **Example input** This example is copied from the public input of a successful current-beta run: ```json { "discoveryMethod": "postUrls", "postUrls": [ { "url": "https://x.com/tonysimons_/status/2060527240098587085" } ] } ``` #### 🧾 Output The output schema provides a link to the default Apify dataset. Each saved row follows the nested shape documented below. **Run output** | Field | Type | What it does | | --- | --- | --- | | `dataset` | URL | Opens the default dataset for the run. | **Dataset row** | Field | Type | What it does | | --- | --- | --- | | `article` | object | Groups the public Article, its content, media, and links. | | `article.id` | string | Stable Article ID assigned by X. | | `article.url` | string | Canonical public URL for the Article. | | `article.title` | string | Article title when the source provides one. | | `article.previewText` | string | Source-provided Article preview when available. | | `article.publication` | object | Source dates for the Article. | | `article.publication.createdAt` | date-time string | Time when the source says the Article was created. | | `article.publication.firstPublishedAt` | date-time string | Time when the source says the Article was first published. | | `article.publication.lastModifiedAt` | date-time string | Time when the source says the Article was last modified. | | `article.content` | object | Holds readable and structured Article content. | | `article.content.markdown` | string | Complete publicly available Article body rendered as Markdown. | | `article.content.plainText` | string | Article body rendered as plain text. | | `article.content.blocks` | array of objects | Source content blocks when available. | | `article.content.blocks[].type` | string | Source-reported type of a content block. | | `article.content.blocks[].text` | string | Text carried by a block when available. | | `article.content.blocks[].level` | integer | Source-reported heading or nesting level when available. | | `article.content.blocks[].items` | array of strings | Text items in a list-like block when available. | | `article.content.blocks[].inline` | array of objects | Inline text runs and formatting in a block when available. | | `article.content.blocks[].inline[].text` | string | Text in an inline run. | | `article.content.blocks[].inline[].formats` | array of strings | Source formatting marks on an inline run. | | `article.content.blocks[].inline[].url` | string | Link destination for an inline run when available. | | `article.content.references` | array of objects | Source references used by structured Article content. | | `article.content.references[].id` | string | Source ID for a content reference when available. | | `article.content.references[].type` | string | Source-reported reference type. | | `article.content.references[].url` | string | Destination URL for a content reference when available. | | `article.media` | object | Source references for cover and inline media. | | `article.media.cover` | object | Cover media reference when available. | | `article.media.cover.id` | string | Source media ID when available. | | `article.media.cover.url` | string | URL where the cover media is available. | | `article.media.cover.type` | string | Source-reported cover media type when available. | | `article.media.cover.width` | integer | Cover media width in pixels when available. | | `article.media.cover.height` | integer | Cover media height in pixels when available. | | `article.media.inline` | array of objects | Source references for media inside the Article body. | | `article.media.inline[].id` | string | Source media ID when available. | | `article.media.inline[].url` | string | URL where the inline media is available. | | `article.media.inline[].type` | string | Source-reported inline media type when available. | | `article.media.inline[].width` | integer | Inline media width in pixels when available. | | `article.media.inline[].height` | integer | Inline media height in pixels when available. | | `article.links` | array of objects | Links embedded in the Article. | | `article.links[].text` | string | Text shown for a link in the Article. | | `article.links[].url` | string | Resolved destination of the link. | | `sourcePost` | object | The public X post that contains the Article. | | `sourcePost.id` | string | Stable ID of the containing post. | | `sourcePost.url` | string | Canonical public URL of the containing post. | | `sourcePost.engagement` | object | Public engagement counts for the containing post when available. | | `sourcePost.engagement.likes` | integer | Reported like count. | | `sourcePost.engagement.replies` | integer | Reported reply count. | | `sourcePost.engagement.reposts` | integer | Reported repost count. | | `sourcePost.engagement.quotes` | integer | Reported quote-post count. | | `sourcePost.engagement.bookmarks` | integer | Reported bookmark count. | | `sourcePost.engagement.views` | integer | Reported view count. | | `author` | object | Public profile shown for the Article author. | | `author.id` | string | Author ID assigned by X when available. | | `author.name` | string | Public author display name when available. | | `author.username` | string | Public author username. | | `author.profileUrl` | string | Public author profile URL when available. | | `author.isVerified` | boolean | Whether X marks the author as verified. | | `author.description` | string | Public profile description when available. | | `author.location` | string | Public profile location when available. | | `author.websiteUrl` | string | Website shown on the public profile when available. | | `author.profileImageUrl` | string | Public profile image URL when available. | **Genuine current-beta row** The row below is shortened. The `markdown`, `plainText`, `blocks`, `references`, and inline media values use `"..."` as omission markers. The remaining values are from a successful current-beta run. ```json { "article": { "id": "2052796100608974848", "url": "https://x.com/trq212/status/2052809885763747935", "title": "Using Claude Code: The Unreasonable Effectiveness of HTML", "previewText": "This is now also on the Claude Blog.\n\nMarkdown has become the dominant file format used by agents to communicate with us. It’s simple, portable, has some rich text capability and is easy for you to", "publication": { "createdAt": "2026-05-08T17:56:30.000Z", "firstPublishedAt": "2026-05-08T17:56:30.000Z", "lastModifiedAt": "2026-05-19T21:04:23.000Z" }, "content": { "markdown": "...", "plainText": "...", "blocks": "...", "references": "..." }, "media": { "cover": { "id": "2052796450510348288", "url": "https://pbs.twimg.com/media/HHz_ftzaIAAwkQs.jpg", "type": "ApiImage", "width": 2400, "height": 960 }, "inline": "..." }, "links": [ { "text": "Claude Blog", "url": "https://claude.com/blog/using-claude-code-the-unreasonable-effectiveness-of-html" }, { "text": "https://thariqs.github.io/html-effectiveness", "url": "https://thariqs.github.io/html-effectiveness/" }, { "text": "https://x.com/trq212/status/2017024445244924382", "url": "https://x.com/trq212/status/2017024445244924382" }, { "text": "https://thariqs.github.io/html-effectiveness/", "url": "https://thariqs.github.io/html-effectiveness/" } ] }, "sourcePost": { "id": "2052809885763747935", "url": "https://x.com/trq212/status/2052809885763747935", "engagement": { "likes": 17841, "replies": 1098, "reposts": 2292, "quotes": 1938, "bookmarks": 34604, "views": 14717658 } }, "author": { "username": "trq212", "id": "352806502", "name": "Thariq", "profileUrl": "https://x.com/trq212", "isVerified": true, "description": "Claude Code @anthropicai. prev YC W20, @spc, @medialab", "location": "SF via Toronto", "websiteUrl": "http://thariq.io", "profileImageUrl": "https://pbs.twimg.com/profile_images/1976939058741039104/r3GgzqRh_200x200.jpg" } } ``` #### πŸ’³ Pricing This Actor uses pay-per-event pricing. The `X Article` event is charged only when one public long-form Article is extracted and saved. Check the Actor page for the current event price. #### πŸ”Œ Integrations Results are available in the default Apify dataset and through the Apify API. Use the dataset export tools or the dataset API URL to pass Article rows to your own research, archive, or text workflow. For a short walkthrough of using an Actor in an Apify workflow, see: https://www.youtube.com/watch?v=bNACk1\_S\_6w\&list=PLObrtcm1Kw6MUrlLNDbK9QRg8VDJg0gOW\&index=4 #### ❓ FAQ ##### Can I submit URLs and numeric IDs in one run? No. Choose one discovery method and fill in its matching section for each run. ##### What if a post is an ordinary tweet or its Article is not public? The Actor is for public long-form X Articles only. The source must be readable without an X login, and ordinary posts and restricted content are outside the supported scope. ##### Do I need an X login, developer API key, or cookies? No. You do not need to give the Actor an X login, developer API key, or cookies. ##### Does the Actor find Articles from a profile or keyword? No. Submit the public post URL or numeric post ID for each Article you want to read. The Actor does not perform profile discovery or X search. ##### Which post URL should I submit when an Article was reposted? Use the public post that contains the Article. An unrelated repost is not a reliable Article source. ##### What are content blocks and references for? They preserve source-provided structure and reference details for workflows that need more than the Markdown or plain-text rendering. ##### Are engagement counts historical? No. They are public counts captured with the Article and can change after extraction. Optional counts may be absent when X does not expose them. ##### Does the Actor download Article images? No. It returns source media references, including available URLs and dimensions, but it does not create permanent media copies. ##### Can it read private, subscriber-only, or paywalled Articles? No. The Actor does not bypass login, payment, geographic, deletion, or other access restrictions. ### πŸ“ Changelog **v0.0** (18-09-2026) - Initial release. ### πŸ†˜ Support For issues, questions, or feature requests, [file a ticket](https://console.apify.com/actors/maximedupre~twitter-article-scraper/issues) and I'll fix or implement it in less than 24h 🫑 ### πŸ”— Related Actors - [X Articles to Markdown Scraper](https://apify.com/maximedupre/x-articles) - Convert selected public X Articles to Markdown and plain text with a sibling Article workflow. - [Twitter Scraper](https://apify.com/maximedupre/twitter-scraper) - Collect ordinary X posts, profiles, and timelines when the source is not an Article. - [Webpage Text Extractor](https://apify.com/maximedupre/webpage-text-extractor) - Extract Markdown and article text from public web pages outside X. - [URL to BibTeX Converter](https://apify.com/maximedupre/url-to-bibtex-converter) - Turn Article URLs into BibTeX citations for reference lists. - [Substack API Scraper: Posts, Authors & Newsletters](https://apify.com/maximedupre/substack-api) - Collect public newsletter posts when the long-form source is on Substack. **Made with ❀️ by Maxime DuprΓ©** # Actor input Schema ## `discoveryMethod` (type: `string`): Choose how to identify the public posts that contain the Articles. ## `postUrls` (type: `array`): Use this section when Find Articles by is set to Public post URLs. Add one or more full X post URLs. Each post must contain a public long-form Article that can be read without an X login. ## `postIds` (type: `array`): Use this section when Find Articles by is set to Numeric post IDs. Add one or more numeric IDs for public X posts that contain a long-form Article. Each post must be readable without an X login. ## Actor input object example ```json { "discoveryMethod": "postUrls", "postUrls": [ { "url": "https://x.com/username/status/1234567890123456789" } ], "postIds": [ "1234567890123456789" ] } ``` # Actor output Schema ## `dataset` (type: `string`): Open the Article results for this run. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "discoveryMethod": "postUrls", "postUrls": [ { "url": "https://x.com/tonysimons_/status/2060527240098587085" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("maximedupre/twitter-article-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "discoveryMethod": "postUrls", "postUrls": [{ "url": "https://x.com/tonysimons_/status/2060527240098587085" }], } # Run the Actor and wait for it to finish run = client.actor("maximedupre/twitter-article-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "discoveryMethod": "postUrls", "postUrls": [ { "url": "https://x.com/tonysimons_/status/2060527240098587085" } ] }' | apify call maximedupre/twitter-article-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,maximedupre/twitter-article-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/MnFtan1U2Z5PaLIoG/builds/1pbsx2qDM5S0NTLAR/openapi.json