Twitter X Tweet Embed HTML Scraper

Prepare known public posts for website publishing: turn a batch of X/Twitter post URLs into source-native twitter embeds, canonical post links, author links and publisher metadata. This Actor calls the public oEmbed service; it does not reconstruct markup from scraped text or require an X account.

Who is it for?

CMS editors, newsroom producers and developers who already have a list of posts and need reusable embed fragments with explicit retrieval outcomes. It is not a post discovery, search, account monitoring or social analytics tool.

Why use it?

• Resolve supplied URLs in one batch rather than copying each embed manually.
• Deduplicate the same post across X/Twitter domain and username variants.
• Preserve native HTML, attribution and publisher metadata together.
• Separate unavailable posts from transient upstream failures.
• No cookies, account session, proxy configuration or external API key.

Quick start

1. Open the Actor and paste public status URLs into Public post URLs.
2. Choose a maximum number of unique posts to attempt.
3. Run the Actor and open its default dataset.
4. Filter status to ok before inserting html into your CMS.
5. Review unavailable/error rows instead of publishing placeholders.

{
  "startUrls": [{"url":"https://x.com/jack/status/20"}],
  "maxItems": 100
}

Input parameters

Parameter	Behavior
startUrls	Required, 1–1000 public status URLs as objects containing a url string.
maxItems	Default 100; integer 1–1000. Global cap on unique post IDs attempted, including unavailable/error outcomes.

Accepts x.com, www.x.com, twitter.com, www.twitter.com and mobile.twitter.com status URLs, including /i/status/ID and /i/web/status/ID. Query strings do not affect lookup. HTTP inputs are upgraded for lookup. Bare IDs, profiles, search URLs, media subpaths and other hosts are rejected. All supplied entries are validated before work, even beyond the cap.

Duplicates are processed once by post ID, preserving the first input URL. The cap is not a promised number of successful embeds. Runtime and billing limits can stop a batch earlier. There is no discovery or pagination.

Extracted data

Field	Meaning
inputUrl, postId	Original first URL and stable source identity.
postUrl	Canonical URL supplied by X on success; ID-based lookup URL otherwise.
status	ok, unavailable or error.
html	Unmodified native blockquote/script fragment; null on failure.
authorName, authorUrl	Public attribution returned by the source.
providerName, providerUrl	Publisher identity, normally X and its homepage.
width, height	Suggested dimensions; height is often null.
embedType, oembedVersion	Source oEmbed metadata.
cacheAge	Suggested cache duration in seconds, represented as a string.
error, scrapedAt	Explicit error or null, and retrieval timestamp.

Output example

A current control for the original public post returned the following fields (the HTML is shortened here only for readability; actual output retains the complete fragment):

{
  "inputUrl": "https://x.com/jack/status/20",
  "postId": "20",
  "postUrl": "https://x.com/jack/status/20",
  "status": "ok",
  "html": "<blockquote class=\"twitter-tweet\">…just setting up my twttr…</blockquote>…",
  "authorName": "jack",
  "authorUrl": "https://x.com/jack",
  "providerName": "X",
  "providerUrl": "https://x.com",
  "width": 550,
  "height": null,
  "embedType": "rich",
  "cacheAge": "3153600000",
  "oembedVersion": "1.0",
  "error": null
}

How much does it cost to extract Twitter embeds?

Pay per event: a $0.001 start fee per valid run plus a fee for each successful embed. Unavailable/error rows and duplicates have no item charge. A valid all-unavailable batch still incurs the start fee. Invalid input fails before charging.

Apify plan	Per successful embed
FREE	$0.0092
BRONZE	$0.008
SILVER	$0.00624
GOLD / PLATINUM / DIAMOND	$0.0048

At BRONZE, 1 successful embed costs $0.009, 10 cost $0.081 and 100 cost $0.801 including one start fee. FREE examples: 1 costs $0.0102 and 100 cost $0.921. Charges are modeled from these events; your configured maximum charge can stop processing early. Check the Store pricing panel for the currently applicable prices.

Unavailable posts and failures

HTTP 404/410 from the public embed service produces status=unavailable. This may mean missing, deleted, protected, restricted or otherwise not embeddable; it does not establish which reason applies. Other HTTP failures and malformed/mismatched responses produce status=error.

Network timeouts, 429 and temporary 5xx responses receive at most three attempts with backoff and a 15-second request timeout. There is no paid fallback. Deterministic unavailable/forbidden responses are not retried blindly. Processing is sequential to avoid rate spikes and keep resource use bounded.

A mixed batch with useful embeds and unavailable posts succeeds with explicit unavailable rows. Any unresolved upstream error, or a batch with no useful embeds, fails after retaining available output. A failed run can contain partial rows; never treat its dataset as a completed successful batch.

Rendering and caching limits

Native HTML can include an external X widgets script. Some CMS editors strip scripts or block third-party widgets; configure your renderer appropriately and avoid loading the widget loader repeatedly. This Actor returns source-native markup, not sanitized HTML. Apply your own HTML/content-security policy before rendering untrusted content.

X may serve cached embeds. cacheAge is a provider suggestion, not proof of current availability or a promise that old/deleted content can be archived. The Actor does not guarantee protected, age-restricted, geographic or historical coverage. It does not download media, preserve rendered screenshots or provide account sessions.

Integrations

Export the dataset as JSON or CSV, or send successful rows to a CMS through Make, Zapier, n8n or an Apify webhook. Use postId as your CMS deduplication key, save postUrl for provenance and keep error rows in an editorial review queue. HTML exports require correct escaping when stored inside CSV/JSON.

Scheduling can refresh the same supplied list. It does not discover new posts or automatically compare changes, send alerts or publish into your CMS.

API examples

Use your Apify token as a secret. Start a run once, retain its ID and read its default dataset after successful completion. The synchronous endpoint below is suitable for small lists; its timeout is not a reason to blindly start a duplicate run.

curl -X POST 'https://api.apify.com/v2/acts/automation-lab~x-tweet-embed-html/run-sync-get-dataset-items' \
  -H "Authorization: Bearer $APIFY_TOKEN" -H 'Content-Type: application/json' \
  -d '{"startUrls":[{"url":"https://x.com/jack/status/20"}],"maxItems":1}'

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/x-tweet-embed-html').call({
  startUrls: [{ url: 'https://x.com/jack/status/20' }], maxItems: 1,
});
if (run.status !== 'SUCCEEDED') throw new Error(`Run ${run.id}: ${run.status}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems({ limit: 100 });

import os
from apify_client import ApifyClient
client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/x-tweet-embed-html').call(
    run_input={'startUrls': [{'url': 'https://x.com/jack/status/20'}], 'maxItems': 1})
if run['status'] != 'SUCCEEDED':
    raise RuntimeError(run['status'])
rows = client.dataset(run['defaultDatasetId']).list_items(limit=100).items

MCP access

claude mcp add --transport http apify \
  'https://mcp.apify.com?tools=automation-lab/x-tweet-embed-html'

Claude Desktop, Cursor and VS Code: use the equivalent HTTP-client configuration below, where supported by your client's MCP settings (authentication belongs in its protected settings):

{"mcpServers":{"apify":{"url":"https://mcp.apify.com?tools=automation-lab/x-tweet-embed-html"}}}

Example prompts:

• “Fetch embeds for these two public post URLs, then return canonical post links and author names for successful rows.”
• “Prepare the supplied post list for my CMS and list unavailable posts separately without publishing anything.”

Discover actual tool names and argument schemas with tools/list; selecting this Actor also adds run/data/abort helper tools, not exactly one tool. Discovery is read-only, not permission to execute. For an authorized invocation, start once, retain run/status/storage IDs, and follow that same run. call-actor waits at most 45 seconds (waitSecs, default 30). If pending, check with 2/4/8-second backoff capped at 10 seconds and a 120-second total deadline. Configure the client timeout above server wait plus transport margin. At deadline report pending with the run ID; on timeout recover the known run rather than restart it.

After success, read defaultDatasetId with explicit fields/limit/offset. Example consumer budget: pages of 20 source rows, at most 100 rows and 64 KiB serialized UTF-8 across admitted pages, whichever comes first. Enforce bytes host-side before model injection; oversized rows may need omission or disclosed summarization. If interception is unavailable, do not promise a hard byte guarantee. Track source offsets, report truncation/continuation, and keep full exports outside model context. Failed runs have partial output, not complete results. Client discovery/caching does not guarantee universal token savings.

Legality, responsible use and privacy

Use public posts only when you have the necessary rights to display them. Public availability does not transfer copyright or override X terms, attribution requirements or applicable law. This Actor is not affiliated with X Corp. It does not bypass login or private-post permissions.

There is no AI processing in this Actor. Public post IDs are sent to X's publishing service; results remain in your Apify run storage under your account's retention settings. Delete datasets and key-value stores through Apify when no longer needed. No additional cross-run cache or account/session store is maintained. For help, open a Store issue with the run ID and a non-sensitive description.

Failed operations send sanitized diagnostic input, exceptions and Actor/build/run IDs to our private GlitchTip service for repair. Secret fields and URL queries are removed; reports are retained for 30 days. Do not submit secrets in post URLs or input fields.

FAQ

Why is a visible post unavailable? The logged-out publishing service may restrict it or return a stale response. A browser-visible page is not a guarantee of public oEmbed support.

Why does HTML show plain text? Your CMS may strip scripts or block widgets. Inspect its embedding and CSP policy; the Actor does not render the fragment for you.

Can I supply IDs, search terms or profiles? No. Supply complete known status URLs. There is no discovery, search or authenticated mode.

Why are there fewer rows than URLs? Duplicate IDs are collapsed, maxItems caps unique attempts and billing/runtime limits can stop early. Inspect run status and counters.

Related Actors

For different jobs, see X User Timeline Tweets Scraper for timeline records and X Post Replies & Conversation Threads Scraper for reply relationships. Neither is needed to retrieve an embed for a supplied URL.