X / Twitter Post Scraper

Collect structured data from public X (Twitter) posts using direct post URLs. Paste one or more status links — from x.com or twitter.com — and receive clean JSON records in your Apify dataset.

Use it for social listening, brand monitoring, research, content archiving, competitive analysis, and analytics pipelines — without copying posts manually from the site.

---

What you get

Each saved row is one post, organized into clear sections:

Section	What it contains
Identity	Post ID, canonical URL, post type (tweet, media, link, reply, quote, and related types)
Content	Post text, created date, and timestamp
author	Username, display name, avatar, verification status, and badge when available
metrics	Likes, reposts, replies, bookmarks, and view count when published
entities	Hashtags, mentions, and expanded URL entities
media	Photos, videos, and GIFs with preview URLs and dimensions
link_preview	Title, description, domain, and image for link-card posts
context	Reply, quote, promoted, edited, and related flags
authorProfile	Optional full public profile (bio, counts, banner, links) when enabled

Results stream to the dataset as posts are collected, so you can preview progress before the run finishes.

---

Quick start on Apify

1. Open the Actor in the Apify Console.
2. Add one or more Post URLs — public X status links such as https://x.com/username/status/1234567890.
3. Choose output options:
   • Flatten output (default) — clean, normalized JSON for most use cases.
   • Include author profile — attach the author's public profile to each post (each unique handle is fetched once per run).
4. Click Start and open the Dataset tab when the run finishes.

Example input:

{
  "postUrls": [
    { "url": "https://x.com/elonmusk/status/2067438979667140620" },
    { "url": "https://x.com/hackernoon/status/2066573502832906644" }
  ],
  "flattenOutput": true,
  "includeAuthorProfile": true
}

Supported URL formats:

• https://x.com/{username}/status/{post_id}
• https://twitter.com/{username}/status/{post_id}

---

Input parameters

Post URLs

Type	List of URLs
Required	Yes — at least one URL
Format	Public X/Twitter status links

All URLs in the list are processed in parallel. Invalid URLs are saved as error rows with a message; other URLs in the same run can still succeed.

Flatten output

Type	Boolean
Default	true

When enabled, each result is a compact, user-friendly record with normalized field names (author, metrics, entities, and related sections).

When disabled, each result includes additional detail — top-level engagement fields, grouped media with video variants, link card data, and a richer author block — while staying readable (no raw technical payloads).

Include author profile

Type	Boolean
Default	false

When enabled, the Actor fetches public profile data for each post author and adds an authorProfile object to the result.

• Profile data includes bio, follower/following/post counts, verification, avatar, banner, and profile links when available.
• If multiple input URLs share the same author, that profile is fetched once per run and reused.

---

What you will see during a run

The run log is written for end users — progress messages describe collection activity only.

Typical messages:

• Starting scrape for N post URL(s) in parallel.
• Progress: 25/100 post(s) collected. (periodic updates on large runs)
• Run finished. Saved N post(s) to the dataset.

If a post cannot be processed, you will see a warning for that URL; other URLs in the same run can still succeed. Failed URLs are saved to the dataset with an error field.

On the Apify free plan, you may also see a short notice when free-tier limits apply (see Free plan limits).

---

Output overview

Dataset structure

• Results are saved to a single Apify dataset
• Each row is one post object (or one error object for a failed URL)
• With Include author profile enabled, profile data is nested under authorProfile on each post row

Field presence varies by post: not every post includes media, link previews, view counts, or full profile details. Empty arrays ([]) or null mean the data was not published for that post — not necessarily an error.

Output modes

Mode	Best for
Flatten output enabled (default)	Exports, dashboards, APIs, and most integrations
Flatten output disabled	Deeper media/card fields, video variants, and reply metadata while keeping a clean structure

---

Output fields (flattened mode)

Top-level post

Field	Description
id	Post ID
url	Canonical post URL
post_type	Post category such as tweet, media, link, reply, or quote
text	Post text
created_at	Human-readable post date
created_at_ms	Post timestamp in milliseconds

author

Field	Description
id	Author account ID
username	Handle (without @)
name	Display name
avatar_url	Profile image URL
verified	Whether the account shows as verified
verified_type	Verification type when available (for example Business)
badge	Affiliation or label badge when shown
profile_shape	Profile image shape when available

metrics

Field	Description
likes	Like count
retweets	Repost count
replies	Reply count
bookmarks	Bookmark count
views	View count when published (string on X)

entities

Field	Description
hashtags	Hashtag tags
mentions	Mentioned usernames and names
urls	Short, expanded, and display URLs

media

Array of media items (photos, videos, GIFs) with type, URLs, dimensions, duration, and alt text when available.

link_preview

Present on link-card posts: title, description, domain, URL, and preview image.

context

Field	Description
is_promoted	Promoted post flag when detectable
is_reply	Whether the post is a reply
is_quote	Whether the post is a quote
is_edited	Whether the post has been edited
edit_history_tweet_ids	Edit history IDs when available
translation_available	Translation availability flag
sensitive_media_warning	Sensitive media warning when shown

authorProfile (when enabled)

Field	Description
id	Account ID
username	Handle
name	Display name
url	Profile page URL
bio	Profile bio text
bio_urls	Links found in the bio
website	Primary website link when available
location	Location when published
avatar_url	Profile image
banner_url	Profile banner image
verified	Verification status
verified_type	Verification type when available
badge	Profile badge when shown
profile_shape	Profile image shape
protected	Whether the account is protected
counts	Followers, following, posts, and related counts
professional	Professional category when available
business_account	Business account details when available
created_at	Account join date

---

Example output

The excerpts below come from a real run with Flatten output and Include author profile enabled. Full output is available in the run dataset.

Example input

{
  "postUrls": [
    { "url": "https://x.com/hackernoon/status/2066573502832906644" }
  ],
  "flattenOutput": true,
  "includeAuthorProfile": true
}

Excerpt — link post with card preview and profile

{
  "id": "2066573502832906644",
  "url": "https://x.com/hackernoon/status/2066573502832906644",
  "post_type": "link",
  "text": "Some engineering teams fix 90% of software issues before customers ever notice. \n\nLearn how AI is changing debugging without replacing developers: https://t.co/rEOC2CXJz7",
  "created_at": "Mon Jun 15 17:28:12 +0000 2026",
  "author": {
    "username": "hackernoon",
    "name": "HackerNoon | Learn Any Technology",
    "verified": true,
    "verified_type": "Business"
  },
  "metrics": {
    "likes": 221,
    "retweets": 23,
    "replies": 4,
    "bookmarks": 162,
    "views": "734692"
  },
  "entities": {
    "urls": [
      {
        "short": "https://t.co/rEOC2CXJz7",
        "expanded": "https://hackernoon.com/the-technical-infrastructure-of-automated-debugging",
        "display": "hackernoon.com/the-technical-…"
      }
    ]
  },
  "link_preview": {
    "title": "The Technical Infrastructure of Automated Debugging | HackerNoon",
    "description": "PlayerZero combines AI, telemetry, and system modeling to accelerate root cause analysis and help engineering teams debug faster.",
    "domain": "hackernoon.com",
    "url": "https://t.co/rEOC2CXJz7",
    "image_url": "https://pbs.twimg.com/card_img/2066562404574531584/4F9VBmbS?format=jpg&name=orig"
  },
  "authorProfile": {
    "username": "hackernoon",
    "name": "HackerNoon | Learn Any Technology",
    "bio": "how hackers start their afternoons. where 50k+ technologists publish blog posts for 4M+ monthly readers. write your story 👉https://t.co/PGmtSCSd5V",
    "counts": {
      "followers": 92589,
      "following": 5448,
      "posts": 135804
    },
    "verified": true,
    "verified_type": "Business"
  }
}

Excerpt — video post

{
  "id": "2067438979667140620",
  "url": "https://x.com/elonmusk/status/2067438979667140620",
  "post_type": "media",
  "text": "https://t.co/ujL9EUqFK4",
  "author": {
    "username": "elonmusk",
    "name": "Elon Musk",
    "verified": true
  },
  "metrics": {
    "likes": 104664,
    "retweets": 11595,
    "replies": 5069,
    "views": "26356947"
  },
  "media": [
    {
      "type": "video",
      "url": "https://pbs.twimg.com/ext_tw_video_thumb/1848648724500672512/pu/img/NMjczVOjehsLLTWy.jpg",
      "width": 1280,
      "height": 720,
      "duration_ms": 60821
    }
  ]
}

Video-only posts may show a short link as the post text; media details are under media.

---

Scraping many URLs

You can add hundreds or thousands of post URLs in one run.

• URLs are processed in parallel for faster collection.
• Progress is logged at intervals (for example every 25 posts) instead of one line per post.
• Duplicate authors with Include author profile enabled still trigger only one profile fetch per unique handle per run.

Tip: Start with a small batch (5–10 URLs) to validate output shape before a large export.

---

Free plan limits

On the Apify free plan, the Actor may automatically apply:

Limit	Value
Post URLs per run	5 (only the first URLs in your list)

Paid Apify plans can use the full input without this cap. If limits apply, the run log will include a short notice.

---

Pricing

This Actor uses pay-per-event billing on the Apify Store:

What you pay for	When it applies
Post scraped	Each post successfully saved to your dataset
Author profile scraped	Each unique author profile fetched when Include author profile is enabled

See the Pricing tab on the Store listing for current rates and plan discounts.

Posts saved with an error field (failed URLs) are still written to the dataset; check the Pricing tab for how failed items are billed.

---

Data quality and limitations

Public post data only
The Actor reads data shown on public X post and profile pages. It does not access private accounts, direct messages, or content that requires login beyond what X shows to a guest visitor.

Coverage varies by post
View counts, link previews, media metadata, and profile fields are not always published. Some fields may be null or empty even when the post is live.

Post text
Media-heavy posts sometimes contain only a short link in the text field; full media details appear under media.

Profile links in bio
Expanded destination URLs for t.co links in bios may not always be available.

Live data changes
Engagement counts and profile stats change over time. Re-run on a schedule to refresh data.

Scope
This Actor scrapes individual post URLs only. Profile timelines, search, followers, and hashtag pagination are not included.

Compliance
You are responsible for using collected data in line with applicable laws and X's terms of service.

---

Tips for best results

1. Use direct status URLs — Copy the link from the post's share menu or browser address bar.
2. Enable profiles when you need bio and counts — Turn on Include author profile for enrichment; leave it off for faster, lower-cost post-only runs.
3. Keep flatten on for most workflows — Use the default flattened output for CSV exports and APIs; disable it when you need video variants or extra card fields.
4. Test with a few URLs first — Confirm field coverage before large batch runs.
5. Schedule recurring runs — Monitor key accounts or campaigns with Apify schedules.
6. Handle errors in the dataset — Filter rows where success is false or an error field is present.
7. Deduplicate by post ID — Use the id field if the same URL appears more than once in your input list.

---

Frequently asked questions

Why did my post URL fail?
The post may have been deleted, made private, or temporarily unavailable. Check the run log and the dataset row for the error message.

Why is view count a string?
X publishes view counts as formatted strings (for example "734692"). This matches what appears in the source data.

Why is post text only a t.co link?
Some posts — especially video-only posts — contain little or no caption text. Check the media section for the full content.

Why is the same profile attached to multiple posts?
When Include author profile is enabled, each unique author is resolved once per run and reused for all their posts.

Does this scrape a user's full timeline?
No. Provide specific post URLs. Timeline and search scraping are out of scope for this Actor.

Can I use twitter.com links instead of x.com?
Yes. Both domains are supported.

What happens if I submit duplicate post URLs?
Each URL is processed independently. You may receive duplicate rows unless you deduplicate by id afterward.

---

Support

For Actor-specific issues, use the Issues tab on the Apify Store listing or contact the publisher through Apify.

For platform questions (runs, billing, API, schedules), see Apify documentation and Apify support.