Twitter (X.com) Tweets & Profiles Scraper

X (Twitter) Scraper — Posts, Profiles and Geo-Tagged Place Data

Scrape any public X (Twitter) profile timeline logged out, or search X for posts that carry a real place tag — city, admin area, country, or an exact X place ID — plus posts published inside a radius of a named place or a lat/lon pair. Every row returns the post text, engagement counts, media, links, and the author's profile block, and every place-filtered search reports proof that X's geographic operator actually applied, not just the query you sent it. Built for social teams, market researchers, and agents that need X data broken down by where it came from, not just what it says.

🧭 What is the X Posts From A Place Scraper?

It's an Apify Actor that reads X profile timelines and runs X search queries, then flattens the response into typed JSON rows. It returns two kinds of records: post rows (one per tweet, whether from a profile timeline or a search) and place_search rows (one accounting record per place-filtered search, logging what was asked for and whether X honoured it). No X account is required to read profile timelines — search and every place filter need your own logged-in browser's session cookies, because X's search endpoint answers logged-out callers with an empty HTTP 404.

Key capabilities:

• Two ways to target posts — a profile URL, bare handle, @handle, or numeric user ID reads that account's timeline; a bare keyword, hashtag, or a search:-prefixed term runs an X search
• Place narrowing on the search endpoint — filter by an exact X place ID, an ISO-3166 country code, a named place with a radius, or a lat/lon pair with a radius
• Geo-filter verification built in — every place-filtered search proves the operator was honoured (geoFilterVerified) instead of silently handing back an unfiltered sweep
• Full author profile block — bio, avatar, join date, follower/following/post/media counts, website, and self-typed location attached to every post
• Engagement and media data — likes, replies, reposts, quotes, view counts, and every photo/video URL on the post
• Quality and volume filters — media-only, mention, language, sort order (newest / oldest / most-liked), and a per-target result cap

Output lands in an Apify dataset and can be exported to JSON, CSV, Excel, XML, or RSS directly from the Actor run's Export results button, or pulled with the Apify API / apify-client.

📦 What data can I extract with the X Posts From A Place Scraper?

Every post row carries the tweet's identity, content, engagement, media, place, and author fields below — pulled straight from build_tweet() in src/base_engine.py and enrich() in src/main.py.

Field	Example value	Use case
rowType	"post"	Filter out place_search accounting rows before analysis
id	"1823456789012345678"	De-dupe and re-fetch a specific post
url	"https://x.com/nasa/status/1823456789012345678"	Direct link back to the live post
verified	true	Flag posts from blue-checked accounts
username	"@nasa"	Group posts by author
fullname	"NASA"	Display name for reports
timestamp	"Tue Aug 04 14:02:11 +0000 2026"	X's raw format, kept for parity with the base actor
timestampIso	"2026-08-04T14:02:11Z"	Sort/filter by date without reparsing X's format
text	"Watching the coffee culture of Tokyo this morning ☕"	Sentiment, keyword, or topic analysis
links	["https://nasa.gov/live"]	Extract outbound links from a post
isPinned	false	Exclude pinned posts from timeline trend analysis
isQuote	false	Separate quote posts from originals
isRetweet	true	Identify reposts (row still carries the ORIGINAL post's author/text/counts)
isReply	false	Filter to top-level posts only
likes / replies / retweets / quotes	1042, 88, 310, 14	Engagement ranking and virality scoring
views / viewCount	48213	Reach measurement (same value under two keys; viewCount is an alias kept for readers who look for that name)
mediaCount	2	Filter for posts with attachments
mediaTypes	["photo"]	Distinguish photo vs. video posts before downloading media
hasPhoto / hasVideo	true / false	Quick media-type filters without inspecting the media array
media	[{"type": "photo", "url": "https://pbs.twimg.com/..."}]	Download or embed attached photos/videos
tweetUserId	"11348282"	Stable numeric author ID for joins
retweetedFrom	"@user123"	The account that reposted, when isRetweet is true
originalTweetId	"1823400000000000000"	The original post's ID behind a repost
isGeoTagged	true	True only when X attached a real place object
language	"en"	ISO 639-1 code X assigned to the post
conversationId	"1823400000000000000"	Group a post with the thread it belongs to
bookmarkCount	56	Additional engagement signal from X's payload
authorLocationText	"Washington, DC"	The author's self-typed profile location (not where the post was made)
authorVerifiedType	"Business"	X's verification category for the author, when present
sourceTarget	"nasa"	Which startUrls entry produced this row
placeSearch	{"mode": "profile_timeline", "searchTerm": null, ...}	Nested context object on every post row — see below
scrapedAt	"2026-08-04T14:05:02Z"	When this Actor collected the row

Every post row also carries a placeSearch object recording where it came from: mode (profile_timeline, plain_search, or place_search:<placeMode>), searchTerm, resolvedQuery (the exact string sent to X), geoFiltersApplied (array of geo tokens used), geoFilterVerified, verificationMethod, and product (X's Latest or Top ranking). It's the row-level trail back to the query that produced this specific post.

📍 Place and geo-verification data

When a post carries a real X place object, seven flat columns are added: placeId, placeName, placeFullName, placeType, placeCountry, and placeCountryCode (e.g. "6863fd050de21120", "Leeds", "Leeds, England", "city", "United Kingdom", "GB"), plus placeApiUrl linking to X's own place JSON. These keys are absent, not null, on posts X did not geo-tag — every place search also emits one place_search accounting row per search term carrying geoFilterVerified (true/false/null), verificationMethod (place_object_evidence or control_probe), postsWithPlaceObject, distinctPlaces, distinctCountries, and topPlaces, so you can confirm a run's geographic narrowing actually worked before trusting the results. No coordinate or bounding-box column is shipped — X's own place.bounding_box.coordinates field was measured empty on every sampled place object.

👤 Author profile data

When includeUserInfo is on (default), every post row carries a nested user object: username, userFullName, description (bio), avatar, location, website, joinDate, verified, totalFollowers, totalFollowing, totalTweets, totalMediaCount, totalLikes, url, and userId. This is the field set analysts use to qualify an author (follower count, bio keywords, verified status) without a second lookup.

⚡ How does this differ from the official X API?

X's own API (X API v2) requires a registered developer app, OAuth or bearer credentials, and — for anything beyond a small free allowance — a paid access tier; its search and geographic operators (has:geo, place_country, near, geocode) exist on the API too, but only behind that same paid, approved-app gate. This Actor reaches the identical search grammar through your own already-logged-in browser session — no developer application, no app review, no separate API bill from X.

Feature	X API v2	This Actor
Credentials needed	Registered developer app + OAuth/bearer token, subject to X's approval process	Your own auth_token + ct0 browser cookies (optional — not needed for profile timelines at all)
Setup time	App registration and, for paid tiers, a billing/plan decision	Paste two cookie values, or nothing for profile reads
Geo-filter proof	Returns whatever the query matches; verifying the operator applied is on you	Every place search emits a place_search row with geoFilterVerified and the evidence behind it
Profile timeline access	Requires the same authenticated app	Works fully logged out
Output shape	Raw X GraphQL/REST JSON, deeply nested	Flat, typed dataset rows ready for a table or CSV
Result cap per call	Governed by the developer's plan and X's own pagination	maxTweets (1–100) per target, capped by this Actor regardless of plan

Why does bulk access to X's own API cost money?

As of 2026, X's developer platform gates meaningful read/search volume behind paid access — reported by several third-party X API pricing trackers as of 2026-08-04 as a pay-per-use model (with legacy flat-rate subscription tiers being phased out for existing customers). Either way, it means a developer account, an approved app, and an ongoing bill are the starting point for programmatic X access at any scale. This Actor sidesteps that: profile timelines need nothing, and search/place filters need only the same session cookies your browser already holds.

Use X's own API when you need write access (posting), real-time streaming, or a support relationship with X. Use this Actor when you want structured post and profile data fast, without standing up a developer account.

💡 How to use data extracted from X

Social listening and brand monitoring teams

Feed placeSearchTerms with your brand or product name plus placeCountry set to each market you operate in, and pull text, likes, views, authorLocationText, and placeCountryCode into a dashboard. Because onlyGeoTaggedPosts or a country filter narrows the stream to posts X itself attached a place to, you get a genuinely regional read on sentiment rather than a keyword search that happens to include people talking about somewhere else.

Agencies running repeated client audits

Point startUrls at a client's own handle plus their top competitors', run on a schedule, and diff totalFollowers, likes, and text week over week from the nested user object and post fields. The same setup, re-run with placeCountry swapped, turns into a quick city-by-city or country-by-country activity audit for a client opening a new market.

Market research and geographic intelligence

Run nearPlace or latitude/longitude with radiusKm across a set of cities, and use each run's place_search row (distinctPlaces, distinctCountries, postsWithPlaceObject) to compare posting density and topic mix by location — useful for site-selection or expansion research where you need to know where a topic is actually being discussed, not just mentioned globally.

AI agents and automated pipelines

Because output is flat JSON keyed the same way on every run, this Actor slots into a RAG pipeline or an agent tool call: fetch recent geo-tagged posts about a topic in a target city, hand the text and placeFullName fields to an LLM for summarization, and repeat on a schedule without touching a parser.

🔼 Input sample

Every input is read directly from .actor/input_schema.json; startUrls is the only required field.

Parameter	Required	Type	Description	Example value
startUrls	Yes	array	Profile URLs, bare handles, @handle, numeric user IDs, or search terms. A term with a space, a leading #, or a search: prefix is treated as a search; anything that looks like a handle reads that profile's timeline. Profile timelines work with no cookie.	["https://x.com/nasa", "search: street food"]
placeSearchTerms	No	array	Search terms that always run through the place filters below. Leave empty to search on the place filters alone.	["coffee", "traffic"]
onlyGeoTaggedPosts	No	boolean	Sends X's has:geo operator so every returned post carries a place object. Default true. X will not combine this with a radius search — if a radius is also set, the radius wins and this is dropped with a log warning.	true
placeCountry	No	string	Two-letter ISO-3166 country code sent as place_country:. Default "".	"GB"
placeId	No	string	X's own place identifier, sent as place:. Default "".	"6863fd050de21120"
nearPlace	No	string	Free-text place name sent as near:"…" within:<radius>km. Default "". X does not attach a place object to these posts.	"London"
latitude	No	string	Decimal degrees, −90 to 90, as text. Latitude and longitude together override nearPlace. Default "".	"40.7128"
longitude	No	string	Decimal degrees, −180 to 180. Only used when latitude is set. Default "".	"-74.0060"
radiusKm	No	integer	Radius in kilometres for the named-place or coordinate search. Min 1, max 500, default 25.	50
onlyWithMedia	No	boolean	Adds filter:media so returned place-search posts carry a photo or video. Default false.	false
mentioningHandle	No	string	Restrict the place search to posts mentioning one account. Default "".	"nasa"
postLanguage	No	string	ISO 639-1 code sent as lang:. Leave empty for every language. Default "".	"en"
verifyGeoFilter	No	boolean	Proves the geographic operator was honoured; logs an error if it wasn't. Default true.	true
authToken	No	string (secret)	Your X auth_token session cookie. Needed only for search and place filters.	(your cookie value)
ct0	No	string (secret)	Your X ct0 cookie from the same session as authToken. Sent as a cookie and as x-csrf-token.	(your cookie value)
sortOrder	No	string (enum)	newest (X "Latest"), oldest, or popular (X "Top", sorted by likes). Default "newest".	"popular"
maxTweets	No	integer	Upper bound on posts per profile/search term. Min 1, max 100, default 10.	20
withReplies	No	boolean	Keep posts that are replies to someone else. Default true.	true
includeUserInfo	No	boolean	Attach the author's profile object to every post row. Default true.	true
proxyConfiguration	No	object	Apify Proxy configuration; every outbound request goes through it. Default {"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}.	see below

{
  "startUrls": ["https://x.com/nasa"],
  "placeSearchTerms": ["street food"],
  "placeCountry": "JP",
  "onlyGeoTaggedPosts": false,
  "onlyWithMedia": true,
  "sortOrder": "popular",
  "maxTweets": 20,
  "verifyGeoFilter": true,
  "authToken": "YOUR_AUTH_TOKEN_COOKIE",
  "ct0": "YOUR_CT0_COOKIE",
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

Common pitfall: onlyGeoTaggedPosts and a radius search (nearPlace, or latitude+longitude) cannot run together — X returns zero rows for that combination. Set one or the other; if both are set, the radius wins and has:geo is silently dropped (with a warning in the run log). Also, placeSearchTerms and every place filter need authToken + ct0 — X hard-404s its search endpoint for logged-out callers, so leaving those cookies blank means only startUrls profile timelines will return rows.

🔽 Output sample

Every row is typed, flat JSON with a stable key set per rowType. Export straight from the dataset as JSON, CSV, Excel, XML, or RSS.

{
  "rowType": "post",
  "id": "1823456789012345678",
  "url": "https://x.com/streetfoodjp/status/1823456789012345678",
  "verified": false,
  "username": "@streetfoodjp",
  "fullname": "Street Food Japan",
  "timestamp": "Tue Aug 04 06:12:41 +0000 2026",
  "timestampIso": "2026-08-04T06:12:41Z",
  "text": "Takoyaki stall lines wrapping the block in Osaka this morning 🐙",
  "links": [],
  "isPinned": false,
  "isQuote": false,
  "isRetweet": false,
  "isReply": false,
  "likes": 214,
  "replies": 12,
  "retweets": 31,
  "quotes": 2,
  "views": 9840,
  "viewCount": 9840,
  "mediaCount": 1,
  "mediaTypes": ["photo"],
  "hasPhoto": true,
  "hasVideo": false,
  "media": [{ "type": "photo", "url": "https://pbs.twimg.com/media/example.jpg" }],
  "tweetUserId": "998877665",
  "retweetedFrom": null,
  "originalTweetId": null,
  "isGeoTagged": true,
  "placeId": "6d3a3f1f9b8c7e21",
  "placeName": "Osaka",
  "placeFullName": "Osaka, Japan",
  "placeType": "city",
  "placeCountry": "Japan",
  "placeCountryCode": "JP",
  "placeApiUrl": "https://api.x.com/1.1/geo/id/6d3a3f1f9b8c7e21.json",
  "language": "en",
  "conversationId": "1823456789012345678",
  "bookmarkCount": 5,
  "authorLocationText": "Osaka, Japan",
  "authorVerifiedType": null,
  "sourceTarget": "street food",
  "user": {
    "avatar": "https://pbs.twimg.com/profile_images/example_normal.jpg",
    "username": "streetfoodjp",
    "userFullName": "Street Food Japan",
    "description": "Documenting street food stalls across Japan.",
    "location": "Osaka, Japan",
    "website": "https://streetfoodjapan.example",
    "joinDate": "Wed Mar 14 09:00:00 +0000 2018",
    "verified": false,
    "totalLikes": 4021,
    "totalTweets": 6188,
    "totalFollowing": 340,
    "totalFollowers": 28110,
    "url": "https://streetfoodjapan.example",
    "userId": "998877665",
    "totalMediaCount": 2210
  },
  "placeSearch": {
    "mode": "place_search:country",
    "searchTerm": "street food",
    "resolvedQuery": "street food place_country:JP",
    "geoFiltersApplied": ["place_country:JP"],
    "geoFilterVerified": true,
    "verificationMethod": "place_object_evidence",
    "product": "Top"
  },
  "scrapedAt": "2026-08-04T06:15:02Z"
}

The place_search accounting row

Each place-filtered search term (from placeSearchTerms, or from a startUrls entry combined with a place filter) also emits one accounting row with "rowType": "place_search", alongside its post rows. This is the full field set, read from the summary dict built in _run_search_job() (src/main.py):

Field	Example value
rowType	"place_search"
searchTerm	"street food"
placeMode	"country" (geo_tagged / radius / place_id / country / plain)
resolvedQuery	"street food place_country:JP"
controlQuery	"street food" (the same query without geo tokens, used for radius verification)
product	"Top" or "Latest"
geoFiltersApplied	["place_country:JP"]
otherFiltersApplied	["filter:media"]
droppedTokens	["has:geo"] when the has:geo/radius conflict dropped a token, else []
nearPlace	null unless a named-place radius search was used
latitude / longitude	null unless a coordinate radius search was used
radiusKm	null unless a radius search was used
placeIdFilter	null unless placeId was set
placeCountryFilter	"JP"
geoFilterVerified	true / false / null (null when verification was skipped or couldn't run)
verificationMethod	"place_object_evidence" or "control_probe"
controlRowCount	null unless a radius search ran the control probe
controlOverlap	null unless a radius search ran the control probe
geoRowCount	20 — raw post count fetched before sorting/cutoff
postsReturned	18 — posts actually saved after sort and maxTweets cutoff
postsWithPlaceObject	18
placeCoverage	"18 of 18"
distinctPlaces	6
distinctCountries	1
topPlaces	["Fukuoka, Japan", "Osaka, Japan", "Tokyo, Japan"] (up to 10, sorted)
skippedReason	null, or "auth_required" / "no_results" / an error reason
rateLimitRemaining	147 (X's x-rate-limit-remaining header at push time)
scrapedAt	"2026-08-04T06:15:05Z"

Filter these out of downstream analysis with rowType == "post" if you only want the tweets themselves. They are useful on their own for auditing a run — but note they are pushed to the dataset and billed the same as post rows under the row_result charged event, so a run with many place searches and few matching posts still consumes events for the summary rows.

🎯 How do you filter and target specific posts?

Getting the right slice of X starts with choosing the right target type in startUrls: a handle or profile URL reads that account's timeline directly (fast, no cookie needed), while a bare keyword or search:-prefixed term runs a full X search — use search whenever you want posts from anyone, not one account. placeSearchTerms always routes through the place filters, so use it when the term itself is secondary to where the post came from.

Scope precision comes from four mutually-exclusive geographic modes, in priority order: an exact placeId pins to one X-defined place; placeCountry narrows to a whole country; nearPlace or a latitude/longitude pair searches a radius (1–500 km, set via radiusKm) around a point — but note the measured conflict: onlyGeoTaggedPosts and any radius search cannot combine, and the radius always wins. Quality thresholds are onlyWithMedia (photo/video only), postLanguage (ISO 639-1), and mentioningHandle to anchor a place search to posts about one account. Volume is controlled by maxTweets (1–100 per target) and sortOrder — since the sort runs before the row-count cutoff, popular genuinely returns the highest-liked posts rather than the first ones found, which matters most on sparse radius searches.

Every geo-tagged post in a country, only with media:

{ "startUrls": [], "placeSearchTerms": [], "placeCountry": "FR", "onlyGeoTaggedPosts": true, "onlyWithMedia": true, "maxTweets": 50 }

Posts about a keyword within 25 km of a named place:

{ "startUrls": [], "placeSearchTerms": ["food truck"], "nearPlace": "Austin", "radiusKm": 25, "sortOrder": "popular" }

Posts carrying one exact X place ID, mentioning a specific account:

{ "startUrls": [], "placeSearchTerms": [""], "placeId": "6863fd050de21120", "mentioningHandle": "nasa", "maxTweets": 30 }

▶️ Want to try other SimpleAPI scrapers?

Scraper	What it extracts
Twitter X Tweets Scraper Scraper By Keyword & Hashtag	X posts by keyword or hashtag with typed filters (date window, engagement floors, language, media, from/to) compiled into X's search grammar
Threads Search Post Scraper	Threads posts by search term
LinkedIn Profile Posts Scraper By Media & Hashtag	LinkedIn profile posts filtered by media type and hashtag
Reddit Subreddit Members Scraper By Time Window	Public Reddit accounts that posted or commented in a subreddit within a time window
Pinterest Profile Scraper With Email & Website Contacts	Public Pinterest profiles plus contact details published on the account's bio and linked website

🧑‍💻 How to extract X data programmatically

This Actor runs as a standard Apify Actor: one API call starts the run, and results come back as structured JSON from the dataset — no separate X API key involved.

Python example

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_API_TOKEN>")

run_input = {
    "startUrls": [],
    "placeSearchTerms": ["street food"],
    "placeCountry": "JP",
    "authToken": "YOUR_AUTH_TOKEN_COOKIE",
    "ct0": "YOUR_CT0_COOKIE",
    "maxTweets": 20,
}

run = client.actor("YOUR_USERNAME/x-twitter-user-profile-and-tweets-scraper-posts-from-a-place").call(
    run_input=run_input
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("rowType") != "post":
        continue
    print(item["username"], item.get("placeFullName"), item["likes"])

Export to spreadsheets or CRM

From the run's dataset, use Export results → CSV (or the Apify API's ?format=csv dataset endpoint) to map username, text, likes, views, placeFullName, and authorLocationText straight into spreadsheet columns or a CRM import template — no post-processing needed since the row is already flat.

⚖️ Is it legal to scrape X (Twitter)?

Scraping publicly visible X posts and profile pages is generally legal — the data is published for public viewing, and no login is required to read a profile timeline with this Actor. Because the output includes personal data (usernames, display names, bios, follower counts, self-typed location, and post content tied to identifiable accounts), GDPR and CCPA considerations apply to how you store and use it, not to the act of reading a public page: you need a lawful basis for retention, should honor deletion/access requests, and should avoid re-identifying or profiling individuals beyond what your stated purpose requires. Search and place filters additionally require your own X session, so you are bound by X's Terms of Service when you supply your own cookies for that traffic. Consult legal counsel for commercial applications involving bulk storage of personal data.

❓ FAQ

What happens to posts that get deleted or accounts that get suspended?

This Actor reflects X's live state at request time — it doesn't retroactively flag rows already saved in a previous run's dataset. Re-running against the same handle simply won't return a deleted post's ID again; a suspended or protected account resolves with "not a public X account" logged and the target skipped for that run.

Can I get engagement metrics along with the post text?

Yes — likes, replies, retweets, quotes, and views/viewCount are on every post row by default; no separate flag needed. One caveat from the source: X occasionally serves a whole session with the views entitlement stripped (observed roughly once in five runs) — when that happens, views and viewCount come back null for every row in that run, and the Actor logs a warning rather than guessing a number. Re-running usually restores it.

How accurate is the place/location data?

The place fields (placeId, placeName, placeCountry, etc.) are exactly the place object X itself attached to the post server-side — this Actor doesn't infer or geocode locations. verifyGeoFilter (on by default) adds a second layer: it proves the requested filter actually matched what came back, either from the payload itself or by re-running the same query without its geographic tokens and comparing post-ID sets. Check geoFilterVerified on the run's place_search row before trusting a "no results" or high-volume run.

How many posts can I get per run?

maxTweets caps results at 1–100 per profile or per search term (default 10) — this is a hard ceiling in the Actor's code, not just the input schema. In practice, narrow place searches often return fewer: X's search index returns roughly 20 posts per page, profile timelines stop after 5 pages and searches after 8 pages, and a search stops early if two consecutive pages come back empty even with a live cursor — geographic search results on X are genuinely sparse.

How do I confirm my place filter really narrowed the results?

Use the default verifyGeoFilter: true and check the place_search accounting row it emits for that search term. For has:geo, place:, and place_country: filters, every returned post must carry a matching place object — the Actor checks this for free from the response itself. For radius searches (nearPlace or latitude/longitude), the same query is re-run once without its geographic tokens and the two result sets are compared; identical sets mean X ignored the operator, and that's logged as an error rather than handed to you silently.

Does this scraper work with Claude, ChatGPT, and AI agent frameworks?

There's no MCP server for this Actor. It's callable as a standard Apify Actor via the Apify API or apify-client from any agent framework capable of making an HTTP request — feed it a target and read the resulting dataset rows as tool output.

Can I use this without an X API key or developer account?

Yes for profile timelines — no cookie, key, or account of any kind is required. Search and every place filter need your own X session cookies (auth_token and ct0, taken from a logged-in browser), because X's search endpoint blocks logged-out requests with an HTTP 404. Neither of those is an X developer account or API key — they're the cookies your own browser already holds.

Conclusion

This Actor turns X profile timelines and place-filtered searches into flat, typed JSON — post content, engagement, media, author profile, and a verified place object when one exists — without requiring an X developer account for any of it. Its distinguishing feature is proof, not just filtering: every geo-targeted search reports whether X actually honoured the operator, so you're not left guessing when a place search comes back empty or suspiciously broad. Start a run from the Apify Console with a handle, a search term, or a place filter, and pull the results straight from the dataset.