X (Twitter) Ads Transparency Scraper

Scrape information on ads from the X (Twitter) Ads Transparency Center. Extract ad details — advertiser name, ad ID, ad text, spend, impressions, targeting, timing — and filter by advertiser, keyword, or location. Use the data for competitor analysis, ad intelligence, and campaign monitoring.

What data sources does it cover?

Source	Period	Data type
Political	May 2018 – Nov 2019	US political campaign ads with spend, impressions, targeting, billing
Issue	Aug 2018 – Nov 2019	US issue/advocacy ads (same schema as political)
Commercial	Dec 2023 snapshot	All promoted tweets (~1.37 GB CSV). Optional tweet hydration for engagement data
DSA Repository	Live / Current	EU-served ads via X's GraphQL API. Requires X API bearer token

Competitive advantage: This is one of the few actors that supports the live DSA Ads Repository for current EU ad data, not just the historical archives.

Features

• Multi-source scraping — Combine political, issue, commercial, and DSA data in a single run
• Smart filtering — Filter by advertiser name/handle, keyword (searches across ad text, URLs, agency), geographic location, date range, minimum spend, and minimum impressions
• Tweet hydration — Enrich commercial archive tweets with engagement data (likes, retweets, replies, media URLs)
• Multiple export formats — Dataset available as JSON and CSV by default, plus optional HTML report and Excel export
• Webhook notifications — Get notified via webhook when ads match your filters. Perfect for monitoring competitor campaigns
• Memory efficient — Streams the large commercial archive (~1.37 GB) line by line instead of loading into memory

Input configuration

Field	Type	Default	Description
sources	array	["political"]	Data sources to scrape: political, issue, commercial, dsa
advertisers	array	[]	Filter by advertiser handle or name (case-insensitive)
keyword	string	""	Search across ad text, advertiser, agency, URLs
location	string	""	Filter by geographic targeting location
startDate	string	""	Ads created on or after this date (YYYY-MM-DD)
endDate	string	""	Ads created on or before this date (YYYY-MM-DD)
minSpendUsd	integer	—	Minimum ad spend in USD (political/issue only)
minImpressions	integer	—	Minimum impressions (political/issue only)
maxItems	integer	1000	Max records to return
hydrateTweets	boolean	false	Fetch engagement data for commercial tweets
dsaAdvertiserUserId	string	""	X user ID for DSA lookup (required with dsa source)
dsaCountry	string	""	ISO country code for DSA filter (e.g. DE, FR)
bearerToken	string	""	X API bearer token (required for dsa source)
exportFormat	string	"none"	Extra export: html, excel, or none
webhookUrl	string	""	URL for webhook notifications on matching ads
maxConcurrency	integer	10	Parallel requests for hydration

Usage examples

1. Search political ads by keyword

{
    "sources": ["political"],
    "keyword": "climate",
    "maxItems": 500
}

2. Filter by advertiser across all archives

{
    "sources": ["political", "issue", "commercial"],
    "advertisers": ["@senwarren", "Elizabeth Warren"],
    "maxItems": 2000
}

3. Current EU ads via DSA Repository

{
    "sources": ["dsa"],
    "dsaAdvertiserUserId": "123456789",
    "dsaCountry": "DE",
    "bearerToken": "AAAA...your_token",
    "exportFormat": "excel"
}

4. Competitor monitoring with webhook

{
    "sources": ["political", "commercial"],
    "advertisers": ["competitor_handle"],
    "keyword": "sale",
    "webhookUrl": "https://hooks.slack.com/services/YOUR/WEBHOOK/URL",
    "maxItems": 100
}

5. Location-based ad analysis

{
    "sources": ["political", "issue"],
    "location": "California",
    "minSpendUsd": 1000,
    "startDate": "2019-01-01",
    "endDate": "2019-06-30",
    "exportFormat": "html"
}

Output schema

Each record in the dataset contains:

Field	Type	Description
source	string	political, issue, commercial, or dsa
adType	string	Ad type label
advertiserHandle	string	X handle of the advertiser
advertiserName	string	Display name of the advertiser
advertisingAgency	string	Agency name (if available)
tweetId	string	Tweet/ad ID
tweetUrl	string	Direct link to the tweet
createdAt	string	Creation or start date
tweetText	string	Ad creative text
reportedUrls	array	URLs referenced in the ad
totalSpend	number	Total spend in USD (political/issue)
totalImpressions	number	Total impressions
campaigns	array	Campaign details with targeting
targeting	array	Targeting criteria (geo, age, gender, language)
targetingCountries	array	Geographic targeting locations
insertionOrders	array	Billing — insertion orders
creditCardBilling	array	Billing — credit card entries
hydration	object	Engagement data (commercial, when hydrated)

Export formats

• JSON / CSV — Always available via the Apify dataset
• HTML — A styled, sortable report saved to key-value store as EXPORT_HTML
• Excel — A formatted .xlsx workbook saved as EXPORT_EXCEL

Access exports from the key-value store tab after the run completes.

Webhook notifications

Set webhookUrl to receive a POST request with matching ads:

{
    "event": "new_ads_found",
    "timestamp": "2024-01-15T10:30:00.000Z",
    "totalAds": 5,
    "filters": {
        "advertisers": ["competitor"],
        "keyword": "sale",
        "location": ""
    },
    "sources": ["political", "commercial"],
    "ads": [
        {
            "source": "political",
            "advertiserHandle": "competitor",
            "tweetText": "Our big sale starts...",
            "totalSpend": 5000,
            "totalImpressions": 250000
        }
    ]
}

Schedule this actor with Apify's scheduler to get periodic notifications of new campaigns.

How it works

1. Political & Issue archives — Downloads the official JSON archives from X, parses advertiser records, and flattens each promoted tweet into a normalized row
2. Commercial archive — Streams the ~1.37 GB CSV line by line (memory efficient). Optionally hydrates tweets from X's syndication endpoint for engagement data
3. DSA Repository — Creates an export report via X's GraphQL API, polls for completion, downloads and parses the resulting CSV
4. Filtering — All records pass through the filter chain: advertiser, keyword, location, date, spend, impressions
5. Export — Results are pushed to the dataset (JSON/CSV) and optionally to HTML or Excel in the key-value store

Pricing

This actor uses a pay-per-event billing model:

Event	Cost
Ad record emitted	$0.005 per record

Example: Scraping 1,000 ads costs approximately $5.00 plus Apify platform usage.

FAQ

Q: Can I scrape current live ads? A: Yes — use the dsa source with your X API bearer token. This gives you current ads served in the EU. The political, issue, and commercial sources are historical archives.

Q: Why do I need a bearer token for DSA? A: X's DSA Ads Repository requires authentication via their developer API. Create a free developer account at developer.x.com and generate a bearer token.

Q: How do I set up campaign monitoring? A: Schedule this actor to run periodically (e.g., daily) with your advertiser/keyword filters and a webhookUrl. You'll receive notifications whenever matching ads are found.

Q: The commercial archive is huge. Will it run out of memory? A: No. The commercial archive is streamed line by line. Set maxItems to limit output and the actor stops as soon as the limit is reached.

Limitations

• Political and issue archives cover 2018–2019 only (X banned political ads in Nov 2019)
• Commercial archive is a Dec 2023 snapshot, not continuously updated
• DSA repository covers EU-served ads only
• Tweet hydration may fail for deleted, suspended, or protected tweets
• Billing data (insertion orders, credit cards) is account-scoped and may repeat across ads
• This actor is independent and not endorsed by X Corp.