# 🌟 X (Twitter) Ads Transparency Scraper (`databuddy/x-ads-transparency-scraper`) Actor

Scrape X/Twitter ads from the Ads Transparency Center. Covers political, issue, commercial archives and the live DSA Ads Repository for current EU ads. Filter by advertiser, keyword, or location. Export to HTML, JSON, CSV, or Excel. Get webhook notifications for new campaigns.

- **URL**: https://apify.com/databuddy/x-ads-transparency-scraper.md
- **Developed by:** [Sarthak](https://apify.com/databuddy) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 ad record scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## X (Twitter) Ads Transparency Scraper

Scrape information on ads from the X (Twitter) Ads Transparency Center. Extract ad details — advertiser name, ad ID, ad text, spend, impressions, targeting, timing — and filter by advertiser, keyword, or location. Use the data for **competitor analysis**, **ad intelligence**, and **campaign monitoring**.

### What data sources does it cover?

| Source | Period | Data type |
|--------|--------|-----------|
| **Political** | May 2018 – Nov 2019 | US political campaign ads with spend, impressions, targeting, billing |
| **Issue** | Aug 2018 – Nov 2019 | US issue/advocacy ads (same schema as political) |
| **Commercial** | Dec 2023 snapshot | All promoted tweets (~1.37 GB CSV). Optional tweet hydration for engagement data |
| **DSA Repository** | **Live / Current** | EU-served ads via X's GraphQL API. Requires X API bearer token |

> **Competitive advantage:** This is one of the few actors that supports the **live DSA Ads Repository** for current EU ad data, not just the historical archives.

### Features

- **Multi-source scraping** — Combine political, issue, commercial, and DSA data in a single run
- **Smart filtering** — Filter by advertiser name/handle, keyword (searches across ad text, URLs, agency), geographic location, date range, minimum spend, and minimum impressions
- **Tweet hydration** — Enrich commercial archive tweets with engagement data (likes, retweets, replies, media URLs)
- **Multiple export formats** — Dataset available as JSON and CSV by default, plus optional HTML report and Excel export
- **Webhook notifications** — Get notified via webhook when ads match your filters. Perfect for monitoring competitor campaigns
- **Memory efficient** — Streams the large commercial archive (~1.37 GB) line by line instead of loading into memory

### Input configuration

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `sources` | array | `["political"]` | Data sources to scrape: `political`, `issue`, `commercial`, `dsa` |
| `advertisers` | array | `[]` | Filter by advertiser handle or name (case-insensitive) |
| `keyword` | string | `""` | Search across ad text, advertiser, agency, URLs |
| `location` | string | `""` | Filter by geographic targeting location |
| `startDate` | string | `""` | Ads created on or after this date (`YYYY-MM-DD`) |
| `endDate` | string | `""` | Ads created on or before this date (`YYYY-MM-DD`) |
| `minSpendUsd` | integer | — | Minimum ad spend in USD (political/issue only) |
| `minImpressions` | integer | — | Minimum impressions (political/issue only) |
| `maxItems` | integer | `1000` | Max records to return |
| `hydrateTweets` | boolean | `false` | Fetch engagement data for commercial tweets |
| `dsaAdvertiserUserId` | string | `""` | X user ID for DSA lookup (required with `dsa` source) |
| `dsaCountry` | string | `""` | ISO country code for DSA filter (e.g. `DE`, `FR`) |
| `bearerToken` | string | `""` | X API bearer token (required for `dsa` source) |
| `exportFormat` | string | `"none"` | Extra export: `html`, `excel`, or `none` |
| `webhookUrl` | string | `""` | URL for webhook notifications on matching ads |
| `maxConcurrency` | integer | `10` | Parallel requests for hydration |

### Usage examples

#### 1. Search political ads by keyword

```json
{
    "sources": ["political"],
    "keyword": "climate",
    "maxItems": 500
}
```

#### 2. Filter by advertiser across all archives

```json
{
    "sources": ["political", "issue", "commercial"],
    "advertisers": ["@senwarren", "Elizabeth Warren"],
    "maxItems": 2000
}
```

#### 3. Current EU ads via DSA Repository

```json
{
    "sources": ["dsa"],
    "dsaAdvertiserUserId": "123456789",
    "dsaCountry": "DE",
    "bearerToken": "AAAA...your_token",
    "exportFormat": "excel"
}
```

#### 4. Competitor monitoring with webhook

```json
{
    "sources": ["political", "commercial"],
    "advertisers": ["competitor_handle"],
    "keyword": "sale",
    "webhookUrl": "https://hooks.slack.com/services/YOUR/WEBHOOK/URL",
    "maxItems": 100
}
```

#### 5. Location-based ad analysis

```json
{
    "sources": ["political", "issue"],
    "location": "California",
    "minSpendUsd": 1000,
    "startDate": "2019-01-01",
    "endDate": "2019-06-30",
    "exportFormat": "html"
}
```

### Output schema

Each record in the dataset contains:

| Field | Type | Description |
|-------|------|-------------|
| `source` | string | `political`, `issue`, `commercial`, or `dsa` |
| `adType` | string | Ad type label |
| `advertiserHandle` | string | X handle of the advertiser |
| `advertiserName` | string | Display name of the advertiser |
| `advertisingAgency` | string | Agency name (if available) |
| `tweetId` | string | Tweet/ad ID |
| `tweetUrl` | string | Direct link to the tweet |
| `createdAt` | string | Creation or start date |
| `tweetText` | string | Ad creative text |
| `reportedUrls` | array | URLs referenced in the ad |
| `totalSpend` | number | Total spend in USD (political/issue) |
| `totalImpressions` | number | Total impressions |
| `campaigns` | array | Campaign details with targeting |
| `targeting` | array | Targeting criteria (geo, age, gender, language) |
| `targetingCountries` | array | Geographic targeting locations |
| `insertionOrders` | array | Billing — insertion orders |
| `creditCardBilling` | array | Billing — credit card entries |
| `hydration` | object | Engagement data (commercial, when hydrated) |

### Export formats

- **JSON / CSV** — Always available via the Apify dataset
- **HTML** — A styled, sortable report saved to key-value store as `EXPORT_HTML`
- **Excel** — A formatted `.xlsx` workbook saved as `EXPORT_EXCEL`

Access exports from the key-value store tab after the run completes.

### Webhook notifications

Set `webhookUrl` to receive a POST request with matching ads:

```json
{
    "event": "new_ads_found",
    "timestamp": "2024-01-15T10:30:00.000Z",
    "totalAds": 5,
    "filters": {
        "advertisers": ["competitor"],
        "keyword": "sale",
        "location": ""
    },
    "sources": ["political", "commercial"],
    "ads": [
        {
            "source": "political",
            "advertiserHandle": "competitor",
            "tweetText": "Our big sale starts...",
            "totalSpend": 5000,
            "totalImpressions": 250000
        }
    ]
}
```

Schedule this actor with Apify's scheduler to get periodic notifications of new campaigns.

### How it works

1. **Political & Issue archives** — Downloads the official JSON archives from X, parses advertiser records, and flattens each promoted tweet into a normalized row
2. **Commercial archive** — Streams the ~1.37 GB CSV line by line (memory efficient). Optionally hydrates tweets from X's syndication endpoint for engagement data
3. **DSA Repository** — Creates an export report via X's GraphQL API, polls for completion, downloads and parses the resulting CSV
4. **Filtering** — All records pass through the filter chain: advertiser, keyword, location, date, spend, impressions
5. **Export** — Results are pushed to the dataset (JSON/CSV) and optionally to HTML or Excel in the key-value store

### Pricing

This actor uses a **pay-per-event** billing model:

| Event | Cost |
|-------|------|
| Ad record emitted | $0.005 per record |

Example: Scraping 1,000 ads costs approximately **$5.00** plus Apify platform usage.

### FAQ

**Q: Can I scrape current live ads?**
A: Yes — use the `dsa` source with your X API bearer token. This gives you current ads served in the EU. The political, issue, and commercial sources are historical archives.

**Q: Why do I need a bearer token for DSA?**
A: X's DSA Ads Repository requires authentication via their developer API. Create a free developer account at [developer.x.com](https://developer.x.com) and generate a bearer token.

**Q: How do I set up campaign monitoring?**
A: Schedule this actor to run periodically (e.g., daily) with your advertiser/keyword filters and a `webhookUrl`. You'll receive notifications whenever matching ads are found.

**Q: The commercial archive is huge. Will it run out of memory?**
A: No. The commercial archive is streamed line by line. Set `maxItems` to limit output and the actor stops as soon as the limit is reached.

### Limitations

- Political and issue archives cover 2018–2019 only (X banned political ads in Nov 2019)
- Commercial archive is a Dec 2023 snapshot, not continuously updated
- DSA repository covers EU-served ads only
- Tweet hydration may fail for deleted, suspended, or protected tweets
- Billing data (insertion orders, credit cards) is account-scoped and may repeat across ads
- This actor is independent and not endorsed by X Corp.

# Actor input Schema

## `sources` (type: `array`):

Which ad archives to scrape. 'political' and 'issue' are 2018-2019 US archives. 'commercial' is the Dec 2023 promoted tweet snapshot. 'dsa' is the live EU DSA Ads Repository (requires X API bearer token).

## `advertisers` (type: `array`):

Filter ads by advertiser handle or name. Case-insensitive substring match. Leave empty for all advertisers.

## `keyword` (type: `string`):

Search keyword to match against ad text, advertiser name/handle, agency, and reported URLs. Case-insensitive.

## `location` (type: `string`):

Filter ads by geographic targeting location (e.g. 'United States', 'California', 'New York'). Matches against targeting geo values. Case-insensitive.

## `startDate` (type: `string`):

Only include ads created on or after this date (YYYY-MM-DD).

## `endDate` (type: `string`):

Only include ads created on or before this date (YYYY-MM-DD).

## `minSpendUsd` (type: `integer`):

Only include ads with total spend >= this amount. Applies to political/issue sources only.

## `minImpressions` (type: `integer`):

Only include ads with total impressions >= this amount. Applies to political/issue sources only.

## `maxItems` (type: `integer`):

Maximum number of ad records to return across all sources.

## `hydrateTweets` (type: `boolean`):

For commercial archive: fetch full tweet data (engagement, media) from X's syndication endpoint.

## `dsaAdvertiserUserId` (type: `string`):

X user ID of the advertiser to look up in the DSA repository. Required when 'dsa' source is selected.

## `dsaCountry` (type: `string`):

ISO country code for DSA repository query (e.g. 'DE', 'FR', 'IT'). Defaults to all EU countries.

## `bearerToken` (type: `string`):

Bearer token from your X Developer account. Required for DSA repository access.

## `exportFormat` (type: `string`):

Additional export format saved to key-value store. Dataset always available as JSON/CSV. Choose 'html' or 'excel' for extra exports.

## `webhookUrl` (type: `string`):

URL to POST new ad records to. Use for notifications when new campaigns match your filters. Each new ad is sent as a JSON payload.

## `maxConcurrency` (type: `integer`):

Maximum parallel requests for tweet hydration and live checks.

## `proxyConfiguration` (type: `object`):

Optional proxy settings. Rarely needed for archive downloads.

## Actor input object example

```json
{
  "sources": [
    "political"
  ],
  "advertisers": [],
  "keyword": "",
  "location": "",
  "startDate": "",
  "endDate": "",
  "maxItems": 1000,
  "hydrateTweets": false,
  "dsaAdvertiserUserId": "",
  "dsaCountry": "",
  "exportFormat": "none",
  "webhookUrl": "",
  "maxConcurrency": 10
}
```

# Actor output Schema

## `defaultDataset` (type: `string`):

Scraped ad transparency records

## `runSummary` (type: `string`):

Scraping statistics and filter details

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sources": [
        "political"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("databuddy/x-ads-transparency-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "sources": ["political"] }

# Run the Actor and wait for it to finish
run = client.actor("databuddy/x-ads-transparency-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sources": [
    "political"
  ]
}' |
apify call databuddy/x-ads-transparency-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,databuddy/x-ads-transparency-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Q6FxLuzshrm3znj4C/builds/eqml6sDpLim6eCNF2/openapi.json