X-Signal

Twitter/X Monitoring & Intelligence for AI Agents

Stateful Twitter/X monitoring with automatic deduplication, NLP enrichment (sentiment, intent, entities), webhook alerts, and an MCP tool interface that lets any AI agent search, monitor, and analyze Twitter in real-time.

---

What is X-Signal?

X-Signal is an Apify Actor that turns Twitter/X into a structured intelligence feed. Instead of raw tweets, you get enriched signals — each tweet automatically analyzed for sentiment, buyer intent, entities, and language — delivered only once, never duplicated.

Perfect for:

• AI Agents — Give your agent real-time Twitter awareness via MCP tools
• Brand Monitoring — Track mentions with sentiment and intent classification
• Lead Generation — Find buyer-intent tweets in real-time ("looking for a tool that...")
• Market Research — Aggregate sentiment and trending entities for any topic
• Competitive Intelligence — Monitor competitor mentions with alert webhooks
• Content Discovery — Surface high-engagement tweets matching your criteria

---

Key Features

Stateful Deduplication

Every run remembers what it already delivered. Schedule it hourly or daily — you'll only ever get new tweets. Under the hood: cursor-based tracking + Bloom filter + recent-ID window ensures zero duplicates even across millions of tweets.

Two-Tier NLP Enrichment

Tier	Speed	Cost	Accuracy	When Used
Fast-Path (default)	Instant	Free	Good	Always — deterministic lexicon + regex
LLM (opt-in)	~200ms/tweet	$0.003/tweet	Excellent	When llmEnabled=true — GPT-4o-mini

Both tiers produce the same output schema. LLM results are cached — repeat analyses are free.

MCP Interface (AI Agent Ready)

Five tools any MCP-compatible agent can call directly:

x_search        → Search tweets with enrichment (stateless)
x_monitor_run   → Run a monitor with dedup (stateful)
x_analyze       → Aggregate sentiment/intent analysis
x_get_thread    → Reconstruct conversation threads
x_find_intent   → Find tweets by specific intent type

Real-Time Webhook Alerts

Get instant notifications when new signals match your criteria:

• HMAC-SHA256 signed payloads for verification
• Idempotency keys for safe retry handling
• Automatic retry with exponential backoff
• Non-blocking — webhook failures never block your data pipeline

Provider Fallback & Resilience

• Primary + fallback scraper with automatic failover
• Circuit breaker prevents cascading failures
• Configurable retry with jitter
• Graceful degradation at every layer

---

Quick Start

Option 1: Apify Console (No Code)

1. Go to X-Signal on Apify Store
2. Click Start
3. Fill in: Monitor ID, Query, and hit Run

Option 2: Apify CLI

apify call x-signal --input='{
  "monitorId": "my-project.typescript",
  "targetKind": "search",
  "query": "typescript lang:en",
  "maxNewItemsPerRun": 500
}'

Option 3: Apify API

curl -X POST "https://api.apify.com/v2/acts/x-signal/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "monitorId": "brand.mentions",
    "query": "\"your brand\" OR @yourbrand",
    "filter": { "minLikes": 5, "excludeRetweets": true },
    "webhookUrl": "https://your-server.com/webhook"
  }'

Option 4: MCP Agent Integration

Connect your AI agent to X-Signal's MCP endpoint:

// Your agent can now call:
const results = await mcp.call('x_search', {
  query: 'looking for a CRM tool',
  limit: 50
});
// → Returns enriched signals with intent classification

---

Input Reference

Required Fields

Field	Type	Description
monitorId	string	Stable identifier for this monitor. Same ID across runs = same dedup state. Pattern: ^[a-z0-9][a-z0-9._-]{2,63}$
targetKind	string	"search" (keyword search) or "userTimeline" (specific user's tweets)
query	string	Search query (e.g. "typescript") or @handle for user timeline

Optional Fields

Field	Type	Default	Description
backfillWindowHours	integer	24	How far back to look on first run (1–168 hours)
maxNewItemsPerRun	integer	1000	Budget cap: max new signals per run (1–10,000)
enrichThread	boolean	false	Fetch parent conversation thread for context
threadDepth	integer	10	Max parent chain depth when threading (1–50)
providerPrimaryId	string	"apidojo/tweet-scraper"	Primary data source
providerFallbackId	string	"kaitoeasyapi/..."	Fallback source (null = disable)
llmEnabled	boolean	false	Enable LLM-powered enrichment
llmProvider	string	"openai"	Provider: openai / anthropic / google
llmModel	string	"gpt-4o-mini"	Model for enrichment
llmApiKey	string	—	Your LLM API key (required if llmEnabled=true)
llmBatchSize	integer	20	Tweets per LLM batch (1–50)
filter	object	{}	Signal filter (see below)
webhookUrl	string	—	HTTPS webhook endpoint for real-time alerts
webhookSecret	string	—	HMAC signing secret for webhook verification
webhookBatch	boolean	false	Send all signals in one webhook call
keepRaw	boolean	false	Include raw provider data in output
logLevel	string	"info"	Verbosity: debug / info / warn / error

Filter Configuration

Narrow down results with powerful AND-composed filters:

{
  "minLikes": 10,
  "minRetweets": 5,
  "minViews": 1000,
  "languages": ["en", "es"],
  "authorAllow": ["elonmusk", "naval"],
  "authorDeny": ["spambot123"],
  "intentTypes": ["buyer", "support"],
  "excludeRetweets": true,
  "excludeReplies": false
}

Rules:

• All conditions combine with AND logic
• Deny lists always override allow lists
• Null/missing metrics fail floor checks (conservative)
• Empty filter = no filtering (all signals pass)

---

Output Schema

Dataset Signal Row

Every new signal is stored in the Actor's default dataset. This is a stable public contract — fields are only added, never removed.

{
  schemaVersion: 1,
  monitorId: "brand.mentions",
  runId: "abc-123-def",

  // Core tweet data
  tweet: {
    id: "1810000000000000001",
    url: "https://x.com/user/status/1810000000000000001",
    text: "Just switched to TypeScript and it's amazing!",
    createdAt: "2026-07-25T10:30:00.000Z",
    authorHandle: "developer42",
    authorId: "123456789",
    lang: "en",
    engagement: { likes: 47, retweets: 12, replies: 3, quotes: 2, views: 8500 },
    isReply: false,
    isRetweet: false,
    hashtags: ["typescript"],
    mentions: [],
    urls: []
  },

  // NLP Enrichment
  sentiment: { label: "positive", score: 0.82, confidence: 0.91 },
  intent: { type: "neutral", confidence: 0.75, signals: ["switched to", "amazing"] },
  entities: [
    { kind: "product", value: "typescript", confidence: 0.95 }
  ],
  language: { code: "en", confidence: 0.99 },
  enrichmentSource: "fast-path",

  // Thread context (when enrichThread=true)
  thread: { rootId: null, depth: 0, truncated: false, parentIds: [] },

  // Metadata
  emittedAt: "2026-07-25T10:31:05.000Z",
  sourceProvider: "apidojo/tweet-scraper"
}

Run Output (KV Store)

Each run also produces a summary in the key-value store:

{
  "monitorId": "brand.mentions",
  "runId": "abc-123-def",
  "status": "ok",
  "newCount": 47,
  "metrics": {
    "fetchedCount": 200,
    "dedupSkipped": 153,
    "enrichedCount": 47,
    "filteredOut": 0,
    "llmEnriched": 0,
    "cacheHits": 0,
    "webhookDelivered": 47,
    "webhookFailed": 0,
    "durationMs": 4520
  },
  "cursor": {
    "sinceId": "1810000000000000047",
    "sinceTs": "2026-07-25T10:30:00.000Z"
  }
}

---

MCP Tools

X-Signal exposes 5 tools via the Model Context Protocol for AI agent integration. All tools support verbosity ("compact" / "verbose") and fields (allowlist projection).

x_search — Search & Enrich

Stateless Twitter search with automatic NLP enrichment. No state, no dedup — just search and analyze.

// Input
{ "query": "AI startup funding", "limit": 50, "llm": true }

// Output
{ "matches": [{ "id": "...", "text": "...", "sentiment": "positive", ... }], "count": 50 }

Param	Type	Required	Default	Description
query	string	Yes	—	Twitter search query
limit	integer	No	25	Results limit (1–200)
llm	boolean	No	false	Use LLM enrichment

---

x_monitor_run — Stateful Monitor

Execute a monitoring run with full deduplication. Only returns tweets not seen in previous runs.

// Input
{ "monitorId": "leads.saas", "query": "looking for a SaaS tool", "maxNewItems": 100 }

// Output
{ "monitorId": "leads.saas", "runId": "...", "newMatches": [...], "newCount": 23, "cursorAdvancedTo": "1810..." }

Param	Type	Required	Default	Description
monitorId	string	Yes	—	Monitor identifier
targetKind	string	No	"search"	search or userTimeline
query	string	Yes	—	Query or @handle
filter	object	No	null	Filter config
maxNewItems	integer	No	1000	Budget cap

---

x_analyze — Batch Analysis

Aggregate sentiment, intent, entity, and language stats across a set of tweets.

// Input
{ "query": "react vs vue", "limit": 100 }

// Output
{
  "count": 100,
  "sentiment": { "positive": 45, "neutral": 38, "negative": 17, "avgScore": 0.24 },
  "intent": { "neutral": 62, "support": 18, "complaint": 12, "buyer": 8 },
  "topEntities": [{ "value": "react", "kind": "product", "count": 78 }],
  "languages": { "en": 89, "es": 7, "ja": 4 }
}

---

x_get_thread — Thread Reconstruction

Reconstruct a conversation thread from any tweet, walking up the reply chain.

// Input
{ "tweetId": "1810000000000000001", "maxDepth": 20 }

// Output: ordered parent chain (oldest → newest)

Param	Type	Required	Default	Description
tweetId	string	Yes	—	Any tweet ID
maxDepth	integer	No	10	Max parents to fetch (1–50)

---

x_find_intent — Intent Discovery

Find tweets expressing a specific intent. Perfect for lead generation and support monitoring.

// Input
{ "query": "CRM software", "intent": "buyer", "limit": 50, "minConfidence": 0.7 }

// Output
{ "matches": [/* only buyer-intent tweets */], "count": 12 }

Param	Type	Required	Default	Description
query	string	Yes	—	Search scope
intent	string	Yes	—	buyer / support / complaint / lead / news
limit	integer	No	25	Max results (1–200)
minConfidence	number	No	0.5	Confidence threshold (0–1)
llm	boolean	No	false	Use LLM for better accuracy

---

Pricing (Pay-Per-Event)

You only pay for what you use. No monthly fees, no minimums.

Event	Price	Triggered When
actor_start	$0.005	Actor run begins
monitor_run	$0.02	Monitor execution completes
new_item	$0.005	Each new unique signal delivered
enrichment_llm	$0.003	Each tweet enriched via LLM
alert_delivered	$0.001	Each successful webhook delivery

Cost Examples

Scenario	Cost
Monitor 1 query, 50 new tweets, no LLM	$0.005 + $0.02 + (50 × $0.005) = $0.275
Search 100 tweets with LLM (one-off)	$0.005 + (100 × $0.003) = $0.305
Daily monitor, avg 20 new/day, webhook	$0.005 + $0.02 + (20 × $0.005) + (20 × $0.001) = $0.145/day

---

Enrichment Details

Sentiment Analysis

Label	Score Range	Example
positive	+0.33 to +1.0	"This tool is incredible!"
neutral	-0.33 to +0.33	"Released version 2.0 today"
negative	-1.0 to -0.33	"Terrible experience, avoid"

Intent Classification

Intent	Description	Example Signal
buyer	Purchase/evaluation intent	"looking for a tool that..."
support	Help request	"how do I fix...", "not working"
complaint	Negative product feedback	"terrible customer service at..."
lead	Hiring/partnership seeking	"we're hiring a...", "seeking partners"
news	Announcement/reporting	"just announced...", "breaking:"
neutral	None of the above	General commentary

Entity Extraction

Automatically identifies: person, org, product, hashtag, cashtag, url, mention, other

---

Webhook Integration

Payload Format

{
  "monitorId": "brand.mentions",
  "runId": "abc-123",
  "signals": [/* DatasetSignalRow[] */],
  "count": 5,
  "emittedAt": "2026-07-25T10:31:05.000Z"
}

Security Headers

Header	Description
X-XSignal-Signature	sha256=<hex> HMAC of raw body (if webhookSecret set)
X-XSignal-Delivery	Unique delivery attempt ID
X-XSignal-Idempotency-Key	<monitorId>:<tweetId> for receiver dedup

Verification (Node.js example)

import { createHmac } from 'crypto';

function verifyWebhook(body: string, signature: string, secret: string): boolean {
  const expected = 'sha256=' + createHmac('sha256', secret).update(body).digest('hex');
  return signature === expected;
}

---

Architecture

Clean Architecture with strict layer separation:

src/
├── domain/           Pure business logic (entities, policies, port interfaces)
├── application/      Use cases (RunMonitor, SearchAndEnrich, AnalyzeBatch, GetThread)
├── infrastructure/   External adapters (Twitter APIs, LLM, KV Store, Dataset)
├── interfaces/       Entrypoints (Actor main, MCP server)
├── composition/      DI container (wires everything together)
├── config/           Schema validation, pricing, defaults
└── shared/           Result type, retry, bloom filter, utilities

Design Principles:

• Dependency Inversion — Domain depends on nothing; infrastructure implements domain ports
• Result Type — No thrown exceptions across boundaries; all failures are typed values
• Port/Adapter — Every external service accessed through an interface with a test fake
• Determinism — No Date.now() or Math.random(); injected Clock and seeds

---

Development

Prerequisites

• Node.js 20 LTS
• npm (no yarn/pnpm)

Commands

npm install                              # Install dependencies
npx tsc --noEmit                         # Type-check
npx eslint src/ tests/                   # Lint
npx prettier --check .                   # Format check
npx vitest run                           # Run all tests (163 tests)
npx vitest                               # Watch mode
npx vitest run --coverage                # With coverage report
npx dependency-cruiser src/ --config     # Verify import boundaries
node scripts/schema-check.mjs            # Verify pricing parity

E2E Testing (requires Apify deployment)

$RUN_E2E=1 APIFY_TOKEN=<token> npx vitest run tests/e2e/

---

FAQ

Q: How does deduplication work? A: Three-layer approach: (1) Cursor-based — only fetch tweets newer than last run's newest tweet. (2) Recent-ID window — exact match against last 5,000 emitted IDs. (3) Bloom filter — probabilistic check for older IDs (100K capacity, 1% FPP). This guarantees at-least-once delivery with near-zero duplicates.

Q: What happens if both providers fail? A: The run returns a structured E_ALL_PROVIDERS_FAILED error. No billing for new_item or monitor_run occurs. The cursor is NOT advanced, so the next run will retry the same time window.

Q: Is LLM enrichment cached? A: Yes. Results are cached by sha256(tweetId + modelVersion + lexiconVersion) in Apify KV Store with a 30-day TTL. Repeat analyses of the same tweet are instant and free.

Q: What if the LLM is unavailable or returns invalid JSON? A: Automatic fallback to fast-path (lexicon) enrichment. The tweet is still delivered with enrichmentSource: "llm-fallback-fastpath". No data loss.

Q: Does this use Twitter's official API? A: No. X-Signal uses third-party scraping actors on Apify. See the Legal Disclaimer below.

Q: Can I use this without MCP? A: Absolutely. The Actor works standalone via Apify Console, CLI, API, or scheduled runs. MCP is an additional interface for AI agents.

Q: What about rate limiting? A: Self-imposed. The upstream scrapers handle their own rate management. X-Signal adds retry with exponential backoff + jitter to prevent thundering herd issues.

Q: How do I monitor multiple queries? A: Create separate monitors with unique monitorId values. Each maintains independent state:

# Monitor 1
{ "monitorId": "brand.twitter", "query": "@yourbrand" }
# Monitor 2
{ "monitorId": "leads.saas", "query": "looking for a SaaS tool" }

---

Apify Integration

Scheduling

Set up recurring monitoring via Apify Schedules:

1. Go to your Actor's page → Schedules tab
2. Set cron expression (e.g., 0 */1 * * * for hourly)
3. Configure input JSON
4. Enable — X-Signal will automatically deliver only new tweets each run

Datasets

Access results programmatically:

# Get all results
curl "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_TOKEN&format=json"

# Get only positive sentiment
curl "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_TOKEN&fields=tweet,sentiment&filter=sentiment.label:positive"

Integration with Other Actors

Chain X-Signal with other Apify Actors:

X-Signal → [webhook] → Slack Notification Actor
X-Signal → [dataset] → Google Sheets Actor
X-Signal → [webhook] → Your AI Agent Pipeline

---

Legal Disclaimer

This Actor accesses publicly available data from Twitter/X through third-party scraping services. It does NOT use Twitter's official API and is NOT affiliated with, endorsed by, or associated with X Corp.

By using this Actor, you acknowledge and agree that:

1. You are solely responsible for compliance with Twitter/X Terms of Service, applicable laws, and regulations in your jurisdiction.
2. This Actor only processes public tweets. Protected/private accounts are automatically detected and skipped.
3. This Actor performs read-only operations. It does not post, like, retweet, follow, or modify any content.
4. Rate limiting is self-imposed to minimize platform impact.
5. Data collected should be used in accordance with applicable data protection regulations (GDPR, CCPA, etc.).
6. The authors and publishers accept no liability for misuse or Terms of Service violations by users.

Use responsibly and ethically.

---

Support & Resources

• Issues & Bugs — Open an issue on the Actor's page
• Input Schema — .actor/INPUT_SCHEMA.json
• Dataset Schema — .actor/DATASET_SCHEMA.json
• Output Schema — .actor/OUTPUT_SCHEMA.json
• Cost Analysis — docs/cost-note.md
• Changelog — CHANGELOG.md

---

Built with Clean Architecture on Apify