Webhook Event Store & Replay System

The ultimate debugging tool for webhook integrations. Capture, inspect, and replay webhook events from any service.

🎯 What It Does

Webhook Event Store provides a complete lifecycle management system for webhooks. Capture incoming webhooks from services like Stripe, GitHub, Shopify, or custom APIs, store them with full metadata, inspect payloads, validate signatures, and replay events to any endpoint for testing and debugging.

Perfect for:

• Backend Developers: Debug webhook integrations without waiting for real events
• QA Teams: Test webhook handling with real production payloads
• DevOps Engineers: Audit webhook activity and troubleshoot delivery issues
• Integration Engineers: Validate webhook signatures and inspect headers

✨ Key Features

🎣 Capture Mode - Receive & Store Webhooks

• HTTP Server: Spin up endpoint to receive webhooks from any service
• Automatic Storage: All webhooks saved to Apify dataset with full metadata
• Signature Validation: Built-in support for GitHub, Stripe, and generic HMAC validation
• Event Type Detection: Automatically identifies event types from headers/payload
• Smart Auto-Stop: Stop capturing after inactivity or max duration
• Health Endpoint: Monitor capture status in real-time

🔄 Replay Mode - Resend Events

• Targeted Replay: Send captured events to any endpoint for testing
• Flexible Filtering: Replay all events, specific IDs, or filter by event type
• Replay Headers: Mark replayed events with special headers
• Rate Limiting: Configure delays between replays to respect limits
• Response Tracking: Record response status and headers for each replay
• Bulk Operations: Replay multiple events in sequence

🔍 Query Mode - Search & Inspect

• Advanced Filtering: Search by event type, source IP, date range
• Payload Inspection: View complete headers, query params, and body
• Chronological View: Events sorted by timestamp
• Result Limiting: Control output size for large datasets
• Age Indicators: Human-readable timestamps ("2 hours ago")

🚀 Use Cases

Use Case 1: Debugging Stripe Webhooks

Problem: Your Stripe integration isn't working, but you can't see the webhook payload

{
  "mode": "capture",
  "captureConfig": {
    "port": 3000,
    "captureEndpoint": "/stripe-webhook",
    "validateSignatures": true,
    "signatureHeader": "Stripe-Signature",
    "signatureSecret": "whsec_your_stripe_secret",
    "maxCaptureMinutes": 30
  }
}

Result: Point Stripe at your Actor's public URL, trigger test events, inspect full payload

Use Case 2: Testing GitHub Webhook Handlers

Problem: Need to test your GitHub webhook handler without creating real PRs/issues

{
  "mode": "capture",
  "captureConfig": {
    "port": 3000,
    "captureEndpoint": "/github",
    "validateSignatures": true,
    "signatureHeader": "X-Hub-Signature-256",
    "signatureSecret": "your_github_webhook_secret",
    "maxCaptureMinutes": 60
  }
}

After capturing real events, replay them to your dev environment:

{
  "mode": "replay",
  "replayConfig": {
    "targetUrl": "https://dev.myapp.com/webhooks/github",
    "replayAll": false,
    "filterByEventType": "github.pull_request",
    "addReplayHeaders": true,
    "delayBetweenRequests": 500
  },
  "storageDatasetId": "your_capture_dataset_id"
}

Use Case 3: Audit Webhook Activity

Problem: Need to investigate webhook delivery issues from last week

{
  "mode": "query",
  "queryConfig": {
    "dateFrom": "2025-11-17T00:00:00Z",
    "dateTo": "2025-11-24T23:59:59Z",
    "filterByEventType": "shopify.orders/create",
    "limit": 100,
    "includePayload": true
  },
  "storageDatasetId": "your_capture_dataset_id"
}

📥 Input Configuration

Mode Selection (Required)

• mode: "capture" | "replay" | "query"

Capture Mode Configuration

{
  "captureConfig": {
    "port": 3000,                    // Server port
    "captureEndpoint": "/webhook",    // URL path for webhooks
    "validateSignatures": false,      // Enable signature validation
    "signatureHeader": "X-Signature", // Header containing signature
    "signatureSecret": "",            // Secret for validation
    "maxCaptureMinutes": 60,         // Max capture duration
    "autoStop": true                  // Stop after 5min inactivity
  }
}

Signature Validation Support:

• GitHub: X-Hub-Signature-256 (HMAC SHA256)
• Stripe: Stripe-Signature (HMAC SHA256 with timestamp)
• Generic: Any HMAC SHA256 signature header

Replay Mode Configuration

{
  "replayConfig": {
    "targetUrl": "https://example.com/webhook",  // Required
    "replayAll": true,                           // Replay all events
    "eventIds": [],                              // Specific event IDs (if not replayAll)
    "filterByEventType": "",                     // Filter by event type
    "addReplayHeaders": true,                    // Add X-Webhook-Replay header
    "delayBetweenRequests": 100                  // Delay in milliseconds
  },
  "storageDatasetId": "abc123"                   // Dataset from capture mode
}

Query Mode Configuration

{
  "queryConfig": {
    "filterByEventType": "",      // Optional: Filter by event type
    "filterBySource": "",          // Optional: Filter by source IP
    "dateFrom": "",                // Optional: ISO 8601 start date
    "dateTo": "",                  // Optional: ISO 8601 end date
    "limit": 100,                  // Max results
    "includePayload": true         // Include full payload
  },
  "storageDatasetId": "abc123"
}

📤 Output Data

Capture Mode Output

Each captured webhook is stored with complete metadata:

{
  "id": "webhook_1700000000000_abc123",
  "timestamp": "2025-11-24T15:30:00.000Z",
  "method": "POST",
  "path": "/webhook",
  "headers": {
    "content-type": "application/json",
    "x-github-event": "pull_request",
    "x-hub-signature-256": "sha256=..."
  },
  "query": {},
  "body": {
    "action": "opened",
    "pull_request": { ... }
  },
  "sourceIp": "140.82.115.0",
  "eventType": "github.pull_request",
  "signatureValid": true
}

Replay Mode Output

Each replay generates a result record:

{
  "originalEventId": "webhook_1700000000000_abc123",
  "replayedAt": "2025-11-24T16:00:00.000Z",
  "responseStatus": 200,
  "responseHeaders": {
    "content-type": "application/json"
  },
  "success": true
}

Query Mode Output

Filtered events with age indicators:

{
  "id": "webhook_1700000000000_abc123",
  "timestamp": "2025-11-24T15:30:00.000Z",
  "eventType": "github.pull_request",
  "method": "POST",
  "path": "/webhook",
  "sourceIp": "140.82.115.0",
  "signatureValid": true,
  "age": "30 minutes ago",
  "headers": { ... },    // If includePayload: true
  "query": { ... },      // If includePayload: true
  "body": { ... }        // If includePayload: true
}

🔐 Security Best Practices

Signature Validation

Always enable signature validation for production webhooks:

{
  "validateSignatures": true,
  "signatureHeader": "X-Hub-Signature-256",
  "signatureSecret": "your_webhook_secret"
}

Network Access

• Use Apify's standby mode to keep capture running
• Configure firewall rules to limit webhook sources
• Use HTTPS for replay targets to protect sensitive data

Data Privacy

• Be mindful of PII in webhook payloads
• Set appropriate dataset retention periods
• Don't share dataset IDs with untrusted parties

💡 Advanced Workflows

Workflow 1: Continuous Webhook Monitoring

1. Run Actor in capture mode on standby
2. Configure webhook provider to send to Actor URL
3. Schedule periodic queries to analyze patterns
4. Set up alerts for failed signature validations

Workflow 2: Integration Testing Pipeline

1. Capture production webhooks once
2. Store dataset ID in your test config
3. Replay to staging environment before deployments
4. Validate handler behavior against real payloads

Workflow 3: Debugging Production Issues

1. Enable capture mode during incident
2. Collect 10-20 minutes of webhook traffic
3. Query for suspicious patterns or errors
4. Replay problematic events to local dev environment
5. Fix issue and verify with replay

🛠 Technical Details

Capture Mode Architecture

• HTTP Server: Express.js with body-parser
• Raw Body Preservation: For accurate signature validation
• Event Type Detection: Heuristic-based header/payload analysis
• Auto-stop Logic: Configurable inactivity timeout

Signature Validation

Supports multiple signature schemes:

• GitHub: SHA256 HMAC with sha256= prefix
• Stripe: SHA256 HMAC with timestamp (t=,v1=)
• Generic: Plain SHA256 HMAC

Uses crypto.timingSafeEqual() to prevent timing attacks.

Replay Mechanism

• Header Sanitization: Removes hop-by-hop headers (host, connection)
• Replay Markers: Adds custom headers to identify replays
• Rate Limiting: Configurable delays between requests
• Error Handling: Continues on failures, logs all results

Query Performance

• In-Memory Filtering: Fast for datasets up to 10k events
• Chronological Sorting: Most recent events first
• Efficient Output: Optional payload exclusion for large datasets

📊 Typical Performance

Operation	Duration	Notes
Capture (1 event)	<50ms	Per webhook received
Replay (1 event)	100-500ms	Depends on target latency
Query (1000 events)	1-3s	With filtering and sorting
Signature validation	<10ms	Per webhook

🔄 Integration Examples

Stripe Setup

# In Stripe Dashboard:
# Webhook URL: https://api.apify.com/v2/acts/YOUR_ACTOR/runs/last/webhook
# Events: Select all or specific events
# Secret: Copy to signatureSecret input

GitHub Setup

# In GitHub Repo Settings → Webhooks:
# Payload URL: https://api.apify.com/v2/acts/YOUR_ACTOR/runs/last/webhook
# Content type: application/json
# Secret: Generate and copy to signatureSecret input
# Events: Select triggers (Push, PR, Issues, etc.)

Custom Webhook Testing

# Send test webhook with curl
curl -X POST https://api.apify.com/v2/acts/YOUR_ACTOR/runs/last/webhook \
  -H "Content-Type: application/json" \
  -H "X-Custom-Signature: your_signature" \
  -d '{"event": "test", "data": {"key": "value"}}'

🚨 Troubleshooting

Webhooks Not Captured

• Check Actor is running in standby mode
• Verify webhook URL matches captureEndpoint
• Check webhook provider for delivery failures
• Review Actor logs for errors

Signature Validation Failing

• Verify signatureSecret matches provider
• Check signatureHeader name (case-insensitive)
• Ensure webhook provider is using supported algorithm
• Review Actor logs for validation details

Replay Not Working

• Verify targetUrl is accessible
• Check target endpoint accepts replayed content-type
• Review replay output for response status codes
• Ensure delayBetweenRequests doesn't cause timeouts

📈 Best Practices

For Development

• Start with capture mode to collect real events
• Use query mode to inspect payload structure
• Replay to localhost with ngrok/tunnels for debugging
• Keep signatureSecret in environment variables

For Production

• Always enable signature validation
• Set reasonable maxCaptureMinutes
• Use autoStop to conserve resources
• Monitor dataset size and clean up periodically

For Testing

• Capture production events once, replay many times
• Filter by event type to test specific handlers
• Use addReplayHeaders to identify test traffic
• Adjust delayBetweenRequests for rate limits

🤝 Contributing

Found a bug? Need a feature? Please open an issue!

Suggested improvements:

• Additional signature validation schemes (HMAC SHA1, RSA)
• Webhook transformation/filtering before storage
• Built-in replay scheduling
• Real-time webhook forwarding mode

📄 License

MIT License - use freely in your projects!

🏆 Apify $1M Challenge

Built for the Apify $1M Challenge to solve real webhook debugging pain points. Help us improve:

• Share feedback on signature validation
• Suggest new webhook providers to support
• Report edge cases or bugs
• Star and share if you find it useful!

---

Debug webhooks like a pro 🎯