Page Pulse - Website Change Monitor & RSS Feed Generator

Automatically monitor any website for content changes and generate RSS feeds. Perfect for tracking competitor prices, news updates, documentation changes, or any webpage content without manual checking.

🔍 Features

• Multi-URL Monitoring - Track changes on multiple websites simultaneously
• CSS Selector Support - Target specific page elements (prices, articles, tables)
• RSS 2.0 Feed Generation - Subscribe to changes in any feed reader
• RSS/Atom Feed Support - Monitor existing feeds for updates
• Webhook Notifications - Get instant alerts when changes occur
• Persistent State - Only reports actual changes between runs
• Content Diff - See what changed with optional diff output

📋 Use Cases

Use Case	Example
E-commerce	Track competitor pricing changes
News Monitoring	Follow updates on sites without RSS feeds
Research	Monitor government or academic publications
Documentation	Track API docs or changelog updates
Job Hunting	Watch career pages for new listings
Social Media	Monitor profile or page changes

⚙️ Input Configuration

Parameter	Type	Required	Default	Description
urls	Array	✅ Yes	-	List of webpage URLs to monitor
selector	String	No	""	CSS selector to extract specific content
generateRss	Boolean	No	true	Generate an RSS feed with changes
feedTitle	String	No	"Website Changes Feed"	Title for the RSS feed
feedDescription	String	No	Auto-generated	Description for the RSS feed
feedLink	String	No	""	Link associated with the RSS feed
webhookUrl	String	No	""	URL to receive POST notifications
notificationEmail	String	No	""	Email for change alerts
maxChangesToKeep	Integer	No	50	Max number of changes in history
includeContentDiff	Boolean	No	false	Include text diff in change records

📝 Example Input

Monitor HTML Pages

{
    "urls": [
        "https://news.ycombinator.com",
        "https://example.com/pricing"
    ],
    "selector": ".price-tag",
    "generateRss": true,
    "feedTitle": "Price Monitor Feed",
    "webhookUrl": "https://hooks.example.com/my-webhook",
    "maxChangesToKeep": 100,
    "includeContentDiff": true
}

Monitor RSS Feeds

{
    "urls": [
        "https://news.un.org/feed/subscribe/en/news/all/rss.xml",
        "https://blog.example.com/feed"
    ],
    "generateRss": true,
    "feedTitle": "Combined News Feed"
}

📤 Output

Dataset Output

Each run produces a dataset with information about processed URLs:

{
    "url": "https://example.com",
    "pageTitle": "Example Page",
    "contentHash": "abc123def456...",
    "contentLength": 5432,
    "hasChanged": true,
    "changeType": "content_modified",
    "checkedAt": "2024-01-15T10:30:00Z"
}

Key-Value Store

Key	Content
OUTPUT	Run summary with statistics
RSS_FEED	Generated RSS 2.0 feed (XML)
MONITOR_STATE	Stored content hashes for comparison
DETECTED_CHANGES	Array of all detected changes

RSS Feed URL

After running, your RSS feed is available at:

https://api.apify.com/v2/key-value-stores/{STORE_ID}/records/RSS_FEED

Add this URL to Feedly, Inoreader, or any RSS reader!

🎯 CSS Selector Examples

Use Case	Selector Example
Monitor article content	article, .post-content
Track prices	.price, .product-price, [data-price]
Watch for new items	.item-list li, .product-card
Monitor tables	table tbody
Track specific element	#main-content
Multiple elements	.news-item, .blog-post

📅 Scheduling

For continuous monitoring, schedule the actor:

1. Go to your actor in Apify Console
2. Click Schedules tab
3. Create a new schedule:
   • Hourly - For fast-changing content
   • Daily - For news or blog updates
   • Weekly - For documentation or static pages

The actor maintains state between runs and only reports new changes.

🔔 Webhook Notifications

When changes are detected, your webhook receives:

{
    "event": "website_changes_detected",
    "timestamp": "2024-01-15T10:30:00Z",
    "changesCount": 2,
    "changes": [
        {
            "url": "https://example.com",
            "pageTitle": "Example Page",
            "changeType": "content_modified",
            "detectedAt": "2024-01-15T10:30:00Z",
            "diff": "Added words: new, feature, available..."
        }
    ]
}

Integrations

Use webhooks to connect with:

• Slack - Post to a channel
• Discord - Send notifications
• Zapier - Trigger workflows
• Make (Integromat) - Automate actions
• Custom APIs - Your own systems

🛠️ How It Works

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│  Load URLs  │ ──▶ │ Fetch Pages  │ ──▶ │ Extract     │
│  from Input │     │ (HTML/RSS)   │     │ Content     │
└─────────────┘     └──────────────┘     └─────────────┘
                                                │
                                                ▼
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│  Generate   │ ◀── │   Compare    │ ◀── │  Generate   │
│  RSS Feed   │     │   Hashes     │     │   Hash      │
└─────────────┘     └──────────────┘     └─────────────┘
       │                   │
       ▼                   ▼
┌─────────────┐     ┌──────────────┐
│  Save to    │     │    Send      │
│  Storage    │     │  Webhooks    │
└─────────────┘     └──────────────┘

1. Fetch - Downloads each URL (supports HTML and RSS/Atom feeds)
2. Extract - Pulls content using CSS selectors or full page body
3. Hash - Creates MD5 hash of content
4. Compare - Checks against previous run's hash
5. Notify - Sends webhooks and generates RSS for detected changes

💡 Tips

• Start simple - Monitor full pages first, then refine with selectors
• Use specific selectors - The more targeted, the fewer false positives
• Schedule wisely - Match frequency to how often content changes
• Test webhooks - Use webhook.site for testing
• Combine sources - Mix HTML pages and RSS feeds in the same run

🔧 Technology Stack

• Apify SDK v3 - Platform integration
• Crawlee - Web scraping framework
• CheerioCrawler - Fast HTML/XML parsing
• RSS - Feed generation library

📄 License

ISC

🤝 Support

• Issues - Open an issue on GitHub
• Questions - Contact via Apify platform
• Feature Requests - Submit via GitHub issues