Social Media Intelligence Scraper 🚀 A powerful, self-contained social media scraper that extracts deep intelligence from Instagram, TikTok, LinkedIn, and Facebook profiles without relying on external actors or APIs.

✨ Features 🎯 Comprehensive Data Extraction Account Overview: Username, bio, verification status, followers, following Audience Metrics: Follower counts, engagement ratios, growth indicators Content Analysis: Post types, performance metrics, best/worst performers Hashtag Intelligence: Top hashtags, usage frequency, diversity scores Posting Patterns: Best days/hours, frequency, consistency analysis Engagement Analytics: Likes, comments, shares, views, engagement rates Competitive Intelligence: Strengths, weaknesses, content themes, recommendations 🔥 Self-Contained Architecture ✅ No external actor dependencies ✅ Direct web scraping using Playwright ✅ Works independently without third-party APIs ✅ Fully customizable and maintainable 📊 Supported Platforms Instagram - Full profile + post analysis TikTok - Videos, sounds, engagement metrics LinkedIn - Professional profile data (limited) Facebook - Page analytics (limited) 🚀 Quick Start Input Configuration Provide at least one social media profile URL:

json { "instagramUrl": "https://www.instagram.com/cristiano/", "tiktokUrl": "https://www.tiktok.com/@khaby.lame", "linkedinUrl": "https://www.linkedin.com/in/williamhgates/", "facebookUrl": "https://www.facebook.com/nike" } Running the Actor On Apify Platform: Create new actor Upload the code Configure input Run! Locally: bash

Install dependencies

npm install

Set environment variable

export APIFY_TOKEN=your_token_here

Run locally

apify run 📥 Input Schema Field Type Required Description Example instagramUrl string No* Instagram profile URL https://www.instagram.com/username/ tiktokUrl string No* TikTok profile URL https://www.tiktok.com/@username linkedinUrl string No* LinkedIn profile URL https://www.linkedin.com/in/username/ facebookUrl string No* Facebook page URL https://www.facebook.com/pagename *At least one URL is required

📤 Output Schema Instagram Output json { "platform": "Instagram", "profileUrl": "https://www.instagram.com/cristiano/", "scrapedAt": "2025-10-27T12:00:00.000Z", "accountOverview": { "username": "cristiano", "displayName": "Cristiano Ronaldo", "followers": 500000000, "following": 500, "totalPosts": 3500 }, "engagementAnalysis": { "averageLikesPerPost": 5000000, "averageCommentsPerPost": 50000, "engagementRate": "3.5%", "totalEngagement": 17500000000 }, "hashtagStrategy": { "totalHashtagsUsed": 150, "topPerformingHashtags": [ {"hashtag": "football", "timesUsed": 45}, {"hashtag": "cr7", "timesUsed": 38} ] }, "postingPatterns": { "mostActiveDay": "Monday", "bestPostingHour": "18:00", "postingFrequency": "5.2 posts/week - Regular poster" }, "topPerformers": { "top3Posts": [...] }, "allPosts": [...], "competitiveIntel": { "keyStrengths": [ "Outstanding engagement - Highly engaged audience", "Excellent hashtag strategy" ], "recommendations": [ "Keep up the great work!" ] } } TikTok Output json { "platform": "TikTok", "profileUrl": "https://www.tiktok.com/@khaby.lame", "accountOverview": { "username": "khaby.lame", "followers": 150000000, "totalVideos": 500 }, "performanceMetrics": { "averageViews": 50000000, "averageLikes": 5000000, "engagementRate": "8.5%", "viralityScore": "3.2" }, "soundStrategy": { "topSounds": [ {"sound": "Original Sound", "timesUsed": 300} ] }, "allPosts": [...], "competitiveIntel": {...} } 🎯 Key Metrics Explained Engagement Rate (Total Engagement / Total Followers) × 100 Excellent: > 5% Good: 2-5% Average: 1-2% Poor: < 1% Performance Score Calculated based on:

Engagement rate (60% weight) Reach (40% weight) Virality Score (TikTok) Average Views / Followers High: > 2.0 (viral content) Medium: 1.0-2.0 (good reach) Growing: < 1.0 (building audience) Consistency Score High: 20+ posts analyzed Medium: 10-20 posts Low: < 10 posts 🔧 Technical Details Architecture Language: Node.js (ES Modules) Browser Automation: Playwright Framework: Apify SDK + Crawlee Scraping Method: Direct DOM parsing + API extraction How It Works URL Validation: Validates and cleans input URLs Browser Launch: Opens headless Chrome via Playwright Page Navigation: Visits profile pages with realistic user agents Data Extraction: Parses JSON-LD structured data Extracts GraphQL/SIGI_STATE data Scrapes visible DOM elements (fallback) Analysis: Processes raw data through intelligence algorithms Output: Returns comprehensive analytics JSON Rate Limiting 5-second delay between multiple profile scrapes Realistic user agent rotation Headless browser with stealth plugins ⚠️ Limitations Instagram ✅ Public profiles fully supported ❌ Private profiles: Basic info only ✅ Up to 50 recent posts analyzed ❌ Stories/Highlights not included TikTok ✅ Public profiles fully supported ❌ Private profiles: Basic info only ✅ Up to 50 recent videos analyzed ⚠️ Rate limiting possible on high volume LinkedIn ⚠️ Limited without authentication ✅ Basic profile info (name, headline, location) ❌ Posts require login 💡 Recommendation: Use LinkedIn data export Facebook ⚠️ Limited without authentication ✅ Page name, likes, followers ❌ Posts require login 💡 Recommendation: Use Facebook Business tools 🛡️ Best Practices

1. Respect Rate Limits Don't scrape too many profiles at once Use delays between requests Monitor for IP blocks
2. Handle Errors Gracefully Check error field in output Implement retry logic Log failures for debugging
3. Data Privacy Only scrape public profiles Don't store personal data unnecessarily Comply with platform ToS
4. Optimize Performance Scrape during off-peak hours Use residential proxies if needed Cache results when possible 📊 Use Cases
5. Influencer Marketing Identify top influencers by engagement Analyze content strategy Compare competitor performance
6. Brand Monitoring Track brand mentions Analyze hashtag performance Monitor competitor activity
7. Content Strategy Identify best posting times Analyze top-performing content types Discover trending hashtags
8. Market Research Understand audience demographics Track industry trends Analyze competitor strategies
9. Competitive Intelligence Compare engagement rates Identify content gaps Benchmark performance 🐛 Troubleshooting "Actor failed to start" Check Dockerfile configuration Verify package.json dependencies Ensure Playwright is installed "No data returned" Profile may be private Check URL format Platform may have changed structure "Rate limited" Reduce scraping frequency Add longer delays Use proxy rotation "Timeout errors" Increase timeout in page.goto() Check internet connection Profile may be slow to load 📝 Example Queries Single Profile json { "instagramUrl": "https://www.instagram.com/nike/" } Multiple Platforms json { "instagramUrl": "https://www.instagram.com/redbull/", "tiktokUrl": "https://www.tiktok.com/@redbull", "facebookUrl": "https://www.facebook.com/redbull" } Competitor Analysis json { "instagramUrl": "https://www.instagram.com/nike/", "tiktokUrl": "https://www.tiktok.com/@nike" } 🔄 Future Enhancements Add Twitter/X support Include YouTube analytics Add Pinterest scraping Implement proxy rotation Add sentiment analysis Create visualization dashboard Add historical data tracking Implement ML-based predictions 📜 License Apache-2.0

🤝 Contributing Contributions welcome! Please:

Fork the repository Create a feature branch Submit a pull request 📧 Support For issues or questions:

Open a GitHub issue Contact: your-email@example.com Documentation: [Link to docs] ⭐ Acknowledgments Built with:

Apify SDK Crawlee Playwright Made with ❤️ for data-driven marketers and analysts