LinkedIn Profile Posts Scraper Actor

🚀 Apify Actor to scrape ALL posts from a LinkedIn profile via n8n webhook

This actor extracts comprehensive data from all posts published by a LinkedIn profile by calling your n8n webhook endpoint. It's designed to work seamlessly with your existing n8n workflow for LinkedIn profile posts extraction.

🎯 Features

• ✅ Scrapes ALL posts from a LinkedIn profile (not just one)
• ✅ Configurable maximum posts limit (default: 50, max: 200)
• ✅ Extracts complete post data (title, content, author, engagement)
• ✅ Retrieves author information (name, profile URL, followers)
• ✅ Captures engagement metrics (reactions, comments)
• ✅ Extracts hashtags from post content
• ✅ Gets media/images from posts
• ✅ Retrieves top comments for each post
• ✅ Captures published date
• ✅ Multiple retry attempts with different user agents
• ✅ Comprehensive error handling and logging
• ✅ Dataset views for easy data visualization
• ✅ Aggregate statistics (total reactions, comments)

📥 Input

{
  "profileUrl": "https://www.linkedin.com/in/cloudwithraj/",
  "maxPosts": 50
}

Input Parameters

Field	Type	Required	Description
profileUrl	String	Yes	Full URL of the LinkedIn profile to scrape posts from
maxPosts	Integer	No	Maximum number of posts to scrape (default: 50, max: 200)

📤 Output

The actor returns an array of posts scraped from the profile:

[
  {
    "postIndex": 1,
    "activityId": "7388952726245629952",
    "url": "https://www.linkedin.com/posts/cloudwithraj_...",
    "title": "Post title",
    "author": {
      "name": "Author Name",
      "url": "https://www.linkedin.com/in/authorname",
      "followers": "10K followers"
    },
    "content": {
      "text": "Post content text...",
      "hashtags": ["#amazon", "#tech", "#news"]
    },
    "engagement": {
      "reactions": 1234,
      "comments": 56
    },
    "media": {
      "images": ["https://media.licdn.com/..."]
    },
    "publishedDate": "2024-12-01T10:30:00.000Z",
    "topComments": [
      {
        "author": "Commenter Name",
        "text": "Comment text...",
        "time": "2h ago"
      }
    ],
    "scrapedAt": "2024-12-01T15:45:00.000Z",
    "source": "linkedin-profile-posts-scraper",
    "profileUrl": "https://www.linkedin.com/in/cloudwithraj/",
    "username": "cloudwithraj"
  },
  {
    "postIndex": 2,
    ...
  }
]

🎨 Dataset Views

The actor provides 3 pre-configured dataset views:

1. Overview

Basic information for quick analysis:

• Post Index (#)
• Activity ID
• Title
• Author
• Post text
• Reactions & Comments
• Published date
• Post URL

2. Detailed View

Complete post information including:

• All overview fields
• Author profile URL and followers
• Hashtags
• Media/images
• Top comments
• Scraping metadata
• Profile URL and username

3. Engagement Metrics

Focus on engagement data:

• Post Index
• Activity ID
• Author
• Reactions & Comments count
• Hashtags
• Published date

🚀 Usage

On Apify Platform

1. Go to Apify Console
2. Create a new Actor
3. Upload all files from this directory
4. Build and run the actor
5. Provide a LinkedIn post URL in the input

Local Development

# Install Apify CLI
npm install -g apify-cli

# Initialize project (if not already)
apify init

# Run locally
apify run -p

# Push to Apify platform
apify push

Via API

curl -X POST https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "profileUrl": "https://www.linkedin.com/in/username/",
    "maxPosts": 50
  }'

🔧 Technical Details

Architecture

• Webhook Integration: Calls n8n webhook at https://eliasse-n8n.onrender.com/webhook/1305b7f8-0602-48f6-85df-0f7f7497e54b
• HTTP Client: Uses axios with retry logic
• Error Handling: Multiple retry attempts with different user agents
• Timeout: 180 seconds per request (3 minutes - profile scraping takes longer)
• Data Validation: Only includes non-empty fields in output
• Bulk Processing: Handles multiple posts from a single profile

Webhook Payload

{
  "profileUrl": "https://www.linkedin.com/in/username/",
  "username": "username",
  "maxPosts": 50,
  "timestamp": "2024-12-01T15:45:00.000Z",
  "source": "linkedin-profile-posts-scraper",
  "version": "1.0.0",
  "requestType": "profile_posts_scraping"
}

Retry Logic

The actor implements smart retry logic:

1. Tests multiple payload formats
2. Tries different user agents
3. Maximum 3 attempts per configuration
4. 2-second delay between retries

⚠️ Limitations

• Requires valid LinkedIn profile URL (linkedin.com/in/username)
• Profile posts must be publicly accessible (or semi-public)
• Depends on n8n webhook availability
• Maximum 200 posts per profile (configurable)
• Some data may not be available for all posts
• Rate limiting may apply
• Scraping time increases with number of posts

📊 Data Quality

The actor ensures data quality by:

• Validating URL format before processing
• Filtering out empty/null/"Not specified" values
• Providing structured nested data (author, content, engagement, media)
• Including metadata for tracking (scrapedAt, source, postIndex)
• Aggregate statistics (total reactions, comments across all posts)
• Post indexing for easy reference

🔐 Privacy & Ethics

• Only scrapes publicly accessible posts
• Respects LinkedIn's robot.txt and ToS
• No authentication required
• Data is stored securely in Apify datasets

🐛 Error Handling

The actor handles various error scenarios:

• Invalid URL format → Returns error with helpful message
• Webhook timeout → Retries with exponential backoff
• Network errors → Logs detailed error info
• Empty response → Returns error item to dataset

📝 Logs

The actor provides detailed logging:

👤 LinkedIn Profile Posts Scraper Actor Starting...
📝 Input received: {...}
🔍 Scraping posts for profile: cloudwithraj (max: 50 posts)
🔄 Attempt 1: Testing payload format
✅ Success with payload format and user agent: Mozilla/5.0...
📊 Received data for processing: 12456 characters
📰 Processing 25 posts from profile
✅ Processed 25 posts from profile
📋 Sample post: {...}
✅ Saved 25 posts to dataset
📊 Final Summary: {
  "profileUrl": "https://www.linkedin.com/in/cloudwithraj/",
  "username": "cloudwithraj",
  "totalPostsScraped": 25,
  "postsWithContent": 25,
  "postsWithEngagement": 23,
  "totalReactions": 15234,
  "totalComments": 456,
  "errors": 0,
  "maxPosts": 50
}
🏁 LinkedIn Profile Posts Scraper Actor Finished

📈 Version History

v1.0.0 (2024-12-01)

• Initial release
• Profile posts scraping functionality (all posts from a profile)
• Configurable max posts limit (1-200)
• Webhook integration with n8n
• Multiple dataset views
• Comprehensive error handling
• Aggregate statistics

🤝 Support

For issues or questions:

1. Check the actor logs in Apify Console
2. Verify the profile URL is valid and accessible (linkedin.com/in/username)
3. Ensure n8n webhook is online and responding
4. Check if profile has public posts available

---

Made with ❤️ for LinkedIn profile data extraction