Lemon8 Profile Scraper

A powerful and comprehensive scraper for extracting data from Lemon8 user profiles, including posts, images, videos, engagement metrics, comments, and more. The scraper supports automatic multi-profile scraping capabilities, allowing you to extract data from an initial profile and all the profiles they follow.

Features

• Profile Scraping: Extract comprehensive user profile information
• Post Extraction: Gather detailed post data with support for both image and video content
• Multi-Profile Automation: Automatically scrape profiles followed by the target user
• Post Details: Navigate to individual posts to extract full content, comments, and engagement metrics
• Media Download: Save images and videos locally or to Apify's storage
• Comment Extraction: Get complete comment threads with replies
• Regional Support: Scrape profiles from multiple regions (US, AU, JP, etc.)
• Flexible Configuration: Customize the scraper based on your needs
• Proxy Support: Use Apify proxies or custom proxies for enhanced performance and reliability

Usage

Basic Usage

To scrape a Lemon8 profile, provide either a username or a complete profile URL:

{
  "username": "example_user",
  "region": "us",
  "limit": 50
}

OR

{
  "profileUrl": "https://www.lemon8-app.com/@example_user?region=us",
  "limit": 50
}

Multi-Profile Scraping

The scraper can automatically follow and scrape profiles that the target user is following:

{
  "username": "example_user",
  "region": "us",
  "limit": 20,
  "getFollowing": true,
  "followingLimit": 25
}

This will:

1. Scrape the main profile (example_user)
2. Find up to 25 profiles that example_user follows
3. Scrape each of those profiles
4. Store all data in a structured format

Detailed Post Data

To extract comprehensive post information, including full content and comments:

{
  "username": "example_user",
  "region": "us",
  "limit": 50,
  "getDetails": true,
  "detailsLimit": 10
}

This will scrape up to 50 posts from the profile, and get detailed information for the first 10 posts.

Media Downloads

To download and save images and videos to the Key-Value Store:

{
  "username": "example_user",
  "region": "us",
  "limit": 20,
  "saveImages": true,
  "saveVideos": true
}

When these options are enabled:

• Images will be downloaded and stored in the Key-Value Store
• Video files and their poster thumbnails will be downloaded
• The dataset will include links to both original media and downloaded versions
• Media files can be accessed directly via the Key-Value Store URLs

This makes it easy to:

• Access the media files programmatically through the Apify API
• Download all media from a profile in one go
• Create backups of Lemon8 content
• Analyze media files locally

Output

The scraper generates output in two places:

1. Dataset: A structured dataset containing all profile and post information
2. Key-Value Store: Contains media files (images and videos) and additional result data

Dataset Output

The dataset contains:

• User Information: Profile details, follower/following counts, bio, social links
• Posts: Title, content, engagement metrics, post URLs
• Detailed Post Content: Full content, hashtags, edit times, related topics
• Comments: Comment threads with replies, author information, timestamps
• Media URLs: Links to images and videos (both original and downloaded versions)
• Following Profiles: Information about profiles the user follows (when enabled)

Key-Value Store Output

The Key-Value Store contains:

• ALL_RESULTS: Complete structured data in JSON format
• METADATA: Summary statistics and configuration information
• Downloaded Media Files: All images and videos downloaded from posts when saveImages or saveVideos options are enabled

When media downloading is enabled, the scraper automatically:

• Downloads images and video files from Lemon8
• Stores them in the Key-Value Store with unique identifiers
• Updates the dataset with links to the downloaded files

Output Structure

The main output structure includes:

{
  "userInfo": {
    "name": "Example User",
    "bio": "User bio...",
    "followers": "10.5K",
    "following": "125",
    "likesAndSaves": "42.2K",
    "profileImageUrl": "https://...",
    "profileUrl": "https://...",
    "socialLinks": ["https://..."]
  },
  "posts": [
    {
      "id": "7242124578234",
      "title": "Post title",
      "content": "Post content preview...",
      "postUrl": "https://...",
      "imageUrl": "https://...",
      "statistics": {
        "likesCount": "423",
        "commentsCount": "25",
        "savedCount": "31"
      },
      "author": {
        "name": "Example User",
        "profileUrl": "https://...",
        "profileImageUrl": "https://..."
      },
      "details": {
        "fullContent": "Full post content...",
        "hashtags": ["fashion", "style"],
        "editTime": "2025-05-10",
        "isVideo": false,
        "videoData": null,
        "statistics": {...},
        "galleryImages": [...]
      },
      "comments": [...],
      "allComments": [...],
      "savedImageUrl": "https://api.apify.com/v2/key-value-stores/YOUR-STORE/records/image_7242124578234_1.jpg",
      "videoData": {
        "savedVideoUrl": "https://api.apify.com/v2/key-value-stores/YOUR-STORE/records/video_7242124578234.mp4",
        "savedPosterUrl": "https://api.apify.com/v2/key-value-stores/YOUR-STORE/records/poster_7242124578234.jpg"
      }
    }
  ],
  "metadata": {
    "profileUrl": "https://...",
    "totalScraped": 50,
    "scrollsPerformed": 5,
    "limit": 50,
    "region": "us",
    "videoPostsFound": 12,
    "detailedPostsScraped": 10,
    "followingProfilesScraped": 25
  }
}

When getFollowing is enabled, the output will include multiple profile objects in an array.

Configuration Options

Option	Type	Description
profileUrl	String	Full Lemon8 profile URL to scrape
username	String	Lemon8 username without the '@' symbol
region	String	Target region (us, au, nz, jp, th, id, vn, my, sg, ca)
limit	Number	Maximum number of posts to extract per profile
saveImages	Boolean	Whether to download post images
saveVideos	Boolean	Whether to download videos from posts
getDetails	Boolean	Whether to get detailed post information
detailsLimit	Number	Maximum number of posts to get details for
getFollowing	Boolean	Whether to scrape profiles the user follows
followingLimit	Number	Maximum number of following profiles to scrape
proxy	Object	Proxy configuration options
dev_transform_fields	Array	Fields to include in the output
dev_dataset_name	String	Custom dataset name with placeholders
dev_dataset_clear	Boolean	Whether to clear the dataset before insertion

Accessing Your Data

Accessing Dataset Results

The dataset contains structured data that can be:

• Viewed directly in the Apify console under the "Dataset" tab
• Exported in various formats (JSON, CSV, Excel, etc.)
• Accessed programmatically via the Apify API

Accessing Downloaded Media

All downloaded media files are stored in the Key-Value Store and can be:

• Viewed and downloaded directly from the Apify console under the "Key-value store" tab
• Accessed via direct URLs (useful for embedding or sharing)
• Downloaded programmatically using the Apify API
• Processed in bulk using the Apify API client libraries

Example of accessing a downloaded image:

https://api.apify.com/v2/key-value-stores/[STORE_ID]/records/image_[POST_ID]_1.jpg

Example of accessing a downloaded video:

https://api.apify.com/v2/key-value-stores/[STORE_ID]/records/video_[POST_ID].mp4

Using Custom Proxies

{
  "username": "example_user",
  "region": "us",
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  }
}

Field Transformation

Selectively include only specific fields in the output:

{
  "username": "example_user",
  "region": "us",
  "dev_transform_fields": [
    "userInfo.name",
    "userInfo.followers",
    "posts.id",
    "posts.title",
    "posts.statistics.likesCount"
  ]
}

Custom Dataset Naming

{
  "username": "example_user",
  "region": "us",
  "dev_dataset_name": "lemon8_{DATE}_{TIME}"
}

This creates datasets like lemon8_20250521_153045.

Legal Disclaimer

This tool is provided for educational and research purposes. Always respect Lemon8's terms of service and ensure you have the right to access and use the data you collect.