we are working on urgent client project can you please provide the solution asap.

Can you share input you are feeding to the actor?

endpoint ?? body ??

We are encountering a limitation with the LinkedIn post scraper when attempting to scrape posts using the "RSV" keyword. Despite LinkedIn showing over 1000 posts for this query, the Actor only returns approximately 345 posts.

{ "body": { "query": "RSV", "page": 1, "sort_by": "date_posted" }, "endpoint": "search-posts" }

output json contain this value: 'total_posts': 379

got only 379 records which is around 5 days data. can you tell me how to get the all data.

Did you go page by page upto page 100?

I believe it returns 10 posts per page

"It is returning 20 records per page, and I ran up to page 18, where I retrieved around 379 posts. After that, on page 19, it returned an 'URN not found' error.

This is because there were only 379 posts found against your specified keyword

no, on ui there are a lot of posts more than 10000.

379 records are only from last 5 days.

I have tested more than 5 keywords and it only fetched the latest 300 to 400 records. I think it is restricted to fetching that much data only.

can you check this?

you also can try one record and check the count on both ui and this actor.

Can you share a screenshot of total_posts attribute at the bottom

this is the script which I am using:

import requests
import json
import os

# API token and Actor URL
API_TOKEN = 'api-key'
ACTOR_URL = f"https://api.apify.com/v2/acts/muhammad_usama~linkedin-data-scraper-every-thing/run-sync-get-dataset-items?token={API_TOKEN}"


def scrape_linkedin(query, max_pages, postedAgo, endpoint):
    all_pages_data = []  # List to store all page data

    # Iterate over the number of pages
    for page in range(1, max_pages + 1):
        try:
            # Create payload matching the input structure used in UI
            payload = {
                "body": {
                    "query": query,
                    "page": page,  # Incrementing pages
                    "sort_by": "date_posted",  # Sorting by most recent
                   
                },
                "endpoint": endpoint  # Specifying the endpoint
            }

            # Make POST request to the Actor URL
            response = requests.post(ACTOR_URL, json=payload)
            data = response.json()  # Get the JSON data
            
            # Store the data in memory and save to a file
            all_pages_data.append(data)

            # # Save each page as a separate JSON file
            # file_path = os.path.join(output_dir, f'page_{page}.json')
            # with open(file_path, 'w') as file:
            #     json.dump(data, file, indent=4)  # Save the JSON data to a file

            print(f"Scraping and storing page {page} completed.... [trimmed]

As you can see, total_posts:379

I see that, but actually, there are more than 379 posts. Can you search for RSV on LinkedIn's UI once? You will understand what I am saying.

387, what's the wrong

I got your point. but I am saying for our client project we need last 3 months data but we got only last 1 week data from the api. can you tell me how to solve this problem. attached is the last post which I also see on mobile application. I want you to help me to fetch last 3 month post. I hope you got my point?

We scrape as per LinkedIn, all the search filters are already there. You can see all possible filters in the actor information