# SEO-Friendly Google SERP Scraper for Ads & Organic Results (`lucrateresults/my-actor-2`) Actor Scrape Google SERPs with ease. Extract organic results, ads, and People Also Ask boxes. Supports queries or custom start URLs, proxy rotation, stealth, and random delays to bypass detection. Perfect for SEO, research, and competitive analysis. - **URL**: https://apify.com/lucrateresults/my-actor-2.md - **Developed by:** [Lucrate Results](https://apify.com/lucrateresults) (community) - **Categories:** SEO tools, Automation, AI - **Stats:** 79 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $5.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Google Search Puppeteer Scraper Lucrate Results' Actor is a production-ready **PuppeteerCrawler** boilerplate for scraping websites that require JavaScript execution.\ It loads web pages using **headless Chrome**, extracts structured data, and stores the results into an Apify **dataset** (JSON, CSV, Excel, or API). You can use it for: - Scraping **Google search results, product listings, or SEO data** - Extracting **titles, links, and structured page content** - Automating **JavaScript-heavy sites** where Cheerio won’t work *** ### πŸš€ How It Works 1. **Input**: You provide start URLs in the `INPUT.json` or Apify console. 2. **Proxy Configuration**: Optional – use Apify Proxy or custom proxy URLs. 3. **Crawler**: - Loads pages with Puppeteer - Executes JavaScript (ensuring dynamic content is visible) - Uses `routes.js` to define custom handlers 4. **Data Extraction**: Example handler extracts the page title + URL. 5. **Storage**: Results are pushed to the default Apify **dataset**. *** ### πŸ“₯ Input Example Input schema is defined in `input_schema.json`. ````json { "startUrls": [ { "url": "https://apify.com" }, { "url": "https://crawlee.dev" } ], "proxyConfig": { "useApifyProxy": true } } ### Resources If you're looking for examples or want to learn more visit: - [Crawlee + Apify Platform guide](https://crawlee.dev/docs/guides/apify-platform) - [Documentation](https://crawlee.dev/api/playwright-crawler/class/PlaywrightCrawler) and [examples](https://crawlee.dev/docs/examples/playwright-crawler) - [Node.js tutorials](https://docs.apify.com/academy/node-js) in Academy - [How to scale Puppeteer and Playwright](https://blog.apify.com/how-to-scale-puppeteer-and-playwright/) - [Video guide on getting data using Apify API](https://www.youtube.com/watch?v=ViYYDHSBAKM) - [Integration with Make](https://apify.com/integrations), GitHub, Zapier, Google Drive, and other apps - A short guide on how to create Actors using code templates: [web scraper template](https://www.youtube.com/watch?v=u-i-Korzf8w) ### Getting started For complete information [see this article](https://docs.apify.com/platform/actors/development#build-actor-at-apify-console). In short, you will: 1. Build the Actor 2. Run the Actor ### Pull the Actor for local development If you would like to develop locally, you can pull the existing Actor from Apify console using Apify CLI: 1. Install `apify-cli` **Using Homebrew** ```bash brew install apify-cli ``` **Using NPM** ```bash npm -g install apify-cli ``` 2. Pull the Actor by its unique ``, which is one of the following: - unique name of the Actor to pull (e.g. "apify/hello-world") - or ID of the Actor to pull (e.g. "E2jjCZBezvAZnX8Rb") You can find both by clicking on the Actor title at the top of the page, which will open a modal containing both Actor unique name and Actor ID. This command will copy the Actor into the current directory on your local machine. ```bash apify pull ``` ### Documentation reference To learn more about Apify and Actors, take a look at the following resources: - [Apify SDK for JavaScript documentation](https://docs.apify.com/sdk/js) - [Apify SDK for Python documentation](https://docs.apify.com/sdk/python) - [Apify Platform documentation](https://docs.apify.com/platform) - [Join our developer community on Discord](https://discord.com/invite/jyEM2PRvMU) # Actor input Schema ## `queries` (type: `array`): Enter one or more search queries to scrape results for. ## `maxResults` (type: `integer`): How many results to scrape per query (max ~300). ## `countryCode` (type: `string`): Google domain country (e.g. 'us', 'de', 'uk'). ## `languageCode` (type: `string`): Search language (e.g. 'en', 'de', 'fr'). ## `device` (type: `string`): Choose whether to scrape desktop or mobile results. ## `includeAds` (type: `boolean`): Scrape sponsored results (ads) in addition to organic. ## `includePAA` (type: `boolean`): Extract 'People Also Ask' (PAA) section if present. ## `startUrls` (type: `array`): Optional: Provide specific Google search URLs to scrape instead of queries. ## Actor input object example ```json { "queries": [ "best marketing agencies", "digital marketing trends 2025" ], "maxResults": 50, "countryCode": "us", "languageCode": "en", "device": "desktop", "includeAds": true, "includePAA": true, "startUrls": [ { "url": "https://www.google.com/search?q=apify" } ] } ```` # Actor output Schema ## `query` (type: `string`): No description ## `position` (type: `string`): No description ## `type` (type: `string`): No description ## `title` (type: `string`): No description ## `url` (type: `string`): No description ## `snippet` (type: `string`): No description ## `displayUrl` (type: `string`): No description ## `siteLinks` (type: `string`): No description ## `adBadge` (type: `string`): No description ## `paaQuestion` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "best marketing agencies", "digital marketing trends 2025" ], "countryCode": "us", "languageCode": "en", "startUrls": [ { "url": "https://www.google.com/search?q=apify" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("lucrateresults/my-actor-2").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": [ "best marketing agencies", "digital marketing trends 2025", ], "countryCode": "us", "languageCode": "en", "startUrls": [{ "url": "https://www.google.com/search?q=apify" }], } # Run the Actor and wait for it to finish run = client.actor("lucrateresults/my-actor-2").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "best marketing agencies", "digital marketing trends 2025" ], "countryCode": "us", "languageCode": "en", "startUrls": [ { "url": "https://www.google.com/search?q=apify" } ] }' | apify call lucrateresults/my-actor-2 --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,lucrateresults/my-actor-2" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/7ttQdrFeqsReyRyVq/builds/t3RT7XYhVSpGevXgL/openapi.json