Google Search Puppeteer Scraper

Lucrate Results' Actor is a production-ready PuppeteerCrawler boilerplate for scraping websites that require JavaScript execution.
It loads web pages using headless Chrome, extracts structured data, and stores the results into an Apify dataset (JSON, CSV, Excel, or API).

You can use it for:

• Scraping Google search results, product listings, or SEO data
• Extracting titles, links, and structured page content
• Automating JavaScript-heavy sites where Cheerio won’t work

---

🚀 How It Works

1. Input: You provide start URLs in the INPUT.json or Apify console.
2. Proxy Configuration: Optional – use Apify Proxy or custom proxy URLs.
3. Crawler:
   • Loads pages with Puppeteer
   • Executes JavaScript (ensuring dynamic content is visible)
   • Uses routes.js to define custom handlers
4. Data Extraction: Example handler extracts the page title + URL.
5. Storage: Results are pushed to the default Apify dataset.

---

📥 Input Example

Input schema is defined in input_schema.json.

{
  "startUrls": [
    { "url": "https://apify.com" },
    { "url": "https://crawlee.dev" }
  ],
  "proxyConfig": {
    "useApifyProxy": true
  }
}

## Resources

If you're looking for examples or want to learn more visit:

- [Crawlee + Apify Platform guide](https://crawlee.dev/docs/guides/apify-platform)
- [Documentation](https://crawlee.dev/api/playwright-crawler/class/PlaywrightCrawler) and [examples](https://crawlee.dev/docs/examples/playwright-crawler)
- [Node.js tutorials](https://docs.apify.com/academy/node-js) in Academy
- [How to scale Puppeteer and Playwright](https://blog.apify.com/how-to-scale-puppeteer-and-playwright/)
- [Video guide on getting data using Apify API](https://www.youtube.com/watch?v=ViYYDHSBAKM)
- [Integration with Make](https://apify.com/integrations), GitHub, Zapier, Google Drive, and other apps
- A short guide on how to create Actors using code templates:

[web scraper template](https://www.youtube.com/watch?v=u-i-Korzf8w)


## Getting started

For complete information [see this article](https://docs.apify.com/platform/actors/development#build-actor-at-apify-console). In short, you will:

1. Build the Actor
2. Run the Actor

## Pull the Actor for local development

If you would like to develop locally, you can pull the existing Actor from Apify console using Apify CLI:

1. Install `apify-cli`

    **Using Homebrew**

    ```bash
    brew install apify-cli
    ```

    **Using NPM**

    ```bash
    npm -g install apify-cli
    ```

2. Pull the Actor by its unique `<ActorId>`, which is one of the following:
    - unique name of the Actor to pull (e.g. "apify/hello-world")
    - or ID of the Actor to pull (e.g. "E2jjCZBezvAZnX8Rb")

    You can find both by clicking on the Actor title at the top of the page, which will open a modal containing both Actor unique name and Actor ID.

    This command will copy the Actor into the current directory on your local machine.

    ```bash
    apify pull <ActorId>
    ```

## Documentation reference

To learn more about Apify and Actors, take a look at the following resources:

- [Apify SDK for JavaScript documentation](https://docs.apify.com/sdk/js)
- [Apify SDK for Python documentation](https://docs.apify.com/sdk/python)
- [Apify Platform documentation](https://docs.apify.com/platform)
- [Join our developer community on Discord](https://discord.com/invite/jyEM2PRvMU)