Actor picture

Web Scraper Experimental Debug

mtrunkat/web-scraper-experimental-dbgr

Experimental version of Apify Web Scraper with Chrome debugger integrated

No credit card required

Author's avatarMarek Trunkát
  • Modified
  • Users17
  • Runs400
Actor picture
Web Scraper Experimental Debug

To run the code examples, you need to have an Apify account. Replace <YOUR_API_TOKEN> in the code with your API token. For a more detailed explanation, please read about running actors via the API in Apify Docs.

const { ApifyClient } = require('apify-client');

// Initialize the ApifyClient with API token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare actor input
const input = {
    "startUrls": [
        {
            "url": "https://apify.com"
        }
    ],
    "pseudoUrls": [
        {
            "purl": "https://apify.com[(/[\\w-]+)?]"
        }
    ],
    "linkSelector": "a",
    "pageFunction": async function pageFunction(context) {
        // See README for context properties. If the syntax is unfamiliar see the link
        // https://javascript.info/destructuring-assignment#object-destructuring
        const { request, log, jQuery } = context;
    
        // To be able to use jQuery as $, one needs save it into a variable
        // and select the inject jQuery option. We've selected it for you.
        const $ = jQuery;
        const title = $('title').text();
    
        // This is yet another new feature of Javascript called template strings.
        // https://javascript.info/string#quotes
        log.info(`URL: ${request.url} TITLE: ${title}`);
    
        // To save data just return an object with the requested properties.
        return {
            url: request.url,
            title
        };
    },
    "proxyConfiguration": {
        "useApifyProxy": false
    },
    "initialCookies": [],
    "waitUntil": [
        "networkidle2"
    ],
    "customData": {}
};

(async () => {
    // Run the actor and wait for it to finish
    const run = await client.actor("mtrunkat/web-scraper-experimental-dbgr").call(input);

    // Fetch and print actor results from the run's dataset (if any)
    console.log('Results from dataset');
    const { items } = await client.dataset(run.defaultDatasetId).listItems();
    items.forEach((item) => {
        console.dir(item);
    });
})();