Probe Resources Plus Webhook

jancurn/probe-resources-plus-webhook

Developer tools

Calls jancurn/probe-page-resources and then invokes a hard-coded webhook. The act takes same input as jancurn/probe-page-resources

No credit card required

Jan Čurn

Modified3 months ago
Users3
Runs21

Probe Resources Plus Webhook

Dockerfile

# This is a template for a Dockerfile used to run acts in Actor system.
# The base image name below is set during the act build, based on user settings.
# IMPORTANT: The base image must set a correct working directory, such as /usr/src/app or /home/user
FROM apify/actor-node-basic:v0.21.10

# Second, copy just package.json and package-lock.json since it should be
# the only file that affects "npm install" in the next step, to speed up the build
COPY package*.json ./

# Install NPM packages, skip optional and development dependencies to
# keep the image small. Avoid logging too much and print the dependency
# tree for debugging
RUN npm --quiet set progress=false \
 && npm install --only=prod --no-optional \
 && echo "Installed NPM packages:" \
 && (npm list --all || true) \
 && echo "Node.js version:" \
 && node --version \
 && echo "NPM version:" \
 && npm --version

# Copy source code to container
# Do this in the last step, to have fast build if only the source code changed
COPY  . ./

# NOTE: The CMD is already defined by the base image.
# Uncomment this for local node inspector debugging:
# CMD [ "node", "--inspect=0.0.0.0:9229", "main.js" ]

package.json

{
    "name": "apify-project",
    "version": "0.0.1",
    "description": "",
    "author": "It's not you it's me",
    "license": "ISC",
    "dependencies": {
        "underscore": "latest",
        "apify": "0.21.10",
        "request-promise": "latest"
    },
    "scripts": {
        "start": "node main.js"
    }
}

main.js

const _ = require('underscore');
const Apify = require('apify');
const request = require('request-promise');

Apify.main(async () => {
    const input = await Apify.getValue('INPUT');
    console.log('Calling jancurn/probe-page-resources with input:');
    console.log(JSON.stringify(input, null, 2));
    
    const run = await Apify.call('jancurn/probe-page-resources', input);
    
    // console.log(JSON.stringify(run,null,2));
    console.dir(run.output);
    for (let rec of run.output) {
         console.dir(rec); 
         
         // Test webhook, review the calls at http://mockbin.org/bin/b652e23e-5e5e-4904-8fc1-c55dcd7f6ee1/log
         if (rec.url === 'https://www.example.com/') {
             console.log('Testing webhook');
             
             const options = {
                method: 'POST',
                uri: 'http://mockbin.org/bin/b652e23e-5e5e-4904-8fc1-c55dcd7f6ee1',
                json: true,
                body: {
                    "csp-report": {
                        "blocked-uri": "https://a.postrelease.com/serve/load.js",
                        "document-uri": "https://www.perf.webmd.com/allergies/default.htm",
                        "referrer": "",
                        "violated-directive": " apifier-rpt-networkerr",
                        "effective-directive": "apifier-rpt",
                        "original-policy": "",
                        "disposition": "apifier-rpt",
                        "status-code": "404/Failed/Blocked/etc – HTTP Status Code",
                        'mime-type': 'application/x-javascript',
                        'type': 'script',
                        'method' : 'GET',
                        "script-sample": "",
                        "source-file": "https://a.postrelease.com/serve/load.js"
                    }
                }
            };
            await request(options);
         }
    }
});

Where next?

Build new tools

Are you a developer? Build your own actors and run them on Apify.

Get a custom solution

Get a custom web scraping or RPA solution.