Any data that you want to have available inside the Extend Output/Scraper Function

Custom data

Enable a more verbose logging to be able to understand what's happening during the scraping

Debug Log

Add or remove properties on the output object or omit the output returning null

You can change the output of the items for your dataset here, or add additional behavior on the scraper.

Extend Output Function

Advanced function that allows you to extend the default scraper functionality, allowing you to manually perform actions on the page

Extend Scraper Function

If you decide to fetch the HTML of the pages, it will take twice as long. Make sure to only enable this if needed

Fetch HTML

Max concurrency

Max request retries

Max items

dope-scraper

INPUT_SCHEMA.json

import Apify from 'apify';

Apify.main(async () => {
    const input = await Apify.getInput();

    await Apify.metamorph('pocesar/shopify-scraper', {
        ...input,
        startUrls: [{
            url: 'http://www.dope.com',
        }],
    });
});

main.js

package.json

# First, specify the base Docker image. You can read more about
# the available images at https://sdk.apify.com/docs/guides/docker-images
# You can also use any other image from Docker Hub.
FROM apify/actor-node:16

# Second, copy just package.json and package-lock.json since those are the only
# files that affect "npm install" in the next step, to speed up the build.
COPY package*.json ./

RUN npm --quiet set progress=false \
 && npm install --only=prod --no-optional \
 && echo "Installed NPM packages:" \
 && (npm list --only=prod --no-optional --all || true) \
 && echo "Node.js version:" \
 && node --version \
 && echo "NPM version:" \
 && npm --version

COPY . ./

ENV APIFY_DISABLE_OUTDATED_WARNING 1
ENV npm_config_loglevel=silent



Dockerfile

Scrape DOPE and extract data on headwear from dope.com. Our DOPE API lets you crawl product information and pricing. The saved data can be downloaded as HTML, JSON, CSV, Excel, and XML.

DOPE Scraper