Newegg Scraper

NewEgg Scraper is an Apify actor that extracts detailed product data and category listings from the Newegg website. NewEgg Scraper can scrape:

• Product details
• Pricing information
• Customer reviews
• Product categories

Why scrape Newegg?

Newegg is a popular e-commerce platform with a wide range of products and a large user base. Scraping data from Newegg can provide valuable insights for price tracking, market analysis, and competitive research. Here are just some of the ways you could use the scraped data:

• Monitor price changes for specific products
• Track product availability and stock levels
• Compare pricing across different product categories

Features

• Scrapes detailed product information from Newegg product pages.
• Extracts product listings from Newegg category pages.
• Stores data in separate named datasets for organized access:
  • products: Contains detailed information of individual products.
  • category-products: Contains product listings from category pages.
• Pushes a merged version of the data into the default dataset for convenience.

How much will it cost to scrape Newegg?

The access to the scraper itself cost $30/month.

Regarding computing costs, Apify provides $5 free usage credits every month on the Apify Free plan. You can get a certain amount of results per month from the NewEgg Scraper for free with these credits. If you need to scrape more data regularly from Newegg, consider upgrading to an Apify subscription. The Starter plan for $49/month allows you to get a higher volume of results every month, while the Scale plan offers even more results for a higher price.

Usage

Input Parameters

The actor accepts the following input parameters:

• products: (Optional) An array of products to scrape, each specified by a URL or Product Identifier.
• categories: (Optional) An array of categories to scrape, each specified by a URL or Category Identifier.
• detailed: (Optional) A boolean, if set to true, it will do a detailed scrape of every product within every Category you scrape.

Notes:

• This scraper utilizes Residential Proxies for all requests to ensure reliable data extraction.
• To avoid being blocked from NewEgg or overloading the website, even whilst using residential proxies, we limit the concurrent requests made by the scraper, thus the runtime is high and it is highly advisable to not set a timeout if you're either using detailed scraping or scraping a lot of products.

Example Input

1{
2    "categories": [
3        {
4            "input": "https://www.newegg.com/p/pl?N=100006676",
5            "start_page": 1,
6            "num_pages": 3
7        },
8        {
9            "input": "100017489",
10            "start_page": 1,
11            "num_pages": 2
12        }
13    ],
14    "detailed": false,
15    "products": [
16        "N82E16814930066",
17        "https://www.newegg.com/asrock-rx6600-cld-8g-amd-radeon-rx-6600-8gb-gddr6/p/N82E16814930066",
18        "N82E16814930067"
19    ]
20}

In this example:

• The products array includes:

  • A direct URL to a product page.
  • A Product Identifier (e.g., N82E16814930066).

• The categories array includes:

  • A direct URL to a category page.
  • A Category Identifier (e.g., 100006676).

Running the Actor

You can run the actor directly from the Apify platform or via API. Ensure you provide the necessary input parameters as specified above.

Output

The actor outputs data in three datasets:

1. Default Dataset: Contains a combined version of both products and category products data. Due to platform limitations, all records are included, and fields not relevant to a record may be empty.
   • Accessible in the Output tab.
   • Views are configured to display relevant fields, but records may include empty values for fields not applicable to them.
2. Named Datasets:
   • products Dataset: Contains detailed product information.
   • category-products Dataset: Contains product listings from category pages.
   • Accessible in the Storage section under Datasets.
   • Data is organized and separated for ease of access.

Data Fields

Products Dataset (products):

• identifier: Product identifier.
• product_title: Title of the product.
• price: Price of the product.
• seller: Seller information.
• description: Product description.
• specifications: List of product specifications.
• url: Product page URL.

Category Products Dataset (category-products):

• identifier: Product identifier.
• name: Name of the product.
• price: Price of the product.
• url: Product page URL.
• category_key: Category key from which the product was listed.
• category_identifier: Category identifier.

Notes on Output Data

• The Output tab displays data from the default dataset. Due to limitations, it may include records with empty fields where data is not applicable.
• For organized and separated data, please refer to the named datasets in the Storage section.
• Fields in the default dataset are tailored to include only relevant fields for each data type, but records of different types are not filtered out from views.

Limitations and Constraints

• The Apify platform's current limitations prevent filtering records in views based on field values (e.g., the type field). Therefore, the Output tab may display records with empty fields.
• Users are advised to use the named datasets in the Storage section for organized and filtered data.
• The actor cannot use the filter property in the dataset_schema to separate data into views based on record types.
• IMPORTANT: For some category pages of newegg - a few ones unrelated to computer components, (e.g., Appliances, Home and Outdoors etc...) due to a different paging scheme the Actor might not properly scrape data. If this is a popular request I might further develop the scraper to include these.

Is it legal to scrape Newegg?

Note that scraping personal data from Newegg, such as customer information, may be subject to data protection regulations. Ensure that you have a legitimate reason for scraping data and comply with applicable laws and regulations. Consult legal counsel if you have any doubts about the legality of your scraping activities.