dm Scraper
Pricing
$20.00/month + usage
dm Scraper
Extract product data from DM.de - prices, availability, images, descriptions & more. Search by keywords, scrape categories or individual products. Location-based stock info with ZIP codes. Fast, efficient scraping up to 1000 products per query. Perfect for market research & price monitoring.
5.0 (5)
Pricing
$20.00/month + usage
5
3
3
Last modified
21 days ago
dm scraper
Since dm doesn't provide a good and free API, this actor should help you to retrieve data from it.
The dm data scraper supports the following features:
-
Search Result Scraping - Collect product information directly from search results using either keywords and/or URLs.
-
Category Scraping - Extract product data from categories and/or subcategories of your choice from URLs.
-
Product Detail Scraping - Retrieve comprehensive and detailed information for each individual product.
-
Location-Based Availability - When a ZIP code is provided, the scraper automatically identifies the nearest store and updates product availability information accordingly.
-
The scraper automatically detects the current page of a given category or search URL and continues scraping from that point.
Limitations
-
For each input URL (whether a category or search result), the scraper can only scrape a maximum of 1000 products.
-
This limit is applied per request, regardless of the offset or start value provided. For example, even if the crawl begins from offset 5 or 10, the scraper will still only return up to 1000 items total for that URL.
-
If the category or search contains more than 1000 products, the scraper will not retrieve the products beyond the 1000th item.
-
There is no limitation on the overall result. If you provide multiple URLs and/or search keyword(s), the scraper will return 1000 products per URL and/or search keyword(s).
Bugs, fixes, updates, and changelog
This scraper is under active development. If you have any feature requests you can create an issue from here.
Input Parameters
The input of this scraper should be JSON containing the list of pages on dm that should be visited. Required fields are:
-
zipcode:(Optional) (String) Zipcode value that enforces website to update the availability of the product. -
startUrls: (Optional) (Array) List of dm URLs. You should only provide a search, category, or product URLs. -
search: (Optional) (Array) Keyword(s) that you want to search on dm. -
endPage: (Optional) (Number) Final number of page that you want to scrape. The default isInfinite. This applies to allsearchrequests andstartUrlsindividually. -
maxItems: (Optional) (Number) You can limit the number of scraped items. This should be useful when you search through the big lists or search results. -
customMapFunction: (Optional) (String) Function that takes each object's handle as an argument and returns the object with executing the function. -
proxy: (Required) (Proxy Object) Proxy configuration.
This solution requires the use of Proxy servers, either your own proxy servers or you can use Apify Proxy.
Tip
When you want to scrape over a specific list URL, just copy and paste the link as one of the startUrl.
If you would like to scrape only the first page of a list then put the link for the page and have the endPage as 1.
With the last approach that is explained above you can also fetch any interval of pages. If you provide the 5th page of a list and define the endPage parameter as 6 then you'll have the 5th and 6th pages only.
Compute Unit Consumption
The actor is optimized to run blazing fast and scrape as many items as possible. Therefore, it forefronts all the detailed requests. If the actor doesn't block very often it'll scrape 100 listings in 2 minutes with ~0.03-0.05 compute units.
dm Scraper Input example
During the Run
During the run, the actor will output messages letting you know what is going on. Each message always contains a short label specifying which page from the provided list is currently specified. When items are loaded from the page, you should see a message about this event with a loaded item count and total item count for each page.
If you provide incorrect input to the actor, it will immediately stop with a failure state and output an explanation of what is wrong.
dm Export
During the run, the actor stores results into a dataset. Each item is a separate item in the dataset.
You can manage the results in any language (Python, PHP, Node JS/NPM). See the FAQ or our API reference to learn more about getting results from this dm actor.
Scraped dm Properties
The structure of each item in dm looks like this:
Item Detail
Contact
Please visit us through epctex.com to see all the products that are available for you. If you are looking for any custom integration or so, please reach out to us through the chat box in epctex.com. In need of support? business@epctex.com is at your service.
