Wordery Web Scraper

✨ Description

The Wordery Web Scraper is a TypeScript-based tool built with Cheerio and Crawlee. It allows users to scrape detailed book information from the Wordery website based on the following:

• 🔍 Keyword: The main topic or genre of the book.
• 👶 Age Group: The intended audience (optional).
• 🌍 Language: Filter results by language using a predefined mapping (optional).

📚 Scraping Limits and Configuration

How Many Results Can You Scrape?

• Wordery includes a wide variety of books:
  • Millions of Titles: The platform offers a large catalog of books.
  • Genre Variability: Some genres may have limited availability.

⚙️ Request Per Crawl

How Many Results Can You Scrape?

• You can configure the maximum number of requests per crawl to control how many pages are scraped in a single run.
• This helps manage performance and prevents overloading the website.

📝 Inputs

• 🔑 Mandatory Input:
  • Keyword: Fetch books using keywords related to the subject.
• 📅 Optional Inputs:
  • Age group: Specify the target audience (e.g., "13+", "6-8").
  • Language: Filter using language like English, Spanish, and French.
  • Request Per Crawl: You can set the Requests Per Crawl to define the number of requests the actor should perform during a single run.

Example:

1{
2    "title": "The Gothic Stories of H.P. Lovecraft",
3    "currentPrice": "€18.18",
4    "available": true,
5    "url": "https://wordery.com/the-gothic-stories-of-hp-lovecraft-h-p-lovecraft-author-xavier-aldana-reyes-editor-british-library-publisher-9780712352468?cTrk=MjA4MDM0ODYyfDY3OTIzMzk4ODZkZjQ6MToxNDo2NzkyMzM5ODcxNTdhNy42OTUzNDMzMTowZGNmZmEwNA%3D%3D",
6    "description": "H. P. Lovecraft is best known for his tales of cosmic horror, in which unnameable nightmares torment the limits of human consciousness. This mastery of weird and unspeakable terror is underpinned by the writer's sizeable contribution to Gothic fiction. This new collection of Lovecraft's stories is the first to concentrate on his Gothic writing and includes tales from the beginning to the very end of the author's career. The writer's weird vision mixes brilliantly with the trappings of earlier Gothic horror to form innovative mosaics of frightful fiction that will long haunt the reader's subconscious.The Gothic Stories of H.P. Lovecraft Hardback edition by H. P. Lovecraft (author), Xavier Aldana Reyes (editor), British Library (publisher)",
7    "bookDetails": {
8        "author": {
9            "author": [
10                "H. P. Lovecraft (author)",
11                "Xavier Aldana Reyes (editor)",
12                "British Library (publisher)"
13            ],
14            "biography": "H. P. Lovecraft (1890-1937) was perhaps the best and most significant American writer of horror fiction in the 20th century. Xavier Aldana Reyes is Senior Lecturer in English Literature and Film at Manchester Metropolitan University, UK, and a founding member of the Manchester Centre for Gothic Studies."
15        },
16        "ISBN-13": "9780712352468",
17        "ISBN-10": "0712352465",
18        "series": "British Library Hardback Classics",
19        "format": "Hardback",
20        "publisher": "British Library Publishing",
21        "publicationDate": "8/23/2018",
22        "pages": 208,
23        "dimensions": "152 x 207 x 26mm",
24        "weight": 550,
25        "condition": "New",
26        "languages": [
27            "English"
28        ]
29    },
30    "tags": [
31        "Fiction Classic",
32        "Fiction Horror & Ghost Stories Classic Horror & Ghost Stories"
33    ],
34    "imageUrl": "https://wordery.com/jackets/69df5ba0/the-gothic-stories-of-hp-lovecraft-h-p-lovecraft-author-xavier-aldana-reyes-editor-british-library-publisher-9780712352468.jpg?width=281"
35 }

The scraper outputs structured JSON data adhering to a predefined schema.

⚠️ Important Notes

• Dynamic Properties: Some properties may not always exist in the data returned by Wordery. The structure of the data may change depending on the product. For example, some books might not have all the properties listed in the schema (e.g., publicationDate, ISBN, etc.).
• Schema Variability: The schema might differ between products and categories. Sometimes, certain fields might be missing, while other times, there may be additional fields not present in the schema options. Always account for possible variability in the scraped data.
• Empty Offers: The actor does not automatically filter out empty or unavailable offers. It returns all available results based on the input configuration, so you may receive results that include offers with missing or incomplete information.