# Goodreads Explorer: Books, Authors & Reviews Scraper (`thescrapelab/apify-goodreads-scraper`) Actor Scrape public Goodreads data from URLs or simple text targets. Collect books, authors, series, search results, and book reviews with clean structured output. Built for Apify with HTTP-first speed, browser fallback for reliability, proxy support, depth controls, and run-ready dataset/KV summaries. - **URL**: https://apify.com/thescrapelab/apify-goodreads-scraper.md - **Developed by:** [Inus Grobler](https://apify.com/thescrapelab) (community) - **Categories:** E-commerce, SEO tools, News - **Stats:** 3 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Goodreads Books, Authors & Reviews Scraper Goodreads Books, Authors & Reviews Scraper is an Apify Actor for scraping public Goodreads data without using the Goodreads API. Use it to scrape Goodreads book pages, author profiles, series pages, search results, ratings, genres, ISBN details, shelves, and public book reviews from either Goodreads URLs or simple text targets such as book titles and author names. This Actor is designed to be easy to use for non-technical users and practical for production users. In the Apify input UI, you only need to provide: - `targets` - `depth` - `searchMode` - `maxReviewsPerBook` when you want to cap reviews per opened book - `proxyConfiguration` If you want a Goodreads scraper for book metadata, author data, Goodreads search results, or Goodreads review scraping from public book pages, this Actor is built for that use case. ### Why Use This Goodreads Scraper - Scrape public Goodreads books, authors, series, search results, and reviews - Start from a Goodreads URL or a plain-text search like `Dune` or `Stephen King` - Use simple depth-based controls instead of a long list of confusing toggles - Get clean structured JSON in the default dataset - Get run summaries and failure reports in the default key-value store - Use HTTP-first crawling for speed, with browser fallback for reliability - Run it on Apify with proxy support and production-ready deployment files ### What This Actor Can Scrape #### Goodreads books - Title, subtitle, full title - Book cover image - Description - Average rating - Ratings count - Reviews count - Genres - Shelves URL - First published date - Edition info - ISBN, ISBN13, ASIN - Page count - Language - Format - Series info - Linked authors - Public review records from the book page #### Goodreads authors - Author name - Goodreads author ID - Photo - Biography - Average rating - Ratings count - Reviews count - Website links - Genres - Bibliography summary #### Goodreads series - Series title - Series description - Books in the series - Linked book URLs - Authors when visible #### Goodreads search results - Book search results - Author search results - Source query and result position #### Goodreads reviews - Reviewer name - Reviewer profile URL - Review ID when available - Review date when available - Star rating when available - Review text - Likes count when available - Comments count when available - Spoiler marker when available Important note: - This Actor scrapes public reviews visible on public Goodreads book pages. - Direct `review/show/...` pages are often login-gated on Goodreads and are not the primary review collection surface here. ### Easiest Way To Use It In the Apify input UI, add one or more items to `targets`. Each target can be: - A Goodreads book URL - A Goodreads author URL - A Goodreads series URL - A Goodreads search URL - A book title - An author name - A mixed query like `The Hobbit J.R.R. Tolkien` Then choose `depth`: - `shallow` - fastest, root entities only, light search and light detail - `standard` - best default, follows the most useful related data including public reviews from book pages - `deep` - richest output, broader follow-up, slower runs Then choose `searchMode`: - `books` - return matched books and follow linked entities by depth - `reviews` - find top matched books for text queries, then crawl reviews across review pages for each matched book Then optionally set `maxReviewsPerBook`: - leave it empty to scrape all visible review pages for each opened book - set it to a number only if you want to cap reviews per book Then keep Apify Proxy enabled for production and run the Actor. ### Best Input Examples #### Scrape a Goodreads book URL with reviews Use `standard` or `deep` if you want review items. ```json { "targets": [ "https://www.goodreads.com/book/show/11588.The_Shining" ], "depth": "standard", "proxyConfiguration": { "useApifyProxy": true } } ```` #### Scrape a Goodreads author URL ```json { "targets": [ "https://www.goodreads.com/author/show/3389.Stephen_King" ], "depth": "standard", "proxyConfiguration": { "useApifyProxy": true } } ``` #### Search Goodreads by book title ```json { "targets": [ "Dune" ], "depth": "standard", "proxyConfiguration": { "useApifyProxy": true } } ``` #### Search by title and collect reviews from top matched books ```json { "targets": [ "Harry Potter" ], "searchMode": "reviews", "depth": "standard", "proxyConfiguration": { "useApifyProxy": true } } ``` #### Cap reviews per book when needed ```json { "targets": [ "Harry Potter" ], "searchMode": "reviews", "depth": "standard", "maxReviewsPerBook": 200, "proxyConfiguration": { "useApifyProxy": true } } ``` #### Search Goodreads by author name ```json { "targets": [ "Stephen King" ], "depth": "standard", "proxyConfiguration": { "useApifyProxy": true } } ``` #### Run multiple Goodreads targets at once ```json { "targets": [ "https://www.goodreads.com/book/show/44767458-dune", "Frank Herbert", "The Hobbit J.R.R. Tolkien" ], "depth": "deep", "proxyConfiguration": { "useApifyProxy": true } } ``` ### How Search Works If a target is not a Goodreads URL, the Actor treats it as a Goodreads search query. For plain-text targets, the Actor: 1. Builds Goodreads search requests internally 2. Uses `searchMode: "books"` to scrape book search results (authors can still be enabled via advanced `searchEntityTypes`), or `searchMode: "reviews"` to scrape book results only 3. Saves search result records to the dataset 4. Follows matched pages: - in `books` mode, follows books/authors according to depth - in `reviews` mode, follows top matched books and paginates through review pages Depth controls how many book results the Actor follows and how much related data it expands. Once a book page is opened and reviews are enabled, the Actor now scrapes all visible review pages by default. Use `maxReviewsPerBook` only when you want to cap reviews for both direct book scraping and review-search runs. This means you can start from simple inputs like: - `Atomic Habits` - `J.R.R. Tolkien` - `The Hobbit J.R.R. Tolkien` and still get structured Goodreads data back. ### Output Format The default output mode is optimized for easy downstream use in Apify, Make, n8n, Python, JavaScript, spreadsheets, databases, and LLM pipelines. Every dataset item includes core fields where available: - `recordType` - `sourceUrl` - `canonicalUrl` - `scrapedAt` - `detailLevel` - `goodreadsId` Depending on the entity, items may also include: - `sourceContext` - `availabilityFlags` - `paginationInfo` - `linkedEntities` - `breadcrumbs` #### Main record types - `book` - `author` - `series` - `review` - `search_result` #### Example `review` item ```json { "recordType": "review", "canonicalUrl": "https://www.goodreads.com/review/show/78615227", "bookUrl": "https://www.goodreads.com/book/show/830502.It", "reviewerName": "Maciek", "starRating": 5, "reviewText": "My short review text...", "sourceContext": { "parentUrl": "https://www.goodreads.com/book/show/830502.It", "discoveredFrom": "search_result", "query": "Stephen King" } } ``` #### Example `book` item ```json { "recordType": "book", "canonicalUrl": "https://www.goodreads.com/book/show/11588.The_Shining", "goodreadsId": 11588, "title": "The Shining", "averageRating": 4.28, "ratingsCount": 1727467, "reviewsCount": 51808, "genres": [ { "name": "Horror", "url": "https://www.goodreads.com/genres/horror" } ], "authors": [ { "id": 3389, "name": "Stephen King", "profileUrl": "https://www.goodreads.com/author/show/3389.Stephen_King" } ] } ``` ### What Gets Saved In Apify #### Output tab The Actor now defines an Apify output schema, dataset schema, and key-value store schema so the run output is easier to browse in the Apify Console. In a finished run, you will see quick links for: - `Results Overview` - a clean table view for books, authors, series, reviews, and search results - `Detailed Results` - a wider table with longer text fields such as descriptions and review text - `Run Summary` - the `OUTPUT` key-value store record - `Failed Requests` - the `FAILED_REQUESTS` key-value store record - `Debug Log` - the `DEBUG_LOG` key-value store record when debug mode is enabled - `Storage Files` - the default key-value store browser for saved HTML, screenshots, and JSON records #### Dataset Structured result items are written to the default dataset. #### Key-value store The Actor also writes: - `OUTPUT` - crawl summary - `FAILED_REQUESTS` - failed URL records - `DEBUG_LOG` - optional structured debug output when debug mode is enabled The `OUTPUT` record includes useful run totals such as: - requests handled - items written - books scraped - authors scraped - series scraped - reviews scraped - search results scraped - failed URLs - blocked requests ### Why The Output Is Cleaner The Actor intentionally removes noisy fields where possible before writing items. That includes: - dropping `null` and empty values from output - compressing redundant `sourceContext` fields - normalizing messy search-result author names The result is easier-to-read JSON and a cleaner dataset for downstream automation. ### Goodreads Reviews FAQ #### Can I scrape reviews from a Goodreads book URL? Yes. Pass the Goodreads book URL in `targets` and use `standard` or `deep`. #### Will I get `review` dataset items? Yes. Review records are written as separate dataset items with `recordType: "review"` when public review content is available on the book page. #### Does the Actor open direct Goodreads review pages? Not as the primary review strategy. Goodreads often login-gates direct `review/show/...` pages for signed-out access. The Actor focuses on public review data visible from public book pages. #### How do I get more reviews? Use `deep` and increase advanced limits only if you need them. For most users, `standard` is the best balance between output richness and speed. ### Advanced Input Support The published Apify input UI is intentionally simple, but the Actor still supports advanced raw JSON input for power users. Advanced fields include: - `searchMode` - `maxReviewsPerBook` - `startUrls` - `searchQueries` - `searchEntityTypes` - `entityTypes` - `expand` - `detailLevel` - `outputMode` - `limits` - `crawlMode` - `requestDelayMinMs` - `requestDelayMaxMs` - `saveHtml` - `saveScreenshots` - `includeRawBlocks` - `debug` If you do not need fine-grained control, use the simple `targets + depth + searchMode + proxyConfiguration` mode. ### Limitations - This Actor scrapes public Goodreads pages only - It does not use the Goodreads API - It does not log in - It does not access private Goodreads user data - Goodreads layout changes can affect selectors over time - Search quality depends on Goodreads' own ranking and matching - Direct `review/show/...` pages are often restricted for signed-out scraping ### Troubleshooting If a run returns fewer results than expected: 1. Try a direct Goodreads URL in `targets` to confirm the Actor can reach the exact page you want 2. Keep Apify Proxy enabled in production 3. Switch from `shallow` to `standard` if you need reviews 4. Switch from `standard` to `deep` if you need broader follow-up 5. Check the `OUTPUT` and `FAILED_REQUESTS` records in the default key-value store If Goodreads changes its layout: 1. Enable `debug` 2. Optionally enable `saveHtml` 3. Optionally enable `saveScreenshots` 4. Review `DEBUG_LOG` and the saved artifacts ### SEO And Discovery Notes This README intentionally includes the terms users actually search for on Apify and search engines: - Goodreads scraper - Goodreads reviews scraper - Goodreads book scraper - Goodreads author scraper - Goodreads data scraper - Goodreads API alternative - scrape Goodreads reviews - scrape Goodreads books and authors These keywords are used naturally in the title, introduction, headings, examples, and FAQ so the Actor is easier to discover without turning the page into keyword spam. ### Local Development Install dependencies: ```bash npm install ``` Build: ```bash npm run build ``` Run locally: ```bash npm run dev ``` Type-check: ```bash npm run check ``` Main project files: ```text .actor/ actor.json input_schema.json src/ extractors/ output/ search/ utils/ config.ts main.ts router.ts state.ts Dockerfile apify.json package.json README.md ``` ### Apify Deployment This repository is ready for Apify deployment. Included files: - `package.json` - `Dockerfile` - `.actor/actor.json` - `.actor/input_schema.json` - `apify.json` Deploy with the Apify CLI: ```bash apify push ``` Or import the Git repository into Apify Console and build there. ### Good Defaults For most users: - Use Goodreads URLs whenever you already know the exact page you want - Use `standard` as the default depth - Use `deep` only when you want the richest output and broader follow-up - Keep Apify Proxy enabled in production If you want a no-code Goodreads scraper for books, authors, search results, and public book reviews, this Actor is the simplest way to get structured Goodreads data on Apify. # Actor input Schema ## `targets` (type: `array`): One or more Goodreads URLs or plain-text queries such as a book title or author name. ## `depth` (type: `string`): How deeply the Actor should follow related Goodreads data. Deeper levels take longer but return richer results. ## `searchMode` (type: `string`): Choose whether text queries should return books or collect reviews from top matched books. ## `maxReviewsPerBook` (type: `integer`): Optional cap for reviews scraped from each opened Goodreads book. Leave empty to scrape all visible review pages for each opened book. ## `proxyConfiguration` (type: `object`): Standard Apify proxy configuration object. ## Actor input object example ```json { "targets": [], "depth": "standard", "searchMode": "books", "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `resultsOverview` (type: `string`): No description ## `resultsDetails` (type: `string`): No description ## `allResultsJson` (type: `string`): No description ## `allResultsJsonl` (type: `string`): No description ## `runSummary` (type: `string`): No description ## `failedRequests` (type: `string`): No description ## `debugLog` (type: `string`): No description ## `storageFiles` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "targets": [], "depth": "standard", "searchMode": "books", "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("thescrapelab/apify-goodreads-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "targets": [], "depth": "standard", "searchMode": "books", "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("thescrapelab/apify-goodreads-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "targets": [], "depth": "standard", "searchMode": "books", "proxyConfiguration": { "useApifyProxy": true } }' | apify call thescrapelab/apify-goodreads-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=thescrapelab/apify-goodreads-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Goodreads Explorer: Books, Authors & Reviews Scraper", "description": "Scrape public Goodreads data from URLs or simple text targets. Collect books, authors, series, search results, and book reviews with clean structured output. Built for Apify with HTTP-first speed, browser fallback for reliability, proxy support, depth controls, and run-ready dataset/KV summaries.", "version": "0.0", "x-build-id": "4EzGb1vBSldhxkTkK" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/thescrapelab~apify-goodreads-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-thescrapelab-apify-goodreads-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/thescrapelab~apify-goodreads-scraper/runs": { "post": { "operationId": "runs-sync-thescrapelab-apify-goodreads-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/thescrapelab~apify-goodreads-scraper/run-sync": { "post": { "operationId": "run-sync-thescrapelab-apify-goodreads-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "targets": { "title": "Targets", "type": "array", "description": "One or more Goodreads URLs or plain-text queries such as a book title or author name.", "items": { "type": "string" } }, "depth": { "title": "Depth", "enum": [ "shallow", "standard", "deep" ], "type": "string", "description": "How deeply the Actor should follow related Goodreads data. Deeper levels take longer but return richer results." }, "searchMode": { "title": "Search mode", "enum": [ "books", "reviews" ], "type": "string", "description": "Choose whether text queries should return books or collect reviews from top matched books." }, "maxReviewsPerBook": { "title": "Max reviews per book", "minimum": 1, "type": "integer", "description": "Optional cap for reviews scraped from each opened Goodreads book. Leave empty to scrape all visible review pages for each opened book." }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Standard Apify proxy configuration object." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```