# Website Reviews, Comments & Testimonials Scraper (`thescrapelab/universal-public-feedback-scraper`) Actor Turn allowed public website reviews, testimonials, comments, and forum posts into clean feedback intelligence for competitor research, reputation monitoring, and voice-of-customer analysis. - **URL**: https://apify.com/thescrapelab/universal-public-feedback-scraper.md - **Developed by:** [Inus Grobler](https://apify.com/thescrapelab) (community) - **Categories:** Marketing, Developer tools, AI - **Stats:** 3 total users, 1 monthly users, 93.3% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 normalized feedback items This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README Turn public website feedback into a clean, analysis-ready dataset. Paste one or more public URLs to collect **reviews, ratings, comments, testimonials, forum posts, and visible replies**β€”with source links, dates, authors, and confidence scores whenever the page provides them. Use this website review scraper for reputation monitoring, voice-of-customer research, competitor research, testimonial collection, and public community analysis. It is built for public, unauthenticated pages only. ### Start here: turn public feedback into an actionable dataset Use this Actor when you need evidence, not just a page copy. Give it a public feedback URL and export a normalized dataset that can answer questions such as: - **Why do customers choose a competitor?** Collect public review and testimonial pages, then group the `text`, `rating`, and `productOrContext` fields. - **What objections keep appearing?** Collect a public forum or comment thread with replies to identify recurring questions and complaints. - **Which customer quotes can support a sales or research brief?** Export attributable public testimonials with their source URLs and dates. For a first run, paste one public URL and keep the default limits. You get a bounded preview of up to 100 feedback items, rather than an open-ended crawl. ### What can this public feedback scraper do? - Extract public reviews, star ratings, testimonials, comments, and discussion posts in one consistent format. - Read ordinary HTML, Schema.org and JSON-LD markup, embedded public JSON, and JavaScript-rendered pages. - Follow safe same-domain pagination within your page and item limits. - Preserve public source URLs, authors, timestamps, ratings, language, and context when available. - Remove duplicates with stable item IDs, making exports safer to combine over time. - Use AI-assisted page-structure discovery only when standard extraction cannot confidently identify feedback cards. ### Who is it for? - **Reputation and review monitoring:** collect public product, service, employer, or customer reviews. - **Customer research:** combine public comments and testimonials from several websites into one dataset. - **Competitor research:** compare recurring public feedback themes, ratings, and customer language. - **Forum and community research:** extract public discussion posts and visible replies from bounded threads. - **Social-proof research:** collect publicly displayed testimonials and comments for reporting or analysis. ### How to scrape public website reviews and comments The only required input is `startUrls`. You do not need to choose a model, crawler, proxy, page limit, or concurrency setting for a normal run. ```json { "startUrls": [ { "url": "https://example.com/customer-reviews" } ] } ```` By default, the Actor collects reviews, comments, forum posts, and testimonials; includes visible replies; respects `robots.txt`; follows up to 10 pages per starting URL; and saves up to 100 items per starting URL. ### Simple, predictable pricing The active Store price and estimated platform usage are shown before every run. When Pay per Event pricing is enabled, the price breakdown can include a small run-start event, each normalized feedback item, and an AI-template event only when a fallback template is successfully validated. Public pages blocked by login requirements, paywalls, CAPTCHAs, or anti-bot controls do not create feedback-item or AI-template events. Dry runs do not create Pay per Event charges. Set Apify's maximum charge for a run to cap spend; the Actor stops before returning further billable results when that limit is reached. ### Public feedback scraper input options Use optional fields only when you need more control. In Apify Console, the normal flow is split into **Add public URLs**, **Choose what to collect**, and a collapsible **Advanced settings** section. ```json { "startUrls": [ { "url": "https://example.com/product-reviews" }, { "url": "https://forum.example.org/topic/public-feedback" } ], "contentTypes": ["review", "comment"], "includeReplies": true, "maxItemsPerUrl": 50, "maxPagesPerUrl": 3 } ``` | Input | Default | What it does | | --- | --- | --- | | `startUrls` | Required | Public pages to scan. Add up to 100 URLs. | | `contentTypes` | all types | Choose `review`, `comment`, `post`, and/or `testimonial`. | | `includeReplies` | `true` | Include visible public replies when their parent can be identified. | | `maxItemsPerUrl` | `100` | Caps saved feedback records for each starting URL. Maximum: 1,000. | | `maxPagesPerUrl` | `10` | Caps public pages visited for each starting URL. Maximum: 100. | | `useLlmFallback` | `true` | Enables AI-assisted selector discovery for unusual page layouts. | | `respectRobotsTxt` | `true` | Honors the website's robots rules. | | `maxConcurrency` | `2` | Controls crawl speed. Keep this low for polite collection. Maximum: 5. | | `dryRun` | `false` | Produces a run summary without saving feedback items. | ### Public feedback scraper output Each dataset record represents one public feedback item. Missing source values are always `null`; the Actor never invents authors, ratings, dates, titles, or context. ```json { "type": "review", "text": "The setup worked well for our team.", "author": "example-user", "rating": 4.5, "title": "Helpful and easy to follow", "publishedAt": "2025-01-15T10:30:00.000Z", "url": "https://example.com/reviews#review-42", "parentUrl": "https://example.com/reviews", "replyToId": null, "itemId": "8fe23eb49a244b7b1dd5d08500d9ba9191504c8140c1be21ef4a36dc0f6053ee", "language": "en", "productOrContext": "Example product", "extractionMethod": "json_ld", "confidence": 0.97, "scrapedAt": "2026-01-01T12:00:00.000Z" } ``` | Field group | Included fields | | --- | --- | | Feedback | `type`, `text`, `title`, `rating` | | Source and author | `author`, `url`, `parentUrl`, `publishedAt`, `language`, `productOrContext` | | Threading and quality | `replyToId`, `itemId`, `extractionMethod`, `confidence`, `scrapedAt` | Use the **Feedback** dataset view for everyday analysis and **Provenance** for item IDs, extraction method, confidence, and source tracing. Each run also includes a `RUN_SUMMARY` record with counts, warnings, and skipped URLs. ### How the website feedback scraper works 1. Reads structured public data, including JSON-LD and Schema.org `Review`, `Comment`, `Article`, and `DiscussionForumPosting` data. 2. Examines embedded public JSON and common review, comment, testimonial, and forum markup. 3. Follows discoverable, same-domain pagination within your limits. 4. Uses a browser fallback only when page rendering is needed. 5. Uses OpenRouter selector discovery only when deterministic extraction has no usable records or low confidence. 6. Validates AI-discovered selectors locally before extracting anything. The AI fallback discovers page structure only. It does not create, rewrite, summarize, or infer feedback. It receives a minimized sample of relevant visible content and the public page URLβ€”never cookies, credentials, form values, or request headers. ### Optional AI-assisted extraction AI fallback is enabled by default, but it runs only when needed. The Actor uses its tested internal model automatically, so there is no model field to configure in input. To enable it, add an Actor runtime secret named exactly `OPENROUTER_API_KEY`: 1. Open the Actor in Apify Console. 2. Open **Source β†’ Environment variables**. 3. Add `OPENROUTER_API_KEY` and set the value to your OpenRouter key. 4. Enable **Secret**. 5. Leave **Apply environment variables also to the build process** disabled, then save. Never put the OpenRouter key in Actor input, API request bodies, datasets, or examples. If the secret is absent, deterministic extraction still works and affected pages are reported in the run summary. ### Run the feedback scraper from Python ```python import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) actor = client.actor("thescrapelab/universal-public-feedback-scraper") run = actor.call(run_input={ "startUrls": [{"url": "https://example.com/customer-reviews"}], "contentTypes": ["review", "testimonial"], "maxItemsPerUrl": 50, "maxPagesPerUrl": 3, }) if run is None: raise RuntimeError("Actor run did not complete") feedback = client.dataset(run["defaultDatasetId"]) for item in feedback.iterate_items(): print(item["type"], item["text"], item["url"]) ``` Keep `APIFY_TOKEN` outside the input object. It is separate from the Actor's `OPENROUTER_API_KEY` runtime secret. Schedule recurring review checks in Apify, send completed-run webhooks, or connect the dataset to another integration for storage and analysis. ### Control cost and collection scope - Standard HTML and structured-data extraction run first, so many pages need no AI fallback. - AI fallback is called only for empty or low-confidence deterministic results and can make one corrective retry when selector validation fails. - Page and item limits apply independently to every starting URL. For example, 3 URLs with `maxItemsPerUrl: 25` can save at most 75 records. - Start with low limits, review the dataset and run summary, then increase limits only when the target website and your use case justify it. ### Responsible use and limitations - Use only public, unauthenticated pages and comply with each website's terms and applicable law. - The Actor respects `robots.txt` by default and does not log in, submit forms, bypass CAPTCHAs, paywalls, or anti-bot controls. - Private, deleted, hidden, region-restricted, blocked, or otherwise inaccessible content cannot be collected. - Very large client-side applications, linkless infinite scrolling, unusual markup, and websites that block automated access can produce partial or empty results. - Browser fallback is limited to bounded public GET resources. Content that requires non-GET GraphQL calls, WebSockets, or an anonymous-cookie handshake is not supported. - Website layouts change. Use `confidence`, `extractionMethod`, source URLs, and `RUN_SUMMARY` when making downstream decisions. ### Frequently asked questions #### Can I scrape public reviews from any website? The Actor works with publicly accessible, unauthenticated pages that allow collection under their rules. It does not log in, bypass CAPTCHAs or paywalls, or access private content. Coverage depends on the page structure, robots rules, and the website's access controls. #### Does it work on JavaScript-rendered review pages? Yes. It starts with efficient page extraction and uses a bounded browser fallback only when rendering is needed. Some complex applications or sites that block automation can still return partial or no data. #### Will it invent missing review information? No. Unavailable public values are returned as `null`. Use `confidence`, `extractionMethod`, source URLs, and the run summary to evaluate records before downstream decisions. #### How do I keep a run small and affordable? Start with one URL and the default limits, or set lower `maxItemsPerUrl` and `maxPagesPerUrl` values. Review the dataset and run summary before expanding the scope. You can also set a maximum charge in Apify before starting a run. ### Troubleshooting and support - **No feedback rows:** confirm the page is public, allowed by its robots rules, and contains visible review or comment text. - **The run summary reports a block:** try another public page; the Actor does not bypass logins, CAPTCHAs, paywalls, or access controls. - **Some fields are `null`:** the source page did not publish them. The Actor does not invent missing authors, dates, or ratings. - **The dataset is too broad:** narrow `contentTypes`, reduce page and item limits, or start from a more specific feedback URL. For repeatable issues, open the Actor's **Issues** tab and include the run ID, a sanitized input, the affected public URL, and the relevant run-summary warning. Never share tokens, cookies, or private page data. # Actor input Schema ## `startUrls` (type: `array`): Public HTTP(S) pages containing reviews, comments, testimonials, or discussion posts. Private, login-protected, paywalled, and CAPTCHA-protected pages are skipped. ## `contentTypes` (type: `array`): Choose the public feedback types you want in the dataset. ## `includeReplies` (type: `boolean`): Include publicly visible replies when the source exposes them. ## `maxItemsPerUrl` (type: `integer`): Maximum feedback items to save for each starting URL, including any discovered pages. ## `maxPagesPerUrl` (type: `integer`): Maximum public pages to visit for each starting URL, including the first page. ## `useLlmFallback` (type: `boolean`): Use OpenRouter only when standard extraction cannot confidently find feedback. The AI discovers selectors; it never writes or invents feedback text. ## `respectRobotsTxt` (type: `boolean`): Honor robots.txt allow/disallow rules. Keep enabled unless you control the target and have a clear reason to change it. ## `proxyConfiguration` (type: `object`): Accepted for API compatibility. This release uses guarded direct egress and records a warning instead of routing through a remote proxy. ## `maxConcurrency` (type: `integer`): Maximum number of pages processed at the same time. Per-domain pacing and robots.txt can reduce the actual speed. ## `dryRun` (type: `boolean`): Check what can be extracted and review the run summary without saving feedback items to the dataset. ## Actor input object example ```json { "startUrls": [ { "url": "https://httpbun.com/base64/PCFkb2N0eXBlIGh0bWw%2BPGh0bWwgbGFuZz0iZW4iPjxoZWFkPjx0aXRsZT5QdWJsaWMgZmVlZGJhY2sgZml4dHVyZTwvdGl0bGU%2BPC9oZWFkPjxib2R5PjxtYWluPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMSI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIGZpcnN0IGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgT25lPC9zcGFuPjx0aW1lIGRhdGV0aW1lPSIyMDI2LTAxLTAxVDEwOjAwOjAwWiI%2BPC90aW1lPjwvYXJ0aWNsZT48YXJ0aWNsZSBjbGFzcz0iY29tbWVudCIgZGF0YS1jb21tZW50LWlkPSJmaXh0dXJlLTIiPjxwIGNsYXNzPSJjb21tZW50LWJvZHkiPlRoZSBzZWNvbmQgaGFybWxlc3MgcHVibGljIGZpeHR1cmUgY29tbWVudC48L3A%2BPHNwYW4gY2xhc3M9ImNvbW1lbnQtYXV0aG9yIj5GaXh0dXJlIEF1dGhvciBUd288L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDJUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMyI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIHRoaXJkIGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgVGhyZWU8L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDNUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjwvbWFpbj48L2JvZHk%2BPC9odG1sPg==" } ], "contentTypes": [ "review", "comment", "post", "testimonial" ], "includeReplies": true, "maxItemsPerUrl": 100, "maxPagesPerUrl": 10, "useLlmFallback": true, "respectRobotsTxt": true, "maxConcurrency": 2, "dryRun": false } ``` # Actor output Schema ## `results` (type: `string`): The default dataset containing normalized, deduplicated public feedback items. ## `runSummary` (type: `string`): Structured counts, warnings, and skipped URLs with reasons. This record never contains credentials. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://httpbun.com/base64/PCFkb2N0eXBlIGh0bWw%2BPGh0bWwgbGFuZz0iZW4iPjxoZWFkPjx0aXRsZT5QdWJsaWMgZmVlZGJhY2sgZml4dHVyZTwvdGl0bGU%2BPC9oZWFkPjxib2R5PjxtYWluPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMSI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIGZpcnN0IGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgT25lPC9zcGFuPjx0aW1lIGRhdGV0aW1lPSIyMDI2LTAxLTAxVDEwOjAwOjAwWiI%2BPC90aW1lPjwvYXJ0aWNsZT48YXJ0aWNsZSBjbGFzcz0iY29tbWVudCIgZGF0YS1jb21tZW50LWlkPSJmaXh0dXJlLTIiPjxwIGNsYXNzPSJjb21tZW50LWJvZHkiPlRoZSBzZWNvbmQgaGFybWxlc3MgcHVibGljIGZpeHR1cmUgY29tbWVudC48L3A%2BPHNwYW4gY2xhc3M9ImNvbW1lbnQtYXV0aG9yIj5GaXh0dXJlIEF1dGhvciBUd288L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDJUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMyI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIHRoaXJkIGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgVGhyZWU8L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDNUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjwvbWFpbj48L2JvZHk%2BPC9odG1sPg==" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("thescrapelab/universal-public-feedback-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://httpbun.com/base64/PCFkb2N0eXBlIGh0bWw%2BPGh0bWwgbGFuZz0iZW4iPjxoZWFkPjx0aXRsZT5QdWJsaWMgZmVlZGJhY2sgZml4dHVyZTwvdGl0bGU%2BPC9oZWFkPjxib2R5PjxtYWluPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMSI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIGZpcnN0IGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgT25lPC9zcGFuPjx0aW1lIGRhdGV0aW1lPSIyMDI2LTAxLTAxVDEwOjAwOjAwWiI%2BPC90aW1lPjwvYXJ0aWNsZT48YXJ0aWNsZSBjbGFzcz0iY29tbWVudCIgZGF0YS1jb21tZW50LWlkPSJmaXh0dXJlLTIiPjxwIGNsYXNzPSJjb21tZW50LWJvZHkiPlRoZSBzZWNvbmQgaGFybWxlc3MgcHVibGljIGZpeHR1cmUgY29tbWVudC48L3A%2BPHNwYW4gY2xhc3M9ImNvbW1lbnQtYXV0aG9yIj5GaXh0dXJlIEF1dGhvciBUd288L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDJUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMyI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIHRoaXJkIGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgVGhyZWU8L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDNUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjwvbWFpbj48L2JvZHk%2BPC9odG1sPg==" }] } # Run the Actor and wait for it to finish run = client.actor("thescrapelab/universal-public-feedback-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://httpbun.com/base64/PCFkb2N0eXBlIGh0bWw%2BPGh0bWwgbGFuZz0iZW4iPjxoZWFkPjx0aXRsZT5QdWJsaWMgZmVlZGJhY2sgZml4dHVyZTwvdGl0bGU%2BPC9oZWFkPjxib2R5PjxtYWluPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMSI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIGZpcnN0IGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgT25lPC9zcGFuPjx0aW1lIGRhdGV0aW1lPSIyMDI2LTAxLTAxVDEwOjAwOjAwWiI%2BPC90aW1lPjwvYXJ0aWNsZT48YXJ0aWNsZSBjbGFzcz0iY29tbWVudCIgZGF0YS1jb21tZW50LWlkPSJmaXh0dXJlLTIiPjxwIGNsYXNzPSJjb21tZW50LWJvZHkiPlRoZSBzZWNvbmQgaGFybWxlc3MgcHVibGljIGZpeHR1cmUgY29tbWVudC48L3A%2BPHNwYW4gY2xhc3M9ImNvbW1lbnQtYXV0aG9yIj5GaXh0dXJlIEF1dGhvciBUd288L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDJUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjxhcnRpY2xlIGNsYXNzPSJjb21tZW50IiBkYXRhLWNvbW1lbnQtaWQ9ImZpeHR1cmUtMyI%2BPHAgY2xhc3M9ImNvbW1lbnQtYm9keSI%2BVGhlIHRoaXJkIGhhcm1sZXNzIHB1YmxpYyBmaXh0dXJlIGNvbW1lbnQuPC9wPjxzcGFuIGNsYXNzPSJjb21tZW50LWF1dGhvciI%2BRml4dHVyZSBBdXRob3IgVGhyZWU8L3NwYW4%2BPHRpbWUgZGF0ZXRpbWU9IjIwMjYtMDEtMDNUMTA6MDA6MDBaIj48L3RpbWU%2BPC9hcnRpY2xlPjwvbWFpbj48L2JvZHk%2BPC9odG1sPg==" } ] }' | apify call thescrapelab/universal-public-feedback-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=thescrapelab/universal-public-feedback-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Website Reviews, Comments & Testimonials Scraper", "description": "Turn allowed public website reviews, testimonials, comments, and forum posts into clean feedback intelligence for competitor research, reputation monitoring, and voice-of-customer analysis.", "version": "1.0", "x-build-id": "tO3bDLwhrXVeTYyQl" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/thescrapelab~universal-public-feedback-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-thescrapelab-universal-public-feedback-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/thescrapelab~universal-public-feedback-scraper/runs": { "post": { "operationId": "runs-sync-thescrapelab-universal-public-feedback-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/thescrapelab~universal-public-feedback-scraper/run-sync": { "post": { "operationId": "run-sync-thescrapelab-universal-public-feedback-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "Start URLs", "minItems": 1, "maxItems": 100, "type": "array", "description": "Public HTTP(S) pages containing reviews, comments, testimonials, or discussion posts. Private, login-protected, paywalled, and CAPTCHA-protected pages are skipped.", "items": { "type": "object", "properties": { "url": { "title": "URL", "type": "string", "description": "A public HTTP(S) URL.", "pattern": "^https?://[^\\s]+$", "minLength": 8, "maxLength": 4096 } }, "required": [ "url" ], "additionalProperties": false } }, "contentTypes": { "title": "What to collect", "minItems": 1, "uniqueItems": true, "type": "array", "description": "Choose the public feedback types you want in the dataset.", "items": { "type": "string", "enum": [ "review", "comment", "post", "testimonial" ], "enumTitles": [ "Reviews", "Comments", "Forum posts", "Testimonials" ] }, "default": [ "review", "comment", "post", "testimonial" ] }, "includeReplies": { "title": "Include public replies", "type": "boolean", "description": "Include publicly visible replies when the source exposes them.", "default": true }, "maxItemsPerUrl": { "title": "Items per URL", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum feedback items to save for each starting URL, including any discovered pages.", "default": 100 }, "maxPagesPerUrl": { "title": "Pages per URL", "minimum": 1, "maximum": 100, "type": "integer", "description": "Maximum public pages to visit for each starting URL, including the first page.", "default": 10 }, "useLlmFallback": { "title": "Use AI fallback for unfamiliar layouts", "type": "boolean", "description": "Use OpenRouter only when standard extraction cannot confidently find feedback. The AI discovers selectors; it never writes or invents feedback text.", "default": true }, "respectRobotsTxt": { "title": "Respect robots.txt", "type": "boolean", "description": "Honor robots.txt allow/disallow rules. Keep enabled unless you control the target and have a clear reason to change it.", "default": true }, "proxyConfiguration": { "title": "Proxy configuration (API compatibility)", "type": "object", "description": "Accepted for API compatibility. This release uses guarded direct egress and records a warning instead of routing through a remote proxy." }, "maxConcurrency": { "title": "Crawl speed", "minimum": 1, "maximum": 5, "type": "integer", "description": "Maximum number of pages processed at the same time. Per-domain pacing and robots.txt can reduce the actual speed.", "default": 2 }, "dryRun": { "title": "Preview only", "type": "boolean", "description": "Check what can be extracted and review the run summary without saving feedback items to the dataset.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```