Website Reviews, Comments & Testimonials Scraper

Turn public website feedback into a clean, analysis-ready dataset. Paste one or more public URLs to collect reviews, ratings, comments, testimonials, forum posts, and visible replies—with source links, dates, authors, and confidence scores whenever the page provides them.

Use this website review scraper for reputation monitoring, voice-of-customer research, competitor research, testimonial collection, and public community analysis. It is built for public, unauthenticated pages only.

Start here: turn public feedback into an actionable dataset

Use this Actor when you need evidence, not just a page copy. Give it a public feedback URL and export a normalized dataset that can answer questions such as:

• Why do customers choose a competitor? Collect public review and testimonial pages, then group the text, rating, and productOrContext fields.
• What objections keep appearing? Collect a public forum or comment thread with replies to identify recurring questions and complaints.
• Which customer quotes can support a sales or research brief? Export attributable public testimonials with their source URLs and dates.

For a first run, paste one public URL and keep the default limits. You get a bounded preview of up to 100 feedback items, rather than an open-ended crawl.

What can this public feedback scraper do?

• Extract public reviews, star ratings, testimonials, comments, and discussion posts in one consistent format.
• Read ordinary HTML, Schema.org and JSON-LD markup, embedded public JSON, and JavaScript-rendered pages.
• Follow safe same-domain pagination within your page and item limits.
• Preserve public source URLs, authors, timestamps, ratings, language, and context when available.
• Remove duplicates with stable item IDs, making exports safer to combine over time.
• Use AI-assisted page-structure discovery only when standard extraction cannot confidently identify feedback cards.

Who is it for?

• Reputation and review monitoring: collect public product, service, employer, or customer reviews.
• Customer research: combine public comments and testimonials from several websites into one dataset.
• Competitor research: compare recurring public feedback themes, ratings, and customer language.
• Forum and community research: extract public discussion posts and visible replies from bounded threads.
• Social-proof research: collect publicly displayed testimonials and comments for reporting or analysis.

How to scrape public website reviews and comments

The only required input is startUrls. You do not need to choose a model, crawler, proxy, page limit, or concurrency setting for a normal run.

{
  "startUrls": [
    { "url": "https://example.com/customer-reviews" }
  ]
}

By default, the Actor collects reviews, comments, forum posts, and testimonials; includes visible replies; respects robots.txt; follows up to 10 pages per starting URL; and saves up to 100 items per starting URL.

Simple, predictable pricing

The active Store price and estimated platform usage are shown before every run. When Pay per Event pricing is enabled, the price breakdown can include a small run-start event, each normalized feedback item, and an AI-template event only when a fallback template is successfully validated.

Public pages blocked by login requirements, paywalls, CAPTCHAs, or anti-bot controls do not create feedback-item or AI-template events. Dry runs do not create Pay per Event charges. Set Apify's maximum charge for a run to cap spend; the Actor stops before returning further billable results when that limit is reached.

Public feedback scraper input options

Use optional fields only when you need more control. In Apify Console, the normal flow is split into Add public URLs, Choose what to collect, and a collapsible Advanced settings section.

{
  "startUrls": [
    { "url": "https://example.com/product-reviews" },
    { "url": "https://forum.example.org/topic/public-feedback" }
  ],
  "contentTypes": ["review", "comment"],
  "includeReplies": true,
  "maxItemsPerUrl": 50,
  "maxPagesPerUrl": 3
}

Input	Default	What it does
startUrls	Required	Public pages to scan. Add up to 100 URLs.
contentTypes	all types	Choose review, comment, post, and/or testimonial.
includeReplies	true	Include visible public replies when their parent can be identified.
maxItemsPerUrl	100	Caps saved feedback records for each starting URL. Maximum: 1,000.
maxPagesPerUrl	10	Caps public pages visited for each starting URL. Maximum: 100.
useLlmFallback	true	Enables AI-assisted selector discovery for unusual page layouts.
respectRobotsTxt	true	Honors the website's robots rules.
maxConcurrency	2	Controls crawl speed. Keep this low for polite collection. Maximum: 5.
dryRun	false	Produces a run summary without saving feedback items.

Public feedback scraper output

Each dataset record represents one public feedback item. Missing source values are always null; the Actor never invents authors, ratings, dates, titles, or context.

{
  "type": "review",
  "text": "The setup worked well for our team.",
  "author": "example-user",
  "rating": 4.5,
  "title": "Helpful and easy to follow",
  "publishedAt": "2025-01-15T10:30:00.000Z",
  "url": "https://example.com/reviews#review-42",
  "parentUrl": "https://example.com/reviews",
  "replyToId": null,
  "itemId": "8fe23eb49a244b7b1dd5d08500d9ba9191504c8140c1be21ef4a36dc0f6053ee",
  "language": "en",
  "productOrContext": "Example product",
  "extractionMethod": "json_ld",
  "confidence": 0.97,
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}

Field group	Included fields
Feedback	type, text, title, rating
Source and author	author, url, parentUrl, publishedAt, language, productOrContext
Threading and quality	replyToId, itemId, extractionMethod, confidence, scrapedAt

Use the Feedback dataset view for everyday analysis and Provenance for item IDs, extraction method, confidence, and source tracing. Each run also includes a RUN_SUMMARY record with counts, warnings, and skipped URLs.

How the website feedback scraper works

1. Reads structured public data, including JSON-LD and Schema.org Review, Comment, Article, and DiscussionForumPosting data.
2. Examines embedded public JSON and common review, comment, testimonial, and forum markup.
3. Follows discoverable, same-domain pagination within your limits.
4. Uses a browser fallback only when page rendering is needed.
5. Uses OpenRouter selector discovery only when deterministic extraction has no usable records or low confidence.
6. Validates AI-discovered selectors locally before extracting anything.

The AI fallback discovers page structure only. It does not create, rewrite, summarize, or infer feedback. It receives a minimized sample of relevant visible content and the public page URL—never cookies, credentials, form values, or request headers.

Optional AI-assisted extraction

AI fallback is enabled by default, but it runs only when needed. The Actor uses its tested internal model automatically, so there is no model field to configure in input.

To enable it, add an Actor runtime secret named exactly OPENROUTER_API_KEY:

1. Open the Actor in Apify Console.
2. Open Source → Environment variables.
3. Add OPENROUTER_API_KEY and set the value to your OpenRouter key.
4. Enable Secret.
5. Leave Apply environment variables also to the build process disabled, then save.

Never put the OpenRouter key in Actor input, API request bodies, datasets, or examples. If the secret is absent, deterministic extraction still works and affected pages are reported in the run summary.

Run the feedback scraper from Python

import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
actor = client.actor("thescrapelab/universal-public-feedback-scraper")

run = actor.call(run_input={
    "startUrls": [{"url": "https://example.com/customer-reviews"}],
    "contentTypes": ["review", "testimonial"],
    "maxItemsPerUrl": 50,
    "maxPagesPerUrl": 3,
})

if run is None:
    raise RuntimeError("Actor run did not complete")

feedback = client.dataset(run["defaultDatasetId"])
for item in feedback.iterate_items():
    print(item["type"], item["text"], item["url"])

Keep APIFY_TOKEN outside the input object. It is separate from the Actor's OPENROUTER_API_KEY runtime secret.

Schedule recurring review checks in Apify, send completed-run webhooks, or connect the dataset to another integration for storage and analysis.

Control cost and collection scope

• Standard HTML and structured-data extraction run first, so many pages need no AI fallback.
• AI fallback is called only for empty or low-confidence deterministic results and can make one corrective retry when selector validation fails.
• Page and item limits apply independently to every starting URL. For example, 3 URLs with maxItemsPerUrl: 25 can save at most 75 records.
• Start with low limits, review the dataset and run summary, then increase limits only when the target website and your use case justify it.

Responsible use and limitations

• Use only public, unauthenticated pages and comply with each website's terms and applicable law.
• The Actor respects robots.txt by default and does not log in, submit forms, bypass CAPTCHAs, paywalls, or anti-bot controls.
• Private, deleted, hidden, region-restricted, blocked, or otherwise inaccessible content cannot be collected.
• Very large client-side applications, linkless infinite scrolling, unusual markup, and websites that block automated access can produce partial or empty results.
• Browser fallback is limited to bounded public GET resources. Content that requires non-GET GraphQL calls, WebSockets, or an anonymous-cookie handshake is not supported.
• Website layouts change. Use confidence, extractionMethod, source URLs, and RUN_SUMMARY when making downstream decisions.

Frequently asked questions

Can I scrape public reviews from any website?

The Actor works with publicly accessible, unauthenticated pages that allow collection under their rules. It does not log in, bypass CAPTCHAs or paywalls, or access private content. Coverage depends on the page structure, robots rules, and the website's access controls.

Does it work on JavaScript-rendered review pages?

Yes. It starts with efficient page extraction and uses a bounded browser fallback only when rendering is needed. Some complex applications or sites that block automation can still return partial or no data.

Will it invent missing review information?

No. Unavailable public values are returned as null. Use confidence, extractionMethod, source URLs, and the run summary to evaluate records before downstream decisions.

How do I keep a run small and affordable?

Start with one URL and the default limits, or set lower maxItemsPerUrl and maxPagesPerUrl values. Review the dataset and run summary before expanding the scope. You can also set a maximum charge in Apify before starting a run.

Troubleshooting and support

• No feedback rows: confirm the page is public, allowed by its robots rules, and contains visible review or comment text.
• The run summary reports a block: try another public page; the Actor does not bypass logins, CAPTCHAs, paywalls, or access controls.
• Some fields are null: the source page did not publish them. The Actor does not invent missing authors, dates, or ratings.
• The dataset is too broad: narrow contentTypes, reduce page and item limits, or start from a more specific feedback URL.

For repeatable issues, open the Actor's Issues tab and include the run ID, a sanitized input, the affected public URL, and the relevant run-summary warning. Never share tokens, cookies, or private page data.