Rakuten Travel Hotel Rate Scraper

Turns any Rakuten Travel (楽天トラベル) hotel or ryokan page into a daily, machine-readable time series of every room, every plan, every future check-in date — including sold-out dates — with the list price, the coupon-discounted price a guest actually sees, every coupon itemised by name and yen amount, points, and remaining-room counts.

What does Rakuten Travel Hotel Rate Scraper do?

Given one or more Rakuten Travel property URLs or hotel numbers, it scans every day from today through N months ahead (default 6) and records, for each date, every room × plan combination that is bookable — with the hotel's list price, the coupon-discounted price a guest is actually shown, each coupon itemised, points, and remaining rooms — plus a row for every date with nothing bookable.

It is a Data Connector, not a one-off scraper: run it on a daily Apify Schedule and it builds up hotel × stay_date × captured_at history you can use for rate-parity monitoring, competitor tracking, revenue-management analysis, or content research.

What it does not do: it does not log in, so member-rank discounts are out of scope, and it does not use Rakuten's official public API — that API cannot return the coupon-discounted price at all, which is the reason this Actor exists.

Why this exists: the price on the page is not the hotel's price

Rakuten Travel layers its own platform-funded coupons — 楽天スーパーSALE宿クーポン, ボーナス プログラムクーポン, seasonal campaigns — on top of whatever rate a hotel configured. A ¥93,736 room is sold for ¥78,739 once two coupons land on it, and that discounted number is not in the page's HTML (it is filled in by JavaScript, lazily, only for rows the guest scrolls past). Rakuten's official public API does not expose it either — it reports the hotel-configured charge only.

This Actor resolves the real number by calling the same coupon-pricing endpoint Rakuten's own page calls, for every room × plan on the date at once. The full technical investigation is in docs/rakuten-travel-research.md.

Why use Rakuten Travel Hotel Rate Scraper?

• Hotels and ryokan listed on Rakuten Travel who want to know what price guests are actually shown, not just what they configured in their site controller.
• Revenue managers and hotel consultancies monitoring OTA rate parity across a portfolio.
• Data analysts studying Japanese hotel pricing — lead-time curves, day-of-week and holiday surge pricing, ADR trends, campaign intensity over time.

Running it on Apify rather than your own machine gets you scheduling, an API and webhooks to trigger runs and collect results, ready-made integrations (Zapier, Make, Google Sheets and others), proxy rotation you do not have to manage, and a per-run health summary you can monitor — all covered below.

Point it at any Rakuten Travel property. It has no hard-coded logic for any specific hotel.

What data can Rakuten Travel Hotel Rate Scraper extract?

One row per (property × check-in date × room × plan). The fields most people come for:

Field	Type	What it is
checkInDate	string	The stay date, YYYY-MM-DD. The axis of the time series.
capturedAt	string	When this observation was made, so repeated runs stack into history.
availability	boolean | null	true bookable, false confirmed not bookable, null not collected — never conflated.
roomName / planName	string	The room type and the plan sold on it.
basePrice	number	The hotel's own price for the stay, before coupons. Campaign-invariant.
sellingPrice	number	What a guest pays once every coupon Rakuten applies has landed.
discountAmount / discountRate	number	basePrice − sellingPrice, and the same as a percentage.
couponNames	string[]	Which coupons applied, in the order Rakuten applies them.
couponDiscountAmount	number	Their total yen discount.
cardOnlyCouponDiscountAmount	number	The part that requires a specific Rakuten card.
sellingPriceWithoutCardOnlyCoupons	number	What a guest without that card pays. Usually the right comparison.
pointRate / pointAmount	number	Points earned, as a rate and an amount.
remainingRoomCount	number | null	Rakuten's own 残りN部屋 badge, when shown.
cancelPolicyStepsJson	string | null	The cancellation-fee ladder for the plan.
validationFlags	string[]	Data-quality flags. Never used to drop a row.

The full set — occupancy by child age-band, meal plan, room size and conditions, tax-excluded variants, per-person prices, couponsJson verbatim — is in .actor/dataset_schema.json.

Features

• "Today + N months" by default — the natural way to monitor a rolling sale window. Run it daily via an Apify Schedule and you get a real time series.
• Room × plan granularity — not just the cheapest price. Every bookable plan for every room is captured separately, on every date.
• Coupons itemised, never collapsed — each coupon's display name, code, rate, yen amount, card restriction and expiry is kept, so you can tell a Rakuten-funded campaign from a hotel-funded one and reproduce any headline number yourself.
• Card-only coupons separated — sellingPriceWithoutCardOnlyCoupons gives the price a guest without the specific Rakuten card would see, alongside the headline price that includes it.
• Sold-out tracking — dates with nothing bookable are recorded as availability: false rows, not silently skipped.
• Rooms sold without a plan are captured too (roomOnlyBooking: true), not dropped.
• Complete pagination — large hotels list their room types across several pages; all of them are followed and merged before a date's rows are written.
• Six child age-bands — Rakuten prices 小学生 高学年/低学年 and four 幼児 bands differently; all six are addressable.
• HTTP-only — no headless browser, so runs are fast and cheap. A 1-night, 2-adult scan of a large hotel with 53 room types and 17 plans costs one page fetch per listing page plus a single coupon call per date.

How to scrape Rakuten Travel prices

1. Open the Actor and paste one or more properties into Property URLs or hotel numbers. Any URL containing the hotel number works, and a bare number like 188779 is accepted too.
2. Leave Months ahead at 6 to scan every date from today through six months out, or set Start date and End date for an explicit window.
3. Set Adults, Children and Rooms to the occupancy you want priced. Rakuten prices a whole stay for a given occupancy, so this changes every price returned.
4. Optional but recommended for monitoring: put a name in Run summary dataset name, so each run appends one health row to the same table.
5. Leave Proxy configuration at its default. Rakuten Travel sits behind Akamai Bot Manager and unproxied datacenter requests are liable to be blocked.
6. Click Start. A six-month single-property run takes 2–20 minutes depending on how many plans the property publishes.
7. Open the Dataset tab and export as JSON, CSV or Excel, or fetch it from the API.

Input

Field	Type	Default	Description
propertyUrls	string[]	— (required)	Rakuten Travel property URLs or bare hotel numbers. Any URL containing the hotel number works.
monthsAhead	integer	6	Scan today → today + N months. Ignored if startDate+endDate are both set.
startDate / endDate	string (YYYY-MM-DD)	—	Advanced: explicit date range. Past dates are skipped.
adults	integer	2	Adult guests.
children	integer	0	Total children, mapped to the 小学生 高学年 band.
childCounts	object	—	Advanced: per-age-band child counts (see below). Overrides children.
rooms	integer	1	Rooms per booking.
nights	integer	1	Nights per stay.
includeSoldOut	boolean	true	Keep rows for dates with nothing bookable.
maxConcurrency	integer	5	Advanced: max parallel requests.
proxyConfiguration	object	{ useApifyProxy: true }	On by default and effectively required on the Apify platform — see Known limitations.

Simple mode (recommended for most users):

{
  "propertyUrls": ["https://hotel.travel.rakuten.co.jp/hotelinfo/plan/188779"],
  "monthsAhead": 6,
  "adults": 2,
  "rooms": 1,
  "nights": 1
}

Advanced mode (explicit range, several properties, mixed child ages): see examples/input-advanced.json.

Child age-bands

Rakuten Travel does not have one "children" number — it has six bands with different meal and bedding rules, and therefore different prices:

childCounts key	Rakuten's label
elementaryUpper	小学生 高学年
elementaryLower	小学生 低学年
preschoolMealAndBedding	幼児 食事・布団付
preschoolMealOnly	幼児 食事のみ
preschoolBeddingOnly	幼児 布団のみ
preschoolNeither	幼児 食事・布団不要

Output

One dataset row per (property × check-in date × room × plan), plus one row per sold-out (property × check-in date). See examples/output-sample.json for two full real rows, and .actor/dataset_schema.json for the formal schema behind the "Rates overview" table in the Apify Console.

A single available row, abbreviated (values captured live during development — re-running shows current prices):

{
  "propertyId": "188779",
  "propertyName": "ＢＯＴＡＮＩＣＡＬ ＰＯＯＬ ＣＬＵＢ",
  "checkInDate": "2026-12-05",
  "checkOutDate": "2026-12-06",
  "roomName": "POOL CLUB ROOM",
  "roomCategory": "ダブル",
  "planName": "【朝食付き】STANDARD PLAN",
  "mealPlan": "朝食あり 夕食なし",
  "availability": true,
  "remainingRoomCount": 6,
  "basePrice": 93736,
  "sellingPrice": 78739,
  "discountAmount": 14997,
  "discountRate": 16,
  "perPersonPrice": 39370,
  "couponCount": 2,
  "couponNames": ["スーパーSALE宿クーポン", "ボーナスプログラムクーポン"],
  "cardOnlyCouponDiscountAmount": 5624,
  "sellingPriceWithoutCardOnlyCoupons": 84363,
  "pointRate": 1,
  "pointAmount": 852,
  "pointAmountAfterDiscount": 716,
  "capturedAt": "2026-09-23T08:47:50.677Z"
}

Pricing data definitions

Rakuten Travel's pricing has more layers than a single "price", so this Actor keeps each one separate rather than deciding for you what "the price" means:

Field	Meaning
basePrice	List price for the whole stay, tax included — the struck-through price on the page.
basePriceExcludingTax	The same total, excluding consumption tax.
sellingPrice	The headline metric. Price after every coupon Rakuten applies, tax included — the number rendered as the price.
sellingPriceExcludingTax	The same, excluding tax.
discountAmount / discountRate	basePrice − sellingPrice, in yen and as a percentage.
perPersonPrice / perPersonPriceExcludingTax	Rakuten's own per-guest split of the discounted price.
couponCount	How many coupons were applied.
couponNames	Their display names, in application order. Rakuten discloses who funds a discount through the coupon's name (a スーパーSALE or ボーナスプログラム coupon is a Rakuten platform campaign), so the names are kept rather than summed away.
couponDiscountAmount	Total coupon discount, tax included. Equals basePrice − sellingPrice.
cardOnlyCouponDiscountAmount	The part of that which comes from coupons requiring a specific Rakuten card.
sellingPriceWithoutCardOnlyCoupons	Selling price counting only coupons any guest can use. Exact, not an estimate: every coupon's discount is computed off basePrice and summed, so excluding a subset is plain subtraction.
couponsJson	Every coupon object verbatim as a JSON string, including fields this Actor does not interpret.
pointRate / pointAmount	Rakuten point rate (%) and the points earned on basePrice.
pointAmountAfterDiscount	Points earned on sellingPrice — the number shown next to the discounted price.
netPriceAfterPoints	sellingPrice − pointAmountAfterDiscount. Not what the guest pays — Rakuten points are awarded after the stay, not spendable at checkout. Provided only for like-for-like comparison against OTAs whose points are instantly redeemable.
remainingRoomCount	Remaining bookable rooms for this exact room × plan × date, from Rakuten's own 残りN部屋 badge.
cancelPolicyStepsJson	The plan's cancellation-fee ladder as JSON (7日前 無料 → 前日 80% → 当日 100% → 不泊 100%), kept as JSON because Rakuten models it as ordered prose rules.
memberDiscountAmount	Membership discount Rakuten reports on the page. Always 0 here — see Known limitations.
validationFlags	Data-quality flags. Never used to drop rows — see below.

Validation flags

Rows are never discarded; anything unusual is flagged instead.

Flag	Meaning
COUPON_LOOKUP_FAILED	The coupon call for this date failed after all retries, so no discount could be determined. sellingPrice falls back to the pre-coupon price. Note this is not raised when a hotel simply has no coupon running — that is reported as couponCount: 0 with no flag.
COUPON_RESULT_MISSING	The coupon call returned results for the date but omitted this specific room × plan, which should not happen.
COUPON_SUM_MISMATCH	The coupons' amounts no longer add up to basePrice − sellingPrice, i.e. Rakuten changed how discounts compose.
PAGE_PRICE_DIFFERS_FROM_COUPON_API	The page's list price and the coupon API's disagree for this row.
SELLING_PRICE_EXCEEDS_BASE_PRICE, NEGATIVE_SELLING_PRICE	Basic sanity violations.
COLLECTION_FAILED	This date was requested but never successfully read. availability is null, not false — see below.

Availability has three states

Value	Meaning
true	Bookable. Price fields are populated.
false	Confirmed not bookable — sold out, marked 空室なし for that room × plan, or past the property's booking window.
null	Not collected. The request failed permanently, or a paginated listing broke partway and the fragment would have understated availability. Always carries COLLECTION_FAILED.

If you are building a time series, treat null as "retry this date", never as a sell-out. These rows are emitted even when includeSoldOut is false, because a collection failure is not a sold-out row and asking for a leaner dataset should not cost you the ability to notice a gap.

Coupons are campaign-scoped, so the time of the scrape matters

Rakuten runs site-wide campaigns (楽天スーパーSALE, お買い物マラソン and similar) that end at 23:59 JST. The coupons those campaigns issue are what make sellingPrice differ from basePrice, so the same property on the same check-in date can return a coupon-discounted price one hour and an undiscounted one the next.

Measured on one property, 25 minutes apart, across midnight JST: 86% of rows carried a coupon discount at 23:46, and 0% at 00:22 — with no coupon-API failure in either run. Both answers are correct; they are simply two different moments in Rakuten's own pricing.

Two practical consequences:

1. Pick one hour and keep it when building a time series, or you will be comparing campaign-window prices against non-campaign prices and reading it as a rate change.
2. couponDiscountAmount: 0 and couponCount: 0 on every row is a normal state, not a broken run. Check totalCouponLookupFailedDates in the run summary to distinguish "no coupon offered" from "we could not ask".

Which field is stable, and which is not. Measured across that same boundary, on 4,529 matched room × plan × date combinations:

Field	What it is	Moves with campaigns?
basePrice	the price the property itself set	No — 100% identical across the boundary
sellingPriceWithoutCardOnlyCoupons	what any guest can pay	Yes, with sale campaigns (days)
sellingPrice	best price with the required Rakuten card	Yes, plus the 5-and-0 day cycle

So pick the field that matches your question rather than trying to pick a "clean" hour: basePrice to see whether the property changed its own pricing, and sellingPriceWithoutCardOnlyCoupons to compare against what a guest without a Rakuten card pays. Some campaigns are card-gated and some are not — on one measured night the same hotel's entire 6% discount required a Rakuten card, while another property's スーパーSALE coupon required nothing. cardOnlyCouponDiscountAmount is the split, and couponNames says which coupon did it.

If you scrape on a cycle, avoid multiples of 5 days. The 5と0のつく日 coupon recurs on dates ending in 5 or 0, so a 5- or 10-day cadence locks onto one phase and reports either always-with or always-without. Daily sampling covers every phase.

Sold-out dates

Sold-out data comes at two levels. Rakuten lists every plan under every room and marks the combinations it cannot sell for a date 空室なし — commonly about half the rows on a near-term date — so each of those becomes its own availability: false row that keeps its roomId, roomName, planId and planName, telling you exactly which plan sold out on which room. When the whole property has nothing bookable, the Actor instead emits a single date-level row with availability: false and the room/plan fields null. Either way every price field is null. Note that Rakuten Travel returns the same response for "sold out" and "past this property's booking window" — properties typically stop accepting bookings somewhere between a few months and a year out — so the two are not distinguishable and both are recorded the same way. Set includeSoldOut: false if you only want bookable rows.

How many months can I scrape?

There is no cap beyond the Input Schema's sanity limit (monthsAhead up to 18). In practice each property stops returning bookable rooms at its own horizon, and dates past it simply come back as sold out, which the Actor records normally rather than erroring. 6 months is the recommended default and matches most properties' realistic sale window.

How much will it cost?

Two charges, because the work has two parts. Rakuten Travel serves exactly one check-in date per page request and publishes no usable availability calendar, so every date costs one page request whatever it contains — that is the fixed part. What a date contains, though, varies enormously: measured over a full six months, one property returned 4.2 room × plan rows per date and another 199.6. That is the variable part.

Charge	Price	What it covers
Date scanned	$0.003	One (property × check-in date): the page request and the coupon pricing call, however heavy the page. Bookable or sold out, both count. A date whose request permanently fails is not charged.
Room × plan row	$0.0002	Each row written: one room × plan on one date, priced with its coupons. A sold-out date writes one row. Rows recording a failed collection are not charged.
Property processed	$0.01	Once per property per run.
Actor start	$0.006 per GB	Once per run per GB of memory — $0.012 at this Actor's 2 GB default. It exists so that very short runs still cover their own container startup.

What that works out to, for one property scanned six months ahead (182 dates), measured on real runs:

Property profile	Rows/date	Total
Ryokan that is mostly sold out	4.2	$0.72
Small ryokan	15.9	$1.15
Boutique resort	64.0	$2.90
Large city hotel, 5–6 listing pages per date	199.6	$7.83

So the floor is predictable — properties × days × $0.003, or $0.55 for a six-month single-property scan — and above that you pay for the rows you actually receive. You are not charged a large hotel's price for a small one, and a large hotel is not priced as if it were small.

For comparison, the nearest row-priced competitors on Apify Store charge $0.0015–$0.005 per row. This Actor's per-row charge is a fraction of that, and the per-date floor is what makes the low row price viable.

Scheduling daily monitoring

Run this Actor on an Apify Schedule once a day with the same simple monthsAhead input. Each run's capturedAt lets you build a property × checkInDate × capturedAt time series — join consecutive days' datasets on (propertyId, checkInDate, roomId, planId) to see price changes, new sell-outs, newly released inventory, and campaign/coupon changes over time.

Run summary: did this run collect everything?

A six-month single-property run writes roughly 12,000 rows, and a large city hotel closer to 45,000. Checking whether it actually collected every date by reading those rows is impractical, so each run answers the question itself and writes the answer to its key-value store as OUTPUT:

{
  "runDateJst": "2026-09-25",
  "complete": true,
  "totalDatesRequested": 182,
  "totalDatesCollected": 182,
  "totalCollectionFailedDates": 0,
  "totalCouponLookupFailedDates": 0,
  "totalRowsPushed": 11988,
  "totalCouponDiscountedRows": 9042,
  "minCheckInDate": "2026-09-25",
  "maxCheckInDate": "2027-03-25",
  "startsOnRunDate": true,
  "httpErrorsByStatus": {},
  "requestRetries": 0,
  "validationFlagCounts": {},
  "properties": [ { "propertyId": "188779", "datesCollected": 182, "collectionFailedDates": [], "...": "..." } ]
}

complete is true only when every property established every requested date and every date was priced with its coupons — no COLLECTION_FAILED row and no failed coupon lookup — so a single boolean tells you whether the day's data is trustworthy. A date where Rakuten simply offers no coupon is not a failure and does not count against it.

Two fields are worth watching specifically:

• httpErrorsByStatus counts every failed attempt, including ones a retry later recovered from. A run that fought its way through repeated blocks looks nothing like one that sailed through, and that difference is invisible in the dataset itself.
• totalCouponDiscountedRows is how many rows carry an actual coupon discount. Zero is not necessarily a fault — Rakuten's coupons are campaign-scoped and switch off when a campaign ends, so a run outside a campaign window legitimately finds none (see "Coupons are campaign-scoped" below). What does indicate a fault is totalCouponLookupFailedDates above zero: that is the coupon API failing, and those rows keep their pre-discount price.

Per property you also get listingPagesFetched (above the date count means the hotel paginates), paginationCapHitDates (dates whose room x plan set may be truncated) and couponLookupFailedDates (the exact dates to re-scan).

If you set runSummaryDatasetName, the same object is also appended as one row per run to that named dataset. On a daily schedule this builds a small health table you can read at a glance instead of opening each run in turn, and named datasets are exempt from storage retention, so the history does not expire.

Webhook / API integration

Like any Apify Actor, you can trigger runs via the Apify API or SDKs, attach a webhook on ACTOR.RUN.SUCCEEDED to push new data into your own pipeline, or use Apify Integrations (Zapier, Make, Google Sheets, …) to route the dataset elsewhere without writing code.

Dataset export

Every run's dataset exports from the Apify Console or API as JSON, CSV, Excel, XML, RSS or HTML. For time-series analysis across many daily runs, export to CSV/Parquet and load into a warehouse keyed on (propertyId, checkInDate, roomId, planId, capturedAt).

Known limitations

• Member-rank pricing is not captured. Rakuten's 楽天会員割引 / Bonus-level discounts depend on a logged-in member's rank, and this Actor deliberately does not log in. memberDiscountAmount is therefore always 0. The coupons that are offered to anonymous visitors — including ボーナス プログラムクーポン — are captured in full, so the publicly quoted price is complete.
• Sold out vs. past the booking window are indistinguishable (see Sold-out dates above).
• Cancellation policies are matched positionally. Rakuten publishes each plan's cancellation ladder in a block keyed by position rather than by plan ID. The Actor aligns it by index and leaves cancelPolicyStepsJson null rather than risk attaching the wrong policy if the structure ever stops lining up.
• One request per date, by design. Rakuten Travel has no hotel-level availability calendar to pre-filter against — its calendar endpoint is scoped to a single room × plan for a single month, so using it would cost more requests than fetching each date directly for any property with more than ~30 room × plan combinations. Sold-out dates therefore still cost a request.
• Datacenter proxy is the supported configuration. { useApifyProxy: true } with no group selects it and is the default. Selecting the RESIDENTIAL group will work but moves enough traffic (37–132 MB for a 6-month single-property scan) to be disproportionately expensive; the Actor logs a warning if you do.
• Runs are fixed at 2 GB of memory, which keeps run costs predictable. Rakuten's listing pages are ~2 MB of HTML each, so parsing them concurrently genuinely needs the headroom — 1 GB was measured to be too tight. maxConcurrency is capped at 10 for the same reason, and because a live production site should not be hit harder than that.
• Proxy is effectively required, not just recommended. Rakuten Travel runs Akamai Bot Manager. Requests from datacenter IP ranges — including Apify's own — are liable to be blocked, so proxyConfiguration defaults to { useApifyProxy: true }. Only turn it off if you have confirmed your setup does not need it (e.g. a self-hosted run from a residential IP).

Tips and advanced options

• Pay for less by collecting less. Rows are charged individually, so includeSoldOut: false drops the sold-out rows and their cost with them. Keep it on if you care when a property sells out — that is rate data too.
• Narrow the window for a cheaper daily run. A regression in date handling, coupon pricing or blocking shows up in a 30-day scan as clearly as in a 182-day one. Scan the full six months when you actually need the far end of the sale window.
• Large hotels cost more because they return more. A property publishing ~200 room × plan rows per date writes about 36,000 rows over six months; a small ryokan writes under 1,000. The run summary's totalRowsPushed tells you which kind you are pointing at.
• Leave maxConcurrency alone unless you have a reason. The default of 5 is deliberately conservative against a live production site, and each listing page is ~2 MB of HTML.
• Keep datacenter proxy. Selecting the RESIDENTIAL group works but moves 37–132 MB per six-month scan at per-GB rates, for no measured benefit — four full scans on datacenter returned zero HTTP errors.

FAQ

Does this need a Rakuten account or login? No. It reads only what an anonymous visitor sees, which is also why memberDiscountAmount is always 0 — member-rank discounts depend on a logged-in account.

Can I scrape past dates? No. Rakuten Travel answers a past check-in date with HTTP 400, so the Actor clamps the range to start no earlier than today in Japan Standard Time.

Why did the same date return a different price this morning? Almost certainly a campaign boundary rather than the hotel repricing. Rakuten's coupons are campaign-scoped and end at 23:59 JST. basePrice is campaign-invariant — measured identical on 4,529 room × plan × date combinations across one such boundary — so compare on that if you want to see what the hotel itself changed. See "Coupons are campaign-scoped" above.

A run came back with fewer dates than I asked for. How do I tell? You do not have to read the rows. Set runSummaryDatasetName, then check complete and totalCollectionFailedDates. Dates that could not be collected are written with availability: null rather than dropped, so they are visible in the data too.

Why is one property so much slower than another? Large hotels paginate. A property with ~200 room × plan combinations needs 5–6 listing pages per date against one for a small ryokan, and those pages are read in sequence within a date.

Support

Found a bug, a property this Actor mishandles, or a field you need that is missing? Open an issue on the Actor's Issues tab. Include the run ID — the run summary in the key-value store has everything needed to reproduce it.

Legal and fair-use notes

This Actor reads only publicly accessible property pages under /hotelinfo/plan/, which Rakuten Travel's robots.txt does not disallow, and never touches the two paths it does disallow. It does not authenticate, does not create bookings, and applies conservative concurrency with automatic backoff.

It collects only publicly available information — the room, plan, price, coupon and points data any visitor can see. It collects no personal data, no guest information, and nothing behind a login, and it never makes, changes or cancels a booking.

It is still your responsibility to use the data lawfully. Review Rakuten Travel's terms of service, keep your request volume reasonable (the defaults are deliberately conservative), and make sure your use has a legitimate basis under the law that applies to you — including the GDPR if you are in the EU or handling EU data. If you plan to republish or commercially redistribute what you collect, take legal advice first. Apify's ethical web scraping guide is a good starting point.