# NerdWallet Personal Finance Scraper (`parseforge/nerdwallet-scraper`) Actor Scrape NerdWallet best-of roundups, product reviews, hub pages, and articles across credit cards, banking, mortgages, loans, insurance, investing, taxes, and small business verticals. - **URL**: https://apify.com/parseforge/nerdwallet-scraper.md - **Developed by:** [ParseForge](https://apify.com/parseforge) (community) - **Categories:** Business, Lead generation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![ParseForge Banner](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg) ## ๐Ÿ’ฐ NerdWallet Personal Finance Scraper > ๐Ÿš€ **Pull every NerdWallet best-of roundup, product review, hub, and editorial guide in seconds.** Filter by vertical or page type, paste a single URL, and ship clean rows to your downstream tools. No API key, no registration, no manual CSV wrangling. > ๐Ÿ•’ **Last updated:** 2026-05-17 ยท **๐Ÿ“Š 26 fields** per record ยท **30,000+ indexed pages across the sitemap** ยท **9 finance verticals from credit cards to small business** ยท **Author and editor attribution on every record** NerdWallet is one of the largest consumer finance editorial sites in the United States, with public best-of rankings, product reviews, hub pages, and long-form guides covering **credit cards, banking, mortgages, loans, insurance, investing, taxes, small business, and personal finance**. Manually copying that catalog into a spreadsheet is days of work and goes stale within a week. This Actor sits in front of the public site, walks the WordPress sitemap, and returns clean structured records covering **headline, vertical, page type, best-of category, summary, key takeaways, table of contents, FAQs, product offers, author and editor bylines, breadcrumbs, related articles, SEO metadata, word count, and publication dates**. You feed in either a single NerdWallet URL, a vertical filter (such as `mortgages` or `banking`), a page-type filter (`roundup`, `review`, `article`, `hub`), or simply a `maxItems` cap to crawl the whole sitemap. The Actor batches the public site in parallel through US residential proxies, deduplicates by URL, and emits one normalized record per page. Free runs preview **10 items** so you can verify the shape before scaling up. Paid runs go up to **1,000,000** items. Schedule it weekly to track editorial updates and new best-of rankings, run it on demand for a one-off competitive snapshot, or wire it into your rate comparison tool, financial planning app, or affiliate site. Every record reflects the public NerdWallet site at the moment of execution. | ๐ŸŽฏ Target audience | ๐Ÿ’ก Primary use cases | |---|---| | ๐Ÿ’ผ Financial advisors and licensed insurance agents | Benchmark editorial verdicts, ratings, and product picks across verticals | | ๐Ÿง‘โ€๐Ÿ’ป Personal finance FinTech and aggregator developers | Power rate-comparison flows and product-recommendation UIs with structured editorial data | | ๐Ÿ“ฐ Affiliate marketers and personal finance bloggers | Map NerdWallet best-of rankings to plan content gaps and competitive coverage | | ๐Ÿ”ฌ Market researchers and consumer protection analysts | Track changes in editorial output, author attribution, and product picks across snapshots | --- ### ๐Ÿ“‹ What the NerdWallet Scraper does - ๐Ÿ’ฐ **Best-of roundup coverage.** Pulls every NerdWallet best-of and "Best X for Y" list across 9 finance verticals with structured editorial ratings, breadcrumbs, FAQs, table of contents, and recommended products. - ๐Ÿ“š **Editorial guide library.** Hub pages, "How to" articles, and methodology pages all returned through the same normalized schema with byline, editor attribution, and publication metadata. - ๐Ÿงญ **Vertical filter.** Narrow a crawl to a single NerdWallet vertical (credit cards, banking, mortgages, loans, insurance, investing, taxes, small business, personal finance) without touching the URL list. - ๐Ÿ” **Page-type filter.** Restrict output to best-of roundups, single-product reviews, rich-text articles, or top-level hub pages so downstream pipelines only see what they care about. - ๐Ÿ‘ค **Author and editor attribution.** Each record carries the byline, job title, bio, profile URL, and headshot URL for the article author, reviewer, and editor when NerdWallet publishes them. Each record bundles imageUrl, title, url, slug, pageType, productVertical, nwRating, ratingCount, bestForCategory, productCount, products, summary, tableOfContents, faqs, author, editor, breadcrumbs, relatedArticles, metaTitle, metaDescription, socialShareImageUrl, wordCount, datePublished, dateModified, contentId, contentSource, and a scrapedAt timestamp. The dataset is downloadable in CSV, Excel, JSON, JSONL, RSS, HTML, and XML formats. > ๐Ÿ’ก **Why it matters:** consumer finance is one of the most affiliate-driven editorial categories on the web, and NerdWallet's product picks shape millions of consumer decisions every month. A dataset you can refresh on a schedule lets you spot ranking changes, new entrants, and editorial pivots within hours, not at the end of the quarter when your brief is stale. --- ### ๐ŸŽฌ Full Demo ๐Ÿšง Coming soon: a 3-minute walkthrough showing input setup, a live run, and how to wire the dataset into a downstream tool. The Actor is fully operational while the video is in production. --- ### โš™๏ธ Input
Field Type Required Description
startUrl string no Any single NerdWallet URL, for example https://www.nerdwallet.com/mortgages/best/mortgage-lenders or https://www.nerdwallet.com/article/insurance/cheap-car-insurance. When set, only that page is scraped and the filters below are ignored.
maxItems integer no Cap on the number of records to push. Free plan caps at 10. Paid plans can go up to 1,000,000.
productVertical string (enum) no Restrict the crawl to a single vertical. Choose from credit-cards, banking, mortgages, loans, insurance, investing, taxes, small-business, or personal-finance.
pageType string (enum) no Only emit pages of one template. roundup = best-of comparison list (/best/...). review = single product or carrier review. article = guide or how-to. hub = vertical landing page (e.g. /mortgages).
proxyConfiguration object no Defaults to Apify Residential proxies routed through US exits. NerdWallet serves US-only content and protects routes with Cloudflare, so US residential is required.
Example: scrape mortgages best-of roundups across the full vertical, 100 items, default proxy. ```json { "productVertical": "mortgages", "pageType": "roundup", "maxItems": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "US" } } ```` Example: pull a single best-of page for credit-card cash back, free preview run. ```json { "startUrl": "https://www.nerdwallet.com/mortgages/best/mortgage-lenders", "maxItems": 1 } ``` > โš ๏ธ **Good to Know:** the Actor walks the NerdWallet WordPress sitemap (35 vertical-specific child sitemaps under wp-sitemap.xml) and requests every matching page through the public URL. Records share one normalized schema across page types, so product-specific fields (products array, nwRating, ratingCount, bestForCategory) populate most strongly on `pageType="roundup"` rows. Other fields like `author`, `editor`, `faqs`, `tableOfContents`, `breadcrumbs`, `relatedArticles`, and the SEO block populate broadly across all page types. A small subset of credit-card comparison routes is heavily protected by NerdWallet anti-bot tooling; the Actor flags any such pages with a CF challenge marker in the log and skips them. Mortgages, banking, insurance, loans, and article URLs scrape cleanly. *** ### ๐Ÿ“Š Output Every record is a flat JSON object with 26 typed fields. Download the dataset as JSON, JSONL, CSV, Excel, HTML, XML, or RSS. #### ๐Ÿงพ Schema | Field | Type | Example | |---|---|---| | ๐Ÿ–ผ๏ธ `imageUrl` | string | null | `https://www.nerdwallet.com/assets/blog/wp-content/uploads/2019/04/GettyImages-1127314780.jpg` | | ๐Ÿ“ `title` | string | null | `11 Best FHA Lenders of May 2026` | | ๐Ÿ”— `url` | string | `https://www.nerdwallet.com/mortgages/best/lenders-for-fha-loans` | | ๐Ÿ†” `slug` | string | null | `lenders-for-fha-loans` | | ๐Ÿท๏ธ `pageType` | string | null | `roundup` | | ๐Ÿงญ `productVertical` | string | null | `mortgages` | | โญ `nwRating` | number | null | `4.8` | | ๐Ÿ”ข `ratingCount` | number | null | `11` | | ๐Ÿ† `bestForCategory` | string | null | `lenders-for-fha-loans` | | ๐Ÿ“ฆ `productCount` | number | null | `11` | | ๐Ÿ›’ `products` | object\[] | `[{ name, advertiserName, nwRating, bonus, apr, annualFee, rewardsRate, minCreditScore, bestFor, productUrl, imageUrl }]` | | ๐Ÿ“ฐ `summary` | string | null | Editorial one-line description from JSON-LD or meta description | | ๐Ÿ“‘ `tableOfContents` | string\[] | `["Who is the best FHA lender?", "How we chose", "Types of FHA loans"]` | | โ“ `faqs` | object\[] | `[{ question, answer }]` pairs lifted from FAQPage microdata or accordion sections | | โœ๏ธ `author` | object | null | `{ name, jobTitle, bio, profileUrl, photoUrl }` | | ๐Ÿง‘โ€๐Ÿ’ผ `editor` | object | null | `{ name, jobTitle, bio, profileUrl, photoUrl }` | | ๐Ÿž `breadcrumbs` | string\[] | `["Mortgages", "11 Best FHA Lenders of May 2026"]` | | ๐Ÿ”— `relatedArticles` | string\[] | URLs to other NerdWallet pages linked from the body | | ๐Ÿท๏ธ `metaTitle` | string | null | `Best FHA Loan Lenders for May 2026 - NerdWallet` | | ๐Ÿ“ `metaDescription` | string | null | SEO meta description string | | ๐Ÿ–ผ๏ธ `socialShareImageUrl` | string | null | Open Graph share image URL | | ๐Ÿ“ `wordCount` | number | null | `20822` | | ๐Ÿ“… `datePublished` | string | null | `2019-04-03T16:45:08.000Z` | | ๐Ÿ”„ `dateModified` | string | null | `2026-05-04T14:18:57.000Z` | | ๐Ÿ†” `contentId` | string | null | `1845302` | | ๐Ÿ“š `contentSource` | string | null | `wp` | | ๐Ÿ“… `scrapedAt` | string (ISO 8601) | `2026-05-17T13:58:23.462Z` | | ๐Ÿšจ `error` | string | Only present on failure rows | #### ๐Ÿ“ฆ Sample records
Typical record (mortgages best-of roundup with author, editor, FAQs, and TOC) ```json { "imageUrl": "https://www.nerdwallet.com/assets/blog/wp-content/uploads/2019/04/GettyImages-1179643709.jpg", "title": "13 Best Home Equity Loan Lenders of May 2026", "url": "https://www.nerdwallet.com/mortgages/best/home-equity-loan-lenders", "slug": "home-equity-loan-lenders", "pageType": "roundup", "productVertical": "mortgages", "nwRating": 4.8, "ratingCount": 1, "bestForCategory": "home-equity-loan-lenders", "productCount": 1, "products": [ { "name": "NerdWallet Mortgages", "advertiserName": "NerdWallet", "nwRating": 4.8 } ], "summary": "See NerdWallet's picks for the best home equity loan lenders and find one that fits your needs.", "tableOfContents": [ "How we chose the best home equity loan lenders", "What is a home equity loan?", "Types of home equity products", "Choosing a home equity loan lender", "Frequently asked questions" ], "faqs": [ { "question": "How do I find the lender with the lowest mortgage rate?", "answer": "Some lenders post mortgage rates on their websites and include tools to see how much your rate might be. But those are just estimates. You'll need to get preapproved for a mortgage to get a rate offer based on your credit score and other financial information." } ], "author": { "name": "Taylor Getler", "jobTitle": "Lead Writer & Content Strategist", "bio": "Taylor Getler is a home and mortgages writer for NerdWallet.", "profileUrl": "https://www.nerdwallet.com/author/taylor-getler", "photoUrl": "https://www.nerdwallet.com/tachyon/2021/06/headshot.jpg?resize=70%2C70" }, "editor": { "name": "Johanna Arnone", "jobTitle": "Managing Editor", "profileUrl": "https://www.nerdwallet.com/author/johanna-arnone" }, "breadcrumbs": ["Mortgages", "13 Best Home Equity Loan Lenders of May 2026"], "metaTitle": "Best Home Equity Loan Lenders of May 2026 - NerdWallet", "metaDescription": "See NerdWallet's picks for the best home equity loan lenders and find one that fits your needs.", "wordCount": 20822, "datePublished": "2019-04-03T16:45:08.000Z", "dateModified": "2026-05-04T14:18:57.000Z", "contentId": "1845302", "contentSource": "wp", "scrapedAt": "2026-05-17T13:58:23.462Z" } ```
Edge case (article guide with rich FAQ and TOC, no per-product offer block) ```json { "imageUrl": "https://www.nerdwallet.com/assets/blog/wp-content/uploads/2024/01/cheap-car-insurance.jpg", "title": "How to Get Cheap Car Insurance in 2026", "url": "https://www.nerdwallet.com/article/insurance/cheap-car-insurance", "slug": "cheap-car-insurance", "pageType": "article", "productVertical": "insurance", "nwRating": null, "ratingCount": null, "bestForCategory": null, "productCount": null, "products": [], "summary": "Strategies and tips for finding the lowest car insurance rates based on driver profile, vehicle, and state.", "tableOfContents": [ "Compare quotes from multiple carriers", "Ask about discounts", "Raise your deductible", "Bundle policies", "Drop unneeded coverage" ], "faqs": [ { "question": "Who has the cheapest car insurance?", "answer": "The cheapest car insurance company for you depends on your driver profile and state." }, { "question": "How can I lower my insurance bill?", "answer": "Increase your deductible, ask about discounts, drop unneeded coverage, and compare quotes from at least three carriers." } ], "author": { "name": "Drew Gula", "jobTitle": "Writer", "profileUrl": "https://www.nerdwallet.com/author/drew-gula" }, "editor": { "name": "Lacie Glover", "jobTitle": "Assigning Editor", "profileUrl": "https://www.nerdwallet.com/author/lacie-glover" }, "breadcrumbs": ["Insurance", "Auto Insurance", "How to Get Cheap Car Insurance in 2026"], "metaTitle": "How to Get Cheap Car Insurance in 2026 - NerdWallet", "wordCount": 1820, "datePublished": "2024-01-15T10:00:00.000Z", "dateModified": "2026-04-01T08:30:00.000Z", "contentId": "1890217", "contentSource": "wp", "scrapedAt": "2026-05-17T13:58:42.000Z" } ```
Sparse record (hub page with minimal product detail, only breadcrumbs and meta) ```json { "imageUrl": "https://www.nerdwallet.com/cdn/banking-hero.jpg", "title": "Banking - NerdWallet", "url": "https://www.nerdwallet.com/banking", "slug": "banking", "pageType": "hub", "productVertical": "banking", "nwRating": null, "ratingCount": null, "bestForCategory": null, "productCount": null, "products": [], "summary": "Compare bank accounts, savings rates, and CDs from top US banks.", "tableOfContents": [], "faqs": [], "author": null, "editor": null, "breadcrumbs": ["Banking"], "relatedArticles": [ "https://www.nerdwallet.com/banking/best/savings-account", "https://www.nerdwallet.com/banking/best/cd-rates", "https://www.nerdwallet.com/banking/best/checking-account" ], "metaTitle": "Banking - NerdWallet", "metaDescription": "Compare bank accounts, savings rates, and CDs from top US banks.", "wordCount": 540, "datePublished": null, "dateModified": "2026-05-10T00:00:00.000Z", "contentId": null, "contentSource": null, "scrapedAt": "2026-05-17T13:59:05.000Z" } ```
*** ### โœจ Why choose this Actor | | Capability | |---|---| | ๐Ÿ’ฐ | **Best-of aware fields.** Title, vertical, page type, best-for category, product count, and a normalized products array surface as typed fields on every roundup record. | | ๐Ÿงญ | **9 verticals in one schema.** Credit cards, banking, mortgages, loans, insurance, investing, taxes, small business, and personal finance share one normalized output so downstream pipelines stay simple. | | ๐Ÿ‘ค | **Author and editor metadata.** Byline, job title, bio, profile URL, and headshot surface for the article author, reviewer, and editor so you can attribute and cite faithfully. | | ๐Ÿ“Š | **28 fields per row.** Image, title, url, page type, vertical, product summary, ratings, FAQs, TOC, author, editor, breadcrumbs, related articles, SEO block, word count, publication dates, content ID, and timestamp. | | ๐Ÿ”„ | **Live data.** Every run hits the public NerdWallet site at the moment of execution. No cache, no stale snapshots. | | ๐Ÿ“‚ | **Any export format.** CSV, JSON, JSONL, Excel, HTML, XML, or stream via the Apify API. | | โฑ๏ธ | **Schedule-friendly.** Drop the Actor into the Apify scheduler and let it refresh weekly, monthly, or on a custom cron. | > ๐Ÿ“Š The NerdWallet WordPress sitemap exposes more than 30,000 indexable URLs across 35 vertical-specific child sitemaps covering articles, best-of roundups, hubs, and reviews in 9 finance verticals, all captured directly from the public site. *** ### ๐Ÿ“ˆ How it compares to alternatives | Approach | Cost | Coverage | Refresh | Filters | Setup | |---|---|---|---|---|---| | Manual copy and paste | Free, but expensive in time | Low | Stale within hours | None | Browser plus spreadsheet | | Generic browser automation scripts | High maintenance | Medium | Brittle | Limited | Custom code, proxy hosting, monitoring | | Paid live finance APIs | $$$ subscription | Variable | Real-time | Vendor-defined | Account, API keys, rate-limit handling | | Aggregator data resellers | $$$ multi-year contracts | Broad but opaque | Quarterly | Limited | Sales call, NDA, integration sprint | | **โญ NerdWallet Scraper** *(this Actor)* | **Pay-per-run on Apify** | **Full sitemap surface** | **On every run** | **Vertical, page type, URL** | **Pick a vertical, click Start** | For analysts and small teams, the trade-off is simple: a maintained Apify Actor with built-in residential proxies, deduplication, and dataset exports beats hand-rolling another scraper. *** ### ๐Ÿš€ How to use 1. ๐Ÿ” **Sign up.** [Create a free Apify account with $5 in monthly credit](https://console.apify.com/sign-up?fpr=vmoqkp). No credit card required at sign up. 2. ๐Ÿ” **Open the Actor.** Search for "NerdWallet Scraper" in the [Apify Store](https://apify.com/parseforge/nerdwallet-scraper) and click "Try for free". 3. โš™๏ธ **Configure inputs.** Pick a vertical from the dropdown or paste a NerdWallet URL, set `maxItems`, and confirm the US residential proxy is enabled. 4. โ–ถ๏ธ **Start the run.** Click "Start" and watch the live log. The first batch of records appears within a minute or two as the Actor walks the sitemap and processes URLs. 5. ๐Ÿ“ฅ **Export the dataset.** Download CSV, Excel, JSON, or pipe the dataset to BigQuery, Google Sheets, Slack, or your own webhook from the Storage tab. > โฑ๏ธ **Total time:** roughly 2 to 4 minutes from signup to first dataset on a paid run; a little more if the public site throttles the sitemap endpoint and the Actor retries. *** ### ๐Ÿ’ผ Business use cases
#### ๐Ÿ’ผ Financial advisor - Brief clients with NerdWallet editorial picks across mortgages, banking, and insurance - Refresh internal product scorecards on the same cadence as NerdWallet edits - Spot new lenders, banks, and carriers the moment they earn a NerdWallet roundup - Cross-reference NerdWallet picks against alternative data sources for due diligence #### ๐Ÿ“Š Insurance agent and broker - Pull editorial reviews and FAQ blocks for auto, home, and life insurance verticals - Map which carriers appear in NerdWallet roundups across multiple states - Build client-ready coverage memos with cited author and editor attribution - Track changes in best-of landing pages to anticipate client questions
#### ๐Ÿง‘โ€๐Ÿ’ป Personal finance FinTech developer - Power a rate-comparison UI with structured editorial picks and ratings - Seed a financial planning aggregator with NerdWallet product data - Wire NerdWallet review snippets into in-app educational flows - Schedule weekly refreshes to keep the in-app knowledge base current #### ๐Ÿ“ฐ Affiliate marketer and personal finance blogger - Map NerdWallet best-of rankings to identify content gaps in your niche - Track which products land in NerdWallet picks to anchor competitive briefs - Surface NerdWallet TOC and FAQ patterns to inform editorial structure - Quantify NerdWallet output by author, vertical, and publication date
*** ### ๐ŸŒŸ Beyond business use cases Data like this powers more than commercial workflows. The same structured records support research, education, civic projects, and personal initiatives.
#### ๐ŸŽ“ Research and academia - Empirical datasets for papers, thesis work, and coursework - Longitudinal studies tracking changes across snapshots - Reproducible research with cited, versioned data pulls - Classroom exercises on data analysis and ethical scraping #### ๐ŸŽจ Personal and creative - Side projects, portfolio demos, and indie app launches - Data visualizations, dashboards, and infographics - Content research for bloggers, YouTubers, and podcasters - Hobbyist collections and personal trackers
#### ๐Ÿค Non-profit and civic - Transparency reporting and accountability projects - Advocacy campaigns backed by public-interest data - Community-run databases for local issues - Investigative journalism on public records #### ๐Ÿงช Experimentation - Prototype AI and machine-learning pipelines with real data - Validate product-market hypotheses before engineering spend - Train small domain-specific models on niche corpora - Test dashboard concepts with live input
*** ### ๐Ÿ”Œ Automating NerdWallet Scraper Run the Actor from your own code or workflow tooling. Every run is exposed as a REST endpoint, with first-class SDKs for Node.js and Python. - ๐Ÿ“ฆ [**Node.js SDK**](https://docs.apify.com/sdk/js) - start runs, stream the dataset, and react to webhooks. - ๐Ÿ [**Python SDK**](https://docs.apify.com/sdk/python) - the same primitives in idiomatic Python for analysts. - ๐Ÿ“š [**Apify API reference**](https://docs.apify.com/api/v2) - call the Actor directly with HTTP, perfect for cron and serverless functions. Schedules: open the Actor in your console and create a Schedule. Pick a cadence (hourly, daily, weekly), supply the same input you would pass manually, and Apify will run the Actor on the clock and post results to the dataset of your choice. *** ### โ“ Frequently Asked Questions
๐Ÿ’ณ Do I need a paid Apify plan to run this Actor? No. You can start on the free plan, which includes **$5 in monthly credit**. That is enough for several preview runs (10 items each) so you can confirm the Actor fits your use case before committing. Paid plans unlock higher caps, more concurrent runs, and larger datasets. [Create a free Apify account here](https://console.apify.com/sign-up?fpr=vmoqkp).
๐Ÿšจ What happens if my run fails or returns no results? Failed page fetches are flagged with a single error row in the dataset rather than aborting the entire run. If the sitemap endpoint itself fails, re-run the Actor a few minutes later or [open our contact form](https://tally.so/r/BzdKgA) and we will investigate.
๐Ÿ“ How many items can I scrape per run? Free users are capped at **10 items per run** so you can preview the output. Paid users can raise `maxItems` up to **1,000,000** per run. [Upgrade here](https://console.apify.com/sign-up?fpr=vmoqkp) if you need full scale.
๐Ÿ•’ How fresh is the data? Every run fetches live data at the moment of execution. There is no cache or delay. The records you get reflect the public NerdWallet site at that moment. Schedule the Actor for a rolling snapshot.
๐Ÿง‘โ€๐Ÿ’ป Can I call this Actor from my own code? Yes. Apify exposes every Actor as a REST endpoint and ships SDKs for [Node.js](https://docs.apify.com/sdk/js) and [Python](https://docs.apify.com/sdk/python). You can start a run, read the dataset, and consume webhooks in a few lines.
๐Ÿ“ค How do I export the data? Every Apify dataset can be downloaded from the console as CSV, JSON, JSONL, Excel, HTML, XML, or RSS. You can also pull results programmatically via the [Apify API](https://docs.apify.com/api/v2) or stream them into BigQuery, S3, and other destinations through built-in integrations.
๐Ÿ“… Can I schedule the Actor to run automatically? Yes. Use the Apify scheduler to run the Actor on any cadence, from hourly to monthly. Results land in your dataset and can be delivered to webhooks, email, Slack, cloud storage, Make, or Zapier.
๐Ÿ” What inputs does the Actor accept? A vertical from the dropdown, a page-type filter (roundup, review, article, hub), a single NerdWallet URL, or simply a `maxItems` cap to crawl the full sitemap. Paid users can also override the proxy group.
๐ŸŒŽ Why do I need a US residential proxy? NerdWallet serves US-only content and protects routes with Cloudflare. A US-routed residential session is required to see the public CMS reliably. The Actor defaults to Apify Residential with a US country code, so this works out of the box.
๐Ÿข Can I use the data commercially? The dataset you produce is yours to use within the bounds of NerdWallet's terms of service and applicable law. Apify is the tooling layer; the legal responsibility for downstream use rests with you. When in doubt, consult counsel.
โš–๏ธ Is this legal? Web scraping of publicly accessible editorial information is broadly permitted in many jurisdictions, but the rules vary by country, by use case, and by the source's terms of service. Use the Actor responsibly, respect rate limits, and check legal guidance for your specific application.
๐Ÿ› ๏ธ I need a custom version, what now? [Open our contact form](https://tally.so/r/BzdKgA) with a short description. We build custom Apify Actors, extend existing ones with new fields, and deliver private deployments for teams.
*** ### ๐Ÿ”Œ Integrate with any app - [**Make**](https://docs.apify.com/platform/integrations/make) - automate multi-step workflows on a visual canvas - [**Zapier**](https://docs.apify.com/platform/integrations/zapier) - connect to 5,000+ apps with no code - [**GitHub**](https://docs.apify.com/platform/integrations/github) - sync runs and datasets with version control - [**Slack**](https://docs.apify.com/platform/integrations/slack) - get notifications when a run finishes - [**Airbyte**](https://docs.apify.com/platform/integrations/airbyte) - move data into your warehouse on a schedule - [**Google Drive**](https://docs.apify.com/platform/integrations/drive) - drop CSVs directly into shared drives *** ### ๐Ÿ”— Recommended Actors - [**๐Ÿ›ก๏ธ Policygenius Insurance Marketplace Scraper**](https://apify.com/parseforge/policygenius-scraper) - pull insurance carrier reviews, comparison guides, and best-of rankings across auto, home, life, health, and other verticals - [**๐Ÿ“ˆ Yahoo Finance Stock & Crypto Scraper**](https://apify.com/parseforge/yahoo-finance-scraper) - retrieve real-time equity quotes, fundamentals, and crypto prices for the issuers behind the products NerdWallet ranks - [**๐Ÿฆ FDIC Bank Financial Data Scraper**](https://apify.com/parseforge/fdic-bank-scraper) - capture FDIC-insured bank balance sheets, deposits, and CALL report metrics for due diligence on banking picks - [**๐Ÿ“Š Alpha Vantage Market Data Scraper**](https://apify.com/parseforge/alpha-vantage-scraper) - tap fundamentals, OHLCV bars, FX, and crypto data through the Alpha Vantage API for investing research - [**๐Ÿ’น Robinhood Stock Data Scraper**](https://apify.com/parseforge/robinhood-scraper) - pull retail trading signals, fundamentals, and similar-stock graphs from Robinhood public pages > ๐Ÿ’ก **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge) for more personal finance, banking, and consumer protection data scrapers. *** **๐Ÿ†˜ Need Help?** [**Open our contact form**](https://tally.so/r/BzdKgA) and we will get back to you within one business day. *** > โš ๏ธ **Disclaimer:** This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by NerdWallet or any of its subsidiaries. All trademarks mentioned are the property of their respective owners. # Actor input Schema ## `startUrl` (type: `string`): Optional. Any single NerdWallet URL to scrape (e.g. https://www.nerdwallet.com/mortgages/best/mortgage-lenders or https://www.nerdwallet.com/article/insurance/auto-insurance-rates). When set, only that single page is scraped and the filters below are ignored. ## `maxItems` (type: `integer`): Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000 ## `productVertical` (type: `string`): Restrict the crawl to a single NerdWallet vertical. Leave empty to scrape across every vertical. Note: credit-cards comparison routes are gated by NerdWallet anti-bot and may not return data; use mortgages, banking, insurance, or article URLs for reliable extraction. ## `pageType` (type: `string`): Only emit pages of this template. 'roundup' = best-of comparison list (/best/...). 'review' = single product or carrier review. 'article' = guide / how-to. 'hub' = vertical landing page (e.g. /mortgages). ## `proxyConfiguration` (type: `object`): Apify Proxy settings. NerdWallet serves US-only content and protects routes with Cloudflare; US residential proxies are required for reliable scraping. ## Actor input object example ```json { "startUrl": "https://www.nerdwallet.com/mortgages/best/mortgage-lenders", "maxItems": 10, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } } ``` # Actor output Schema ## `results` (type: `string`): Dataset with all scraped NerdWallet records # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrl": "https://www.nerdwallet.com/mortgages/best/mortgage-lenders", "maxItems": 10, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }; // Run the Actor and wait for it to finish const run = await client.actor("parseforge/nerdwallet-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrl": "https://www.nerdwallet.com/mortgages/best/mortgage-lenders", "maxItems": 10, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "US", }, } # Run the Actor and wait for it to finish run = client.actor("parseforge/nerdwallet-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrl": "https://www.nerdwallet.com/mortgages/best/mortgage-lenders", "maxItems": 10, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }' | apify call parseforge/nerdwallet-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=parseforge/nerdwallet-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "NerdWallet Personal Finance Scraper", "description": "Scrape NerdWallet best-of roundups, product reviews, hub pages, and articles across credit cards, banking, mortgages, loans, insurance, investing, taxes, and small business verticals.", "version": "0.1", "x-build-id": "EMSiMd0obUXaWFCbI" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/parseforge~nerdwallet-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-parseforge-nerdwallet-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/parseforge~nerdwallet-scraper/runs": { "post": { "operationId": "runs-sync-parseforge-nerdwallet-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/parseforge~nerdwallet-scraper/run-sync": { "post": { "operationId": "run-sync-parseforge-nerdwallet-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "startUrl": { "title": "Start URL", "type": "string", "description": "Optional. Any single NerdWallet URL to scrape (e.g. https://www.nerdwallet.com/mortgages/best/mortgage-lenders or https://www.nerdwallet.com/article/insurance/auto-insurance-rates). When set, only that single page is scraped and the filters below are ignored." }, "maxItems": { "title": "Max Items", "minimum": 1, "maximum": 1000000, "type": "integer", "description": "Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000" }, "productVertical": { "title": "Vertical", "enum": [ "credit-cards", "banking", "mortgages", "loans", "insurance", "investing", "taxes", "small-business", "personal-finance" ], "type": "string", "description": "Restrict the crawl to a single NerdWallet vertical. Leave empty to scrape across every vertical. Note: credit-cards comparison routes are gated by NerdWallet anti-bot and may not return data; use mortgages, banking, insurance, or article URLs for reliable extraction." }, "pageType": { "title": "Page Type", "enum": [ "roundup", "review", "article", "hub" ], "type": "string", "description": "Only emit pages of this template. 'roundup' = best-of comparison list (/best/...). 'review' = single product or carrier review. 'article' = guide / how-to. 'hub' = vertical landing page (e.g. /mortgages)." }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Apify Proxy settings. NerdWallet serves US-only content and protects routes with Cloudflare; US residential proxies are required for reliable scraping." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```