NY Fed Reference Rates Scraper

NY Fed Reference Rates Scraper

Scrape the latest SOFR, EFFR, OBFR, TGCR, and BGCR reference rates straight from the New York Fed's public feeds. Every rate comes with its published date, value, and percentile data. No API key or registration. Export to CSV, JSON, Excel, or XML.

The New York Fed publishes critical overnight funding rates that underpin trillions in derivatives, loans, and mortgages, but its website offers no bulk download API for the latest snapshots. This actor reads the public reference rate feeds directly, letting you pull the most recent published values for all major secured and unsecured rates, or the daily history for a single rate going back thousands of observations. You get a clean, flat dataset without screen-scraping the HTML tables yourself.

Who uses it	What they scrape NY Fed for
Treasury analysts	Pull the latest EFFR and SOFR each morning to update internal funding cost models.
Quantitative researchers	Gather multi-year daily histories of a single rate for backtesting interest rate strategies.
Risk managers	Monitor overnight secured and unsecured rates to track funding market stress signals.
Financial data vendors	Automate the collection of NY Fed reference rates into a data warehouse or client dashboard.

What it does

This Actor collects the latest SOFR, EFFR, OBFR, TGCR, and BGCR reference rates from the New York Fed's public data feeds and returns each observation as a flat row with the rate name, published date, value, and percentile fields.

• 📊 Latest snapshot mode: Pull the single most recent published value for all reference rates at once, or filter to secured only or unsecured only.
• 📅 Single rate history mode: Select one rate type and pull up to 5,000 of its most recent daily observations, with the published date on every row.
• 🔢 Percentile data included: Each rate observation returns the 1st, 25th, 75th, and 99th percentile values alongside the central rate, when the NY Fed publishes them.
• ⚙️ Flexible output control: Cap the total number of records collected per run with the max items setting, so you never exceed your storage or processing budget.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with NY Fed data

📈 Update daily funding benchmarks.

A treasury analyst runs the actor each morning in latest mode to pull the newest SOFR and EFFR values and feed them into an automated cash management dashboard.

📚 Build a reference rate database.

A data engineer schedules the actor in history mode for SOFR, pulling 5,000 daily observations into a cloud data warehouse for internal analytics and model training.

⚠️ Monitor overnight funding stress.

A risk manager runs the actor in latest mode for all rates and compares the EFFR to the OBFR spread to detect early signs of funding market dislocation.

🧪 Backtest interest rate models.

A quant researcher pulls a multi-year daily history of TGCR and BGCR to test a new secured funding curve model against historical percentiles.

Why choose this scraper

	What you get
No API key needed	Reads the same public feeds the NY Fed website uses, with no registration or authentication.
Two collection modes	Grab the latest value for every major rate in one call, or pull a deep daily history for one rate.
Clean flat output	Every observation is a single row with the rate name, date, value, and percentile fields, ready for analysis.
Up to 5,000 history days	Pull decades of daily data for backtesting or long-term trend analysis when you select a single rate.

How it compares

No other Store actor targets NY Fed the same way, so the honest comparison is with the alternatives teams actually weigh.

	NY Fed Reference Rates Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When NY Fed changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor by choosing between a latest snapshot of a rate group or the daily history of a single rate type, and set how many recent observations to pull in history mode. The Input tab lists every parameter.

A first run with the defaults:

{
 "historyDays": 30,
 "maxItems": 10
}

A larger pull:

{
 "historyDays": 30,
 "maxItems": 200
}

Pricing

Pay-per-result: $0.011 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$1.10
1,000 results	$11.00
10,000 results	$110.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the NY Fed Reference Rates Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to NY Fed through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/nyfed-reference-rates-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that you have not set both a rate group and a single rate type. When a single rate type is selected, it overrides the rate group and enters history mode. If the NY Fed feed is temporarily unavailable, wait a few minutes and retry.

Why do some observations have missing percentile values?

The NY Fed does not publish percentile data for every rate type or every observation date. EFFR and OBFR typically include percentiles, while SOFR may not for all dates. Missing values appear as empty fields in your dataset, which is expected behavior.

I requested 5,000 days of history but got fewer rows.

The actor returns only the observations the NY Fed has published. If a rate was introduced less than 5,000 days ago, you will receive all available history up to that date. SOFR, for example, began publication in April 2018, so you will get roughly 1,800 to 2,000 daily observations depending on the current date.

The run failed with a timeout error.

Requesting the maximum 5,000 days of history for a single rate may take longer than the default timeout. Increase the run timeout in the actor's advanced settings, or reduce the history length to a smaller number of days.

Can I get intraday or real-time rate updates?

No. The NY Fed publishes these reference rates once per business day, typically around 8:00 AM Eastern Time. The actor returns the most recent daily published value. For intraday SOFR, you would need a different data source.

FAQ

Question	Answer
What reference rates does this actor collect?	It collects the Secured Overnight Financing Rate (SOFR), Effective Federal Funds Rate (EFFR), Overnight Bank Funding Rate (OBFR), Tri-Party General Collateral Rate (TGCR), and Broad General Collateral Rate (BGCR), plus the SOFR Averages and Index when pulling all rates.
Does this actor require an API key or NY Fed account?	No. It reads the same public data feeds that power the NY Fed's own rate display pages, so no registration, API key, or authentication is needed.
How far back can I pull daily rate history?	You can request up to 5,000 of the most recent daily observations for a single rate type. The actual available history depends on when the NY Fed began publishing that rate, but the actor will return whatever is available up to your requested limit.
Can I get the latest value for all rates in one run?	Yes. Set the rate group to 'All reference rates' and leave the single rate type empty. The actor will return the most recent published observation for SOFR, EFFR, OBFR, TGCR, BGCR, and the SOFR Averages and Index in one dataset.
What percentile data is included with each rate?	When the NY Fed publishes them, each observation includes the 1st, 25th, 75th, and 99th percentile values. Not all rate types have percentile data for every observation, but the fields are present in the output schema.
How do I switch between latest snapshot and history mode?	Leave the 'Single rate type' field empty to use latest snapshot mode and choose a rate group. Select a specific rate type like SOFR or EFFR to enter history mode, which will pull the number of daily observations you set in the history length field.
What export formats are supported?	Your dataset can be exported to CSV, JSON, Excel, or XML from the Apify platform after the run completes.
Does this actor scrape the NY Fed website HTML?	No. It reads the structured data feeds the NY Fed website itself consumes, so it is faster and more reliable than HTML scraping and will not break when the page layout changes.
Can I schedule this actor to run daily?	Yes. You can set up an Apify schedule to run the actor every morning in latest mode and push the results to a database, spreadsheet, or notification system using Apify integrations.
What is the difference between the secured and unsecured rate groups?	The secured group returns SOFR, TGCR, and BGCR, which are rates on loans collateralized by Treasury securities. The unsecured group returns EFFR and OBFR, which are rates on uncollateralized federal funds and bank funding transactions.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Federal Reserve Bank of New York. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.