NYT News Search Scraper

The NYT News Search Scraper is a tool that allows you to search for news articles on The New York Times website and retrieve specific information from the search results. With this scraper, you can easily access details such as article titles, publication dates, URLs, categories, descriptions, authors, article image andd its text. It's a handy tool for researchers, journalists, or anyone looking to gather information from The New York Times.

Features

Search The New York Times for articles based on your query. Retrieve article titles, publication dates, URLs, categories, descriptions, authors, image URL and article text. Option to save retrieved data to a file for further analysis. User-friendly and easy to use.

Input Options

Search phrase

The search phrase is the keyword or phrase you want to search for on The New York Times website. For example, if you want to search for articles about the COVID-19 pandemic, you can enter "COVID-19" or "COVID-19 pandemic" as your search phrase.

Sort type

The sort type is the order in which you want the search results to be displayed. You can choose to sort the results by relevance, newest, or oldest. The default sort type is newest.

Maximum number of news to scrape

The maximum number of news to scrape is the maximum number of articles you want to retrieve from the search results. The default maximum number of news to scrape is 10.

Sample Data

Here's an example of the type of data you can retrieve using the NYT News Search Scraper:

1{
2	"title": "Why Are So Many Players Getting Sick at the U.S. Open?",
3	"date": "2023-09-01T16:20:17-04:00",
4	"url": "https://www.nytimes.com/2023/09/01/sports/tennis/us-open-players-sick.html?searchResultPosition=3",
5	"category": "TENNIS",
6	"description": "Ons Jabeur has won two rounds despite having flu symptoms, but Dominic Thiem was forced to retire from his second-round match with an illness.",
7	"author": "By Jesus Jiménez",
8	"imageUrl": "https://static01.nyt.com/images/2023/09/01/multimedia/usopen-sick2-hlgj/usopen-sick2-hlgj-threeByTwoSmallAt2X.jpg?quality=75&auto=webp&disable=upscale",
9	"articleText": "Early in the second set of her second-round match on Thursday night, a ball bounced just past Ons Jabeur’s reach, and she lost the point, throwing her arms up in..."
10}