Welcome to the Jungle Articles and Full Text

The whole Welcome to the Jungle media archive with the article text, 8,584 articles across five language editions. No key and no login.

The body arrives in pieces

The article text is not in the site's search index. It comes from the article page, and it arrives as a list of content blocks rather than one field.

Reading the first block and calling it the article is the mistake to avoid. On our test run 31 of 60 articles had more than one block, and taking only the first kept 43.8 percent of their text. The worst one kept 19 percent. Nothing about that looks broken. The word count is just smaller, and still believable.

This Actor joins every block, strips the HTML and reports bodyBlockCount so you can see how many there were.

Dates are not searchable at the source

published_at is a string in the index. Ask Algolia for published_at > 0 and it answers HTTP 200 with zero results and no error, which looks exactly like an archive with nothing in it. So publishedFrom and publishedTo are applied here instead, against the date the search already returns, and applied before the article page is fetched. An article outside your window costs no request and is never charged. Our test run dropped 1,251 that way.

What one row is

One article. The five language editions are written separately rather than translated, so an English and a French piece on the same subject are two articles with two ids. 1,000 sampled articles gave 1,000 ids, 1,000 references and 1,000 slugs.

categories is a list because an article is filed under several at once. 8,584 articles carry 10,433 category entries between them.

Two fields worth knowing about

contributors is a newer field. It is on 213 of 400 records we sampled and on none of the 2019 and 2020 articles, so an empty list on an old article means the site records none.

format is only on the article page, so filtering on it means the page has to be read first. That spends the row budget before the filter runs and your answer can come back short. It says so in limitedBy, and you are not charged for rows it drops.

What a run admits

Every row carries searchTotalReported, the count the site's own search gave for your filter, in articles, the same unit as the rows. RUN_SUMMARY in the key-value store holds every cell with what it reported and why it stopped, plus complete, limitedBy, droppedByDateWindow and droppedByFormatAfterReading.

Billing

Two events: a start fee charged only once articles are returned, and a per-article fee charged after each one is written. Reading the full body costs no extra. A run that matches nothing costs nothing.