Tumblr Scraper — Blogs, Tags & Search

Scrape Tumblr posts from any public blog, tag or search query. No API key, no OAuth, no login, no browser — plain HTTP only, so runs are cheap and fast.

Every mode produces the same normalized row shape, so you can mix blog, tag and search runs into one dataset without reshaping anything.

---

What you get

Column	Notes
postId, postUrl	IDs are strings — Tumblr post IDs exceed 2^53
blogName, blogTitle, blogAvatarUrl	Largest available avatar
postType	text / photo / video / audio / quote / link / chat / answer — detected from the actual media, see Post types
date, timestamp	ISO 8601 UTC and Unix seconds
tags[], summary	summary is the title or body text, HTML stripped, capped at 500 chars
noteCount	All modes
likeCount, reblogCount, replyCount	Tag and search modes only — the legacy blog API does not expose these
media[]	{type, url, width?, height?, alt?, thumbnail?}, full resolution by default
videoPosterUrl	Poster frame for video posts
isNsfw, classification	Tag and search modes only
reblogFrom	{blogName, blogTitle, postUrl, rootBlogName, rootPostUrl} when the post is a reblog
notes[]	Only when includeNotes is on — see Notes
mediaFiles[]	Only when downloadMedia is on
_source	Which mode produced the row
raw	The untouched Tumblr payload (disable with includeRaw: false)

---

Input

Field	Type	Default	Description
mode	blog | tag | search	blog	Which source to scrape
blogNames	string	—	Blog mode. Comma-separated. Accepts staff, staff.tumblr.com, a custom domain, or a full URL
tags	string	—	Tag mode. Comma-separated. A leading # and /tagged/ URLs are accepted
searchQueries	string	—	Search mode. Comma-separated
postTypes	string[]	[] (all)	Keep only these types
maxPosts	integer	50	Total across all targets, deduplicated by post ID
includeNotes	boolean	false	Request note details and emit a notes column
downloadMedia	boolean	false	Save media into the key-value store
fullResMedia	boolean	true	Resolve the largest rendition instead of a thumbnail
includeRaw	boolean	true	Keep the raw column
requestDelayMs	integer	1100	Hard floor between requests (min 500)
proxyConfiguration	object	Apify Proxy on	See Proxies

Example — a blog's photo posts at full resolution

{
  "mode": "blog",
  "blogNames": "staff, engineering",
  "postTypes": ["photo"],
  "maxPosts": 200,
  "fullResMedia": true
}

Example — a tag, with media downloaded

{
  "mode": "tag",
  "tags": "photography",
  "downloadMedia": true,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyCountry": "US" }
}

---

How each mode works

blog — full archive access

Uses Tumblr's legacy read API (https://{blog}.tumblr.com/api/read/json), which still works without any credentials. Paginates with the start offset against the authoritative posts-total, so it can walk an entire archive.

This is the only mode with unlimited pagination depth.

tag / search — first page guaranteed, deeper needs a non-EU IP

https://www.tumblr.com/tagged/{tag} and https://www.tumblr.com/search/{query} are server-rendered: the complete timeline JSON is embedded in the page, so no browser is needed. One page yields roughly 7 posts per tag and 14 per query (the raw timeline carries ~11 and ~22 entries, the rest being ads, section titles and carousels — all filtered out).

Going deeper follows the timeline's nextLink (/v2/hubs/… or /v2/timeline/search). Those /v2/ paths redirect to Tumblr's EU consent wall from an EU exit IP — verified live. The SSR page itself serves full data even when Tumblr flags the request as EU (gdprIsEu: true); only the JSON API is gated. The classic /tagged/{tag}/page/2 fallback is also gone (HTTP 404), so there is no SSR-side workaround.

What this means for you: run tag and search modes through a non-EU proxy (apifyProxyCountry: "US"). If the actor hits the wall it logs a clear warning, keeps the posts it already has, and reports stoppedBecause: "consent-wall" in RUN_SUMMARY rather than failing.

---

Behaviors worth knowing

Post types are sniffed, not trusted

The legacy API's type= filter is unreliable, and its type field is not much better. Verified live on staff.tumblr.com: requesting type=photo or type=video returns posts that all declare "type": "regular", with the media inline in the post body HTML and no photo-url-* fields at all.

So this actor never sends a type= filter. Instead it detects media by parsing the post body (NPF data-npf payloads, <img srcset>, <video>/<source>) and NPF content blocks, then classifies from what it actually found.

A post can match more than one type. A regular post carrying inline photos matches both text and photo, so postTypes: ["photo"] finds it — while postType reports the most specific medium present (photo). Filtering is deliberately inclusive; the column is deliberately specific.

Full-resolution media costs no extra requests (usually)

fullResMedia: true reads the largest rendition straight out of the markup already in hand:

1. <figure data-npf='…'> — exact media URL, dimensions and poster (videos)
2. <img srcset> — the widest candidate; Tumblr's w descriptor caps at the original width
3. NPF content[] blocks — the variant flagged hasOriginalDimensions

Only if a post yields nothing better than a thumbnail (e.g. an old photo-url-* post) does the actor fetch that post's page, capped at 25 posts per run. If that parse fails, the thumbnail URLs are kept — you never lose a row over it.

Notes are counts only

noteCount is always populated. Individual note bodies are not available: verified live, notes_info=true on the keyless legacy endpoint returns only note-count, no notes array. With includeNotes: true the notes column is emitted as an explicit null so you can tell "asked, unavailable" from "never asked". The parsing is in place, so if Tumblr ever returns notes they will populate automatically.

Rate limiting

Tumblr's per-IP limit is aggressive — bursts of 3–6 rapid requests start returning 429, and without a browser User-Agent the legacy API returns 429 immediately. The client therefore:

• always sends a browser User-Agent (mandatory, not cosmetic);
• enforces a hard floor of requestDelayMs (default 1100 ms ≈ 1 req/s) between all requests, serialising concurrent callers so no burst can slip through;
• retries 429s with exponential backoff from 2 s (+jitter, 60 s ceiling), honouring Retry-After exactly when present;
• detects the rate-limit page even when it arrives with a 200 status.

Throttling degrades gracefully: posts already collected are kept and pushed.

Deduplication

Rows are deduplicated by postId across pages, targets and modes — overlapping tags and queries genuinely return the same posts, and you are never charged twice for one post.

Media download

downloadMedia: true streams each file into the run's key-value store and adds mediaFiles[] to the row with the store key and a public record URL. Files over 50 MB are skipped, as are third-party iframe embeds (they are player pages, not media). Failures are recorded per entry in mediaFiles[].skipped instead of failing the row.

---

Proxies

• Blog mode: datacenter proxy (the default) is fine.
• Tag / search mode: set a non-EU country, e.g. {"useApifyProxy": true, "apifyProxyCountry": "US"}, or pagination stops after the first page.
• Repeated 429s: switch to RESIDENTIAL. The limit is per IP, so rotating residential IPs is the effective fix.
• Running with no proxy at all works for small runs but rate-limits quickly; the actor warns when it detects this.

---

Pricing (pay per event)

Event	When
apify-actor-start	Once per run
page-scraped	Per post pushed to the dataset (deduplicated)
media-downloaded	Per media file actually stored — only with downloadMedia

Charges never interrupt a run: a failing charge is logged as a warning and the run continues.

---

Limits

• Tag and search depth is capped at one page from an EU exit IP (see above). Use a US proxy.
• Note bodies are not retrievable without authentication.
• likeCount / reblogCount / replyCount / isNsfw / classification are null in blog mode — the legacy API does not carry them.
• Private, password-protected and deleted blogs are not accessible; blog mode reports HTTP 404 and moves on.
• Full-resolution post-page upgrades are capped at 25 posts per run to bound request count.

Every run writes a RUN_SUMMARY record to the key-value store with per-target page counts, stop reasons, dedupe/filter tallies, request counts and rate-limit hits — start there when a run returns less than you expected.

---

Development

npm install
npm test          # 128 tests, node --test, fixtures only — no network

Tests run entirely against captured fixtures in fixtures/, including a real 429 rate-limit page, and assert byte-exact parses (item counts, first post IDs, resolved media URLs).

Path	Purpose
src/main.js	Input handling, proxy setup, orchestration, media download, PPE charging
src/lib/jsonp.js	Legacy JSONP unwrapping, rate-limit detection, read-API URLs
src/lib/ssr.js	SSR state-blob location and parsing, timeline extraction, nextLink, consent-wall detection
src/lib/media.js	Media sniffing (NPF, srcset, video) and post-type classification
src/lib/normalize.js	One row shape across kebab-case, camelCase and snake_case sources
src/lib/http.js	curl transport, pacing, backoff, media download
src/lib/modes.js	Per-mode pagination loops and the full-res upgrade pass
src/lib/collector.js	Dedupe, type filter, maxPosts cap