Training data
RSS Amplifier crawls 534,522 independent feeds — blogs, podcasts and video channels that publish on their own sites rather than inside a platform. The whole directory is available as a licensed corpus, sliced on a 4-hour clock and streamed as gzipped NDJSON, so a training pipeline can pull the new rows every 4 hours and keep a mirror current indefinitely.
What is in it
- Feeds
- 534,522
- Post records
- 4,770,012
- Full-text articles
- 275,735
- Authors
- 195,456
- Feeds re-read in the last day
- 79,114
Read live from the directory as this page loaded — the same numbers the crawler status page reports, from the same cache. Nothing on this page is estimated or rounded up.
Post records and article text are different things
This is the one thing worth reading twice before you talk to us, because it is the thing most likely to be assumed wrongly.
The post records are the large dataset: title, summary, author, canonical URL, publication date and the feed each post belongs to, 4,770,012 of them, growing by hundreds of thousands a day. That is metadata at scale, and for a great many uses — link graphs, recency signals, topic and language distribution, retrieval indexes — it is the part that matters.
The article text is the smaller one: 275,735 articles, sanitized, averaging 7,929 characters (sampled over 2,000 of them, not summed over all — the exact figure is a quarter of a million row lookups this database should not be asked for on a page load). It exists because an article is fetched and cached when a reader opens the post, so it grows with attention rather than with the crawl. Posts ingested before August 2026 also carry the body their feed published; newer ones do not, because storing one for every post was ten gigabytes of a fourteen gigabyte database.
If you need prose at the scale of the metadata, say so — that is a change to what the crawler stores rather than a parameter you can pass, and it is a conversation we are happy to have.
How it is delivered
Four streams — feeds, items, extracts and authors — each one gzipped NDJSON, one JSON object per line, streamed rather than downloaded from a prepared file. The full machine-readable description lives at /api/dataset and needs no account, so you can read the exact shape of every row before you talk to anybody.
Every slice is a half-open range on a fixed 4-hour boundary, currently 2026-08-29T20:00:00.000Z. That matters more than it sounds: a window is the same set of rows whoever asks and whenever they ask, so a pipeline that walks boundaries in order provably sees every row exactly once — no gaps from clock skew, no duplicates from a retry, and a failed pull can simply be repeated. Only closed windows are served, because the one containing the present is still filling.
Authenticate with a session or an API key as a bearer token. The licence belongs to the account rather than to the key, so keys can be rotated without telling us. A full-history pull is available too, separately metered — intended once, to seed a mirror, before switching to windows for good.
Where it comes from
Public feeds their publishers chose to syndicate. Anyone may submit a feed, every feed has its own page here, and the directory has been open and free to read since the day it launched — the JSON API, the OPML export, llms.txt and the MCP server all still answer without an account, and nothing on this page changes that. What is licensed here is bulk access, which is a different artifact and a real cost to serve.
A publisher may be excluded from the corpus while staying in the directory, by writing to hello@rssamplifier.com. It is a separate ask from removal on purpose — “list my blog, but do not sell my writing to a model” is a coherent position and nobody should have to leave the directory to hold it. Excluded feeds, and every post and article belonging to them, are absent from every stream.
Talk to us
Licensing is per buyer, so there is no price list here: what you may keep, whether you may redistribute it and whether attribution travels with the text all move the number more than volume does. Tell us what you want it for and we will come back with terms.
Prefer email? hello@rssamplifier.com. For anything that is not about the corpus, the contact page routes to the right mailbox.