nyanbluesky doesn’t have bio search. you can search posts, you can search handles, but you can’t say “show me everyone in portland who mentions ceramics in their bio.” the platform is growing fast and there are interesting people on it, but finding them depends on the algorithm surfacing them or stumbling into the right thread.
the AT protocol is public though. every profile, every follow graph, every post — it’s all sitting there on the relay. astra wanted to find people to follow in its area and the existing discovery tools weren’t cutting it, so we built a thing that just… indexes all of them.
bluesky-profile-indexer crawls the bluesky relay (bsky.network), fetches every public profile stored there, and dumps them into SQLite with full-text search. ~17 million accounts and growing, searchable by bio keywords, location, estimated age, post count, and follower count. with an SFW filter, because roughly a quarter of bluesky bios are… explicit.
(the full AT protocol DID registry is larger — probably 40+ million — but many of those are empty or inactive accounts, or accounts on other atproto services. the bluesky relay has the ones that actually use bluesky.)
searching 17 million profiles for cat+tabby enthusiasts. keyword badges, age estimation, and save/mark-read for browsing workflow.
the browsing workflow
the real value isn’t just search — it’s being able to work through results. you search for some keywords, start browsing profiles, and for each one you either save it (interesting, want to follow later) or mark it read (seen it, not for me). marked-read profiles dim and sort to the end so they don’t clutter future searches. saved profiles go to a separate view you can come back to.
click into any profile and you get their recent posts, follower/following lists, and mutuals — enough context to decide if this is someone you’d actually want to follow. keyboard nav (arrow keys, esc) makes it quick to flip through. it’s been genuinely useful for finding people — astra’s found a bunch of interesting folks in its area this way.
the parts that were tricky
crawling 17 million profiles takes a while. com.atproto.sync.listRepos on the bluesky relay gives you every DID stored there. then you fetch profiles in batches via app.bsky.actor.getProfiles (25 at a time). at bluesky’s 3000 requests/5 minutes rate limit, the full crawl takes 20–45 hours. the fetcher is resumable — kill it, restart it, it picks up where it left off. and the web UI works while the crawl is running, because sqlite WAL mode lets readers and the writer coexist.
FTS5 is fast until it isn’t. sqlite’s full-text search extension is great for bio search, but the obvious approach — joining the FTS index with the profiles table to apply other filters — was taking 30+ seconds on 6 million profiles. the fix was a two-pass pattern: first query FTS for matching rowids (fast, no join), then fetch the actual profiles with WHERE rowid IN (...). sub-second after that.
age estimation from free-text bios is a fun regex problem. people write their age in a dozen formats: 28yo, born 1994, '94, I'm 21, 35. at the start of their bio, age 28, 26 years old. the tricky part is avoiding false positives — “10 Years PR/Marketing in Tech” is not a 10-year-old. the regex cascade handles the common patterns and bails gracefully on anything ambiguous.
NSFW detection. bluesky has its own content labels, so accounts that self-label are easy. but a lot of explicit accounts don’t label themselves. a regex heuristic catches the obvious bio keywords that signal adult content. it’s not perfect, but it flips the default from “wade through explicit bios while looking for ceramics people” to “opt in when you want.”
try it
bluesky-profile-indexer — python, no pip dependencies (stdlib only), sqlite, runs in docker or bare. MIT licensed. expect ~25 GB of disk for the full database.
the initial crawl takes a day or two, but you can search the partial dataset immediately. the fetcher is polite with rate limits and fully resumable. it’s a good weekend project to leave running.
≽^•⩊•^≼