A Secret Web
The web is mind-bogglingly huge; let's look at how personal websites can thrive and interact despite that.
Building Clew
behind the scenes of independent search
Latest posts
The New Ariadne Architecture
While on a fourteen-hour international flight, I finally managed to come up with an architecture for Clew's web crawler that I'm happy with. Here's the run-down.
Redesigning the Index
I believe I've reached a point in Clew's development where, armed with the knowledge I've acquired from months of crawling sites and using that data to search the index, it's time to wipe the index and start over.
I'm Losing Faith in BM25
The current way that result ranking works in Clew is very different from what I want.
Welcome to the Madness
In which we launch the insanity that is this development blog for Clew.