Recent content in Amith Mathew on Code and Contemplation
I’ve seen all kinds of enterprises, analytical data platforms and data pipelines over the last 18 years or so. I thought it might be an interesting exercise to write down some lessons that I’ve learnt as well as some thoughts about where we’re headed. Some (most?) of this is common sense, but common sense apparently takes a vacation when deadlines loom and everything is on fire.…
Ever been to a friend’s house where they’ve just done up the decor - it’s cohesive, it’s chic, modern, clean, you love it. And then you visit a few month’s later - and the decor’s still the same, but you can tell life has happened. The white is a little scuffed, the plastic is ripped, the lighting throws chipped edges into sharp relief. You can still see what it was supposed to be, and the feeling…
I was talking to a colleague about a RAG system he’s evaluating and he mentioned that the architecture includes a reranking model. This sent me down a bit of a rabbit hole about rerankers and why reranking models make sense within the RAG architecture. A typical RAG workflow looks like this. flowchart TD id1(Document Chunk) id2[[Embedding Model]] id3[( Vector DB )] id5(Query) id6[[Embedding…
These are exciting times we’re living in - AGI is around the corner, with massive implications for the area that I work in, data engineering, among others. I expect the way we developers interact with our computers will become multi-modal as these AI models mature. Apple dipping their toes in VR with the Apple Vision Pro is another thread in what may become this majestic tapestry (the happy…