Recent content in Home on Arnab Roy
I just finished my very first Honour mode run in Baldur’s Gate 3. I have been playing this game for the last 2 months like a crazy person. I have enjoyed the game so much, I wanted to try the hardest mode (Honour) where you get one chance and if your party dies, then it’s game over. There’s no do-over, no reloading from a previous save point. It was a very stressful and…
In my previous article, I tried to compile the theories around LLM and agent evaluations. It gave me a good understanding of the process but there were still a few open questions and/or concerns around what it looks like in practice. So as a follow up, I wanted to build a minimal reference implementation of an end to end eval process to understand the core parts of evals a bit more in depth.
I have been reading a ton about LLM evaluation practices over the past few weeks from Anthropic’s engineering blog, Hamel Husain’s practitioner-focused guides, the Evals for AI Engineers book by Shreya Shankar and Hamel Husain, and several eval framework docs. I wanted to write what I learned and have a synthesis of my understanding of the topic. This is in no way comprehensive, rather…
I have been meaning to clean up my dotfiles for a while. Lately I have noticed that Neovim felt sluggish on first open and it was really annoying me. Then one weekend I actually measured it: 275ms to start Neovim. For reference, a clean Neovim starts in about ~32ms. That is an ~8.5x overhead, and it’s entirely self-inflicted. I have been adding stuff left and right without thinking about…
I recently worked on adding outlier detection in a data pipeline. As a person primarily focused on backend and infra work, I thought this would be a good opportunity to brush up my statistics knowledge and learn some sophisticated outlier detection algorithms. Even though I did learn a few things along that line, I ended up learning something far more valuable. This article aims to compile my…
While writing the LSP article, I noticed that smaller models continuously failed to output in the expected structure. It was especially prone to producing malformed JSON that caused parsing errors and downstream failures. I wondered, how people deal with this kind of situation where we expect a specific structured output from LLMs, as I suspect this is a very common scenario and there must be a…
What is LSP? Language Server Protocol (LSP) is what powers code intelligence in modern editors/IDEs. So things like auto completion, go to definition/references, or showing diagnostics and lint errors are all delivered within your favourite editor through LSP. But let’s start with the problem first, so that you can understand the painpoints before LSP and how it solves that problem in an…
Last updated: February 2026 This page lists the tools and software I use for development and productivity. Terminal & Shell Terminal Emulator: Alacritty Shell: Zsh + Sheldon + zsh-defer Prompt: Starship Multiplexer: Tmux + tmuxinator Editor Editor: Neovim Plugin Manager: lazy.nvim Core Plugins: Fuzzy finder/picker/explorer - snacks.nvim Completion - blink.cmp Syntax highlighting and text objects -…
I’ve been writing code professionally for 7+ years, building backend systems and data infrastructure. Recently, I moved into AI platform and infrastructure. This includes building evaluation pipelines, observability and monitoring, MCP servers with proper auth/authz and data governance. Before this, I built data pipelines, observability stacks, and developer tooling across backend and data…
2025 has been one of the most active years of my life. As I was reviewing my Strava logs I noticed that I have run 325km and cycled 802km in 2025. But more importantly I have learned a lot about training, managing injuries, sustainable progressions and overall training efficacies. This article aims to document my experience and learnings. Warning I don’t claim to know all the details or the…