Why metrics? Suppose a team says: We will write good documentation. The statement describes the work the team plans to do: the implementation. The team will write, review, and publish documents. Completing that work does not tell you whether the documents helped anyone. A metric gives the implementation a test. If the documents work, readers should need less help with the tasks they explain. You…
Imagine that your team runs a customer-support assistant built with a large language model, or LLM. A vendor releases a model that costs less and answers faster. You change one setting and the assistant still works. Ten trial questions produce sensible answers. The team still lacks enough evidence to ship the change. The team needs to check more than whether the new answers sound good. The…
I wrote the first version of this guide for a room of backend engineers. Most of them had called an LLM API. They could ship a prompt, parse the response, and add retries. The API still felt arbitrary. Why does the provider charge different prices for input and output tokens? Why does caching help so much? Why can a batch job cost less than the same synchronous request? Why do reasoning requests…
I’m sharing these ideas as concepts I’d be excited to see programming languages explore. They represent different ways to push implementation details into the language or runtime, allowing developers to focus more on intent rather than mechanics. 1. Incrementality The Problem When a small input to a large function changes, we typically recompute everything. This is inefficient and wastes…
This is one of those posts where I explore what happens when we map concepts from one field to another. Today’s question: What if deep learning is fundamentally like farming? Building vs Growing: Two Ways to Create Two ways to make things There are two fundamentally different ways to make things: Engineering: You understand how components work and compose them deliberately - like building a table…
The Many Universes That Exist I want to share a way of thinking that I found personally useful. It helps me reason about what exists today, and make predictions. I hope that by the end of this post, you’ll see some problems differently—recognizing that when something seems impossible, we’re often blind to which constraint is actually limiting us and how that constraint might be shifted. I’ve been…
Sometimes, insight comes from unexpected places. Today, it came from the pool. Today while swimming, I had a realization: there’s a lot in common between swimming well and running a successful subscription business. The Basics: Drag, Thrust, and Speed When you first start swimming, you think it’s all about moving your hands and legs as powerfully as possible. More force equals more speed, right?…
A Quick Intro to S-curves Today, we’re diving headfirst into the wacky world of S-curves – the Kardashians of the math world. They’re everywhere, they’re unpredictable, and they’ve got more curves than a bag of curly fries. An S-curve is a mathematical tool that helps us understand systems showing exponential growth, which eventually stabilize at a fixed level. This concept can be applied to…
Books Trustworthy Online Controlled Experiments Resources Top Challenges from the first Practical Online Controlled Experiments Summit A/B Testing Pitfalls: Getting Numbers You Can Trust is Hard USF Business Analytics Forum - Ron Kohavi A/B Testing at Scale: Accelerating Software Innovation Trustworthy Online Controlled Experiments at Large Scale Always Valid Inference: Continuous Monitoring of…
A website which allows students of IIT Indore to add courses they are interested in to their calendar. I was tired of doing this manually every time. Use it here
Pottermania 2018 The goal of the is to be something easy that anybody with little experience in quizzing can take part in. created with Bitan, and Keyur as part of Quiz club. Questions Get the questions. Get the video for question number 14 Get the Music for question number 20 Answers Get the answers.
Audio books are seeing a resurgence, because of companies like Audible. Text to speech is good, and is getting better . Wireless Bluetooth earphones are becoming common place, the increase in convenience and battery life allows people to have them on longer. Unlike content that you have to watch/read, you can work on other stuff while you listen. I see a time where we can listen to most of the…
Co-opeartive and Adversarial Environments This is a mini interactive “game”, I made inspired by the paper, without giving too much away, I urge you to play . AI Safety Grid Worlds ,
I thought about this after reading Doing good better . Soon after I found out about kepler.gl which made this the perfect project to test it out. I extracted the data I needed for my analysis from a PDF of an annual summary report by the Government of India, available here , as I couldn’t find a comprehensive and user-friendly source. You can read the analysis here . Here is the kepler.gl map I…
Jargonizer What if instead of aiming for clarity and conciesness, our goal was to make our sentences as unweildy and hard to understand as possible? A simple way to do this is to replace easy to understand phrases with harder ones. Wikipedia maintains 1 one such list for us. Here is the website I made so you can play with it online as well, it all runs in your browser, no text is sent to a server.…
This post will talk about resources for how I’m going about learning machine learning. Books “Hands-On Machine Learning with Scikit-Learn and TensorFlow” by Aurélien Géron “Deep Learning” by Ian Goodfellow, Yoshua Bengio, and Aaron Courville “The Elements of Statistical Learning” by Trevor Hastie, Robert Tibshirani, and Jerome Friedman Blogs Distill.pub : Clear, interactive explanations of machine…
Game Of Thrones Quiz This quiz was created as part of cultural week, right as Game of Thrones season 7 ended. It is broken into 3 seperate rounds first round, acts as elimination to reduce the number of teams that play. second round where each team sees one question at a time and can write their answers, and the quiz master checks if they got it right and awards +1 for correct answers. the correct…
Newbie Quiz Indraneel and I had prepared this quiz as a way of introducting quizzing to new people and narrowing down potential recruits. The quiz covers a broad range of topics, from pop culture, sports, technology, politics Lots of fun! Questions Get the questions. Answers Get the answers.
Some quizes I made so it’s easy to keep track of mental health. Burn’s depression Checklist use it here Novaco’s Anger Scale use it here