Recent content on Akshay Deshpande
The previous write-up ended with the output from the embedding lookup (1 and 2 below). The previous write-up showed that the output from the embedding layer gives the numerical representation for the token IDs, but these embeddings themselves are not yet contextual. Details are in this section of the previous write-up:…
When a new model is announced, we often see a description like this: Llama 2 7B: 4096 dimensions, 32 attention heads, 32 Transformer blocks What do those numbers mean? And when you send a prompt like this: The cat sat on the mat what actually happens under the hood? Those questions sent me down the rabbit hole of trying to understand the internals of large language models. This post is my attempt…
In the previous post, Two Layers, One Mental Model: How I Stopped Fighting My AI Tools , I wrote about a simple way to reason about AI tooling: Context — what the agent knows when it wakes up. Capability — what the agent can do. Hybrid constructs — reusable units that combine procedural knowledge with invocation. That post was mostly about vocabulary. I wanted a mental model that made constructs…
The mess that triggered this A few months ago, my AI tool usage looked like most engineers’ AI usage. I was exploring several AI coding assistants — IDE-integrated ones, terminal agents, chat-based models — often more than one at a time. There was no system, no structure, no baseline. Which meant five things, all bad: Every session started from zero. I re-explained the project to the AI on…
This writeup talks about different metric type that an application can emit as telemetry. It intends to cover the case of when to use which metric types and the usecase for them. Metrics is one of the oldest forms of telemetry. There are many APM solutions, whose billing model runs on the number of metrics you send to them. But it is often not clear to app devs on what kind of metrics to emit from…
This note is a mental model for how Prometheus discovers and scrapes metrics in Kubernetes . The lens I want to keep throughout is: Where will the scrape config file sit? (Prometheus repo vs application repo) In which namespace will the serviceMonitor sit? (and how Prometheus finds it) At a high level there are two ways to tell Prometheus about a /metrics endpoint: Static via in the Prometheus…
In most of distributed datastore systems, there are a lot of techical terms to describe the behavior of the system. While these terms, like, “Leader”, “Follower”, “Replication”, “Consistency”, etc., are widely used and helpful, what I feel missing are the details about internal relationship between these terms. Analogically, while the map of the…
When studying Chapter 4 of Designing Data-Intensive Applications (Encoding and Evolution), I quickly encounters a level of granularity that seems mechanical : binary formats, schema evolution, and serialization techniques. Yet behind this technical scaffolding lies something conceptually deeper. Encoding is not merely a process of serialization; it is the very grammar through which distributed…
I attended Kubecon India 2025 held in Hyderabad this year. I mainly focused and attended talks related to Observability and Scalable designs. Now that the sessions are uploaded to Youtube, linking the ones which I really enjoyed. Observability at Scale With Monitoring as Code: Grafana, Prometheus, & Tempo - Vipin GopalaKrishnapillai & Saiabhinay Bommakanti, Amway Global - link Predictable auto…
I enjoy philosophy. Stoic philosophy in particular. Philosophy, I think, helps us revalidate our purpose. It acts as a yard stick and makes sure that we are not moving away from our First-Principles. Applying the same to Software Engineering, in my opinion, every team should have a “Design Philosophy”. What is that one yard stick which teams can use for making better decisions. Infact,…
The intention of this page is to put together the Observability Signal Guidelines which will provide the required visibility into the systems without hurting the cost aspect of the solution. Three basic observability signals that any application emits are: Metrics, Traces and Logs The general question is - When to emit what signal? The answer lies in the intent behind the signal being emitted.…
I recently did a brush up course on Networking/Protocols 101’s. Making my notes public. Networking Basics / Protocols: Mental model to think about all protocols: A few notes about different Protocols: Network layer protocol: Transport layer protocol: TCP: UDP: Other Protocols: HTTP: SMTP: XMPP: MQTT: AWS Networking: Local Zone: Edge location: Understanding IPv4, IPv6 IPv4 IPv6 Classes IPv4…
Observability is crucial for modern distributed systems, enabling engineers to monitor, debug, and optimize their applications effectively. OpenTelemetry (Otel) has emerged as a comprehensive, vendor-neutral observability framework for collecting, processing, and exporting telemetry data such as traces, metrics, and logs. This blog post will explore how custom processors in OpenTelemetry can…
This page documents a few aspects of memory management on Java containers on K8s clusters. For java containers, memory management on K8s have various factors: Xmx and Xms limits managed by java Request/limit values for the container HPA policies used for scaling the number of pods Misconfigurations / misunderstanding of any of these parameters leads to OOMs of java containers on K8s clusters.…
There isn’t a single book or article on observability (O11y) where there isn’t a mention of MELT. (Metrics, Events, Traces, Logs) While these four are the building blocks of telemetry data in Observability, all the four components haven’t evolved at the same rate. In this write-up, I delve into metrics in observability (mainly custom metrics) and argue how the overuse of metrics…
Books that have shaped how I think about technology, systems, philosophy, and life. Shop Class as Soulcraft Matthew B. Crawford “Craftsmanship means dwelling on a task for a long time and going deeply into it, because you want to get it right.” The Double Helix James Watson “Discovery is usually an act of synthesis, not accumulation.” The Demon-Haunted World Carl Sagan A passionate…
Talks and presentations I have given at conferences, meetups, and internal events. Mostly around observability, distributed systems, cost awareness, and managing large-scale datastores. AI in the Space of Observability May 2026 — Bangalore SRE Meetup Going into the details of AI constructs like skills, workflows, and workpacks. Exploring and building skills for observability use cases with a demo.…
In distributed systems, the principle of message passing between nodes is a core concept. But this leads to an inevitable question: How can we ensure that a message was successfully delivered to its destination? To address this, there are three types of delivery semantics commonly employed: • At Most Once • At Least Once • Exactly Once Each of these offers different guarantees and trade-offs when…
This post is a geek-out journey this festive season on finding the right tool for my personal finance management. Where it all started: Recently, I spoke at the Smarsh Tech Summit on “Cost as an Architectural Pillar,” where I emphasized the importance of considering cost as a first-class citizen in the software development cycle. However, on a similar note, when I was looking through…
This is a brain dump on how taking notes and Obsidian as a tool has helped me. Knowledge management? As one progresses further into career, Knowledge management becomes as equally important as Finance management . Knowledge accumulation is a non-linear trajectory. Majority of the times it is compounding in nature. If one doesn’t organise it, you are always at the mercy of “ I had…
This writeup is an outcome of a side quest while geeking out on System Design. In the book “Designing Data-Intensive Applications,” Bloom Filters are briefly mentioned in the context of datastores, highlighting their significance in preventing database slowness caused by lookup for nonexistent items. Below are a curious set of questions on the topic on Bloom Filters and how it works. What is the…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : The cost crisis in the observability space is a real problem. Here is an article that describes the issue: - link here How many conferences are too many? Here is an exhaustive list of all the popular talks on Kubernetes from 2023: - link here OpenTelemetry is the industry…
Over the past few years, I have worked on different multi-cluster distributed datastores and messaging systems like - ElasticSearch, MongoDB, Kafka etc. From the Platform Engineering/SRE perspective, I have seen multiple incidents with different distributed datastores/messaging systems. Typical ones being : uneven node densities ( ElasticSearch - how are you creating shards? ) client node issues (…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : System Design - Designing a Ticket Booking Site Like Ticketmaster is the most common system design question - link UberEngineering blog on Anomaly detection and alerting system - link P99 CONF 2023 | Always-on Profiling of All Linux Threads by Tanel Poder - YouTube link On…
In this write up, we will try and explore how to make the most out of the resources in K8s cluster for the Pods on them. Resource Types: When it comes to resources on Kubernetes cluster, they can be fairly divided in to two categories: compressible : If the usage of this resource for an application goes beyond the max, it can be throttled without directly killing the application/process. example :…
[Self Notes and Review]: This is a second writeup in the series of reading DDIA book and publishing my notes from the book. The first one can be found here This particular article is from the second chapter of the book. Again, these are just my self notes/extracts and treat this more like an overview/summary. Best way is to read the book in itself. This chapter dwells in to the details of: the…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : The shortest and comprehensive System Design Template for any new service - link here Kafka is one of the most efficiently built transient datastore. This article explains the compute and storage layers of kafka – link here Consistent Hashing has helped solve…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : AI-Powered Search and Chat for AWS Docs. The best way of consuming AWS docs. – link here A Framework for Thinking About Systems Change - link here BookWyrm - Yet another attempt on building a social media based on book. link here git repo for the same - bookwyrm …
[Self Notes and Review]: This is a new series of publications where I am publishing my self notes/extracts from reading the very famous book - DDIA (Designing Data-Intensive Applications) by Martin Kleppmann. This particular article is from the first chapter of the book. Again, these are just my self notes/extracts and treat this more like an overview/summary. Best way is to read the book in…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : Don’t turn that Swap off yet. In defense of swap - common misconceptions. - link here Why histogram and how are they useful. link here If you have used any of the monitoring or APM tools, you would have come across Histogram form of metric being emitter. This writeup…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : A new book released on the hot eBPF - “Learning eBPF” by Liz Rice. This a summary form and a quick introduction to eBPF capabilities when compared to “BPF Performance tools” by Brendan Gregg. To understand latency in detail - “Everything You…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : [Video-57mins] : What is Continuous Profiling in Performance monitoring and What is Pyroscope - with Ryan Perry - link Go 1.20 is here(link) . A thread on all the changes - here “What’s the best lecture series you’ve seen?” - Thread link Some great…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : It is December and Advent of code is here. What is Advent of code ? - link here . An old podcast on Spotify’s Engineering team geeking out every December on AOC - link here [A talk - 31mins ] - Concurrency is not Parallelism - “Concurrency is about dealing with…
This writeup is more of a demo to showcase the power of “ proc ” (process information pseudo-filesystem) interface in linux to get the memory details of process, and also a quick brief on the power of “proc interface”. In the current trend of building abstraction over abstractions in software/tooling, very few tend to care about the source of truth of a metrics. There are…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : [Book-Recommendations]: Cloud-Native Observability with OpenTelemetry by Alex and Charity. (Side note: Half way through the book and learning a lot. ) Take away : Effective ways of adding metrics/traces in cloud native apps without marrying to any APM tools A list of…
This writeup is a dump of my study notes on most frequently used docker commands for reference. This is just a self reference page and will get updated on the go To run a container from an image docker run <image name> To run a docker image with a specific tag. Example below of pulling redis image with tag4.0. You will get these tag details on the dockerhub page for the image docker run redis:4.0…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : A few interesting and hidden features of Python - link Below are the BCP tools that can be used for digging in to Performance analysis of memory parameters on a Linux machine. More in BPF Performance Tools book by Brendan Gregg. Book : BPF Performance Tools How does…
There is no better pleasure than the Joy of solving your own problems. This write up is not to show off coding skill(there is hardly any code in this tool), but to show the ease with which anyone can build tools to solve problems these days. Problem statement: How to retain the most out of the books we read? Maybe receiving daily reminders with extracts from the books? I consume books mainly in…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : Memory leaks on client side - the forgotten side of web performance. Link A list of helpful patterns/commands on "_sed_" command - link A “ Streaming availability ” api to lookup which show/movie is available in which OTT in 60+countries. Something to explore…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : CPU utilization is wrong - PS : idle waits are counted in the %CPU. Python : Comprehensive Python Cheatsheet Finished coding, but waiting for PR to be review/approved ? - The Pull Request Paradox Best practices can slow your application down - Best Practices vs Required…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : P99 CONF (centered around low-latency, high-performance design) recordings are available here Python f-strings can do more than you thought. Video here Web3 is not a hype. An article on what is Web3 and decentralized internet here . Also, a podcast by Tim Ferris with Chris…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : eBPF Summit is live now - Recording of the Keynote and live summit here Conference talk - USENIX LISA2021 Computing Performance: On the Horizon by Brendan Gregg - here [video - 41mins] A project for visualizing codebase - here Self healing systems - the real end goal of…
As a Performance Engineer, time and again you will come across a situation where you want to profile CPU of a system. The reasons might be many; like, CPU usage being high, you want to trace a method to see its CPU cost or you suspect CPU times for a slow transaction. You might use one of the various profilers out there to do this. (I use yourkit and Jprofiler ). All these profilers report the CPU…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : All recording from PyCon US 2021 are up on Youtube here . My fav is Keynote by Robert Erdmann about rebuilding 5 µm resolution picture of Rembrandt’s painting “The Night Watch” from 18th century with Python. Rembrandt’s painting “The Night Watch” from 18th century…
These are the self-notes from managing 100+ node ES cluster, reading through various resources and a lot of production incidents due to unhealthy ES. Memory Always choose ES_HEAP_SIZE 50% of the total available memory. Sorting and aggregations both can be memory hungry, so enough heap space to accommodate these is required. This property is set inside the /etc/init.d/elasticsearch file. A machine…
When you come across a system which is misbehaving, majority of the times the first metrics that we look at is CPU usage. But do we really understand what CPU usage of a system tells us ? In this article let us try and understand what X % usage of a system really means. One of the easy ways to check on CPU is “top” command. The “%Cpu(s)” metrics seen above is a combination…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : Different states of Java Threads and their transitions. - Link A quick look into Sorting in python - RealPython site link (3mins) DevOps in one picture: Source: https://aws.amazon.com/devops/what-is-devops/ A cheat sheet to “When to use which collection in java”…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : “Performance checklist for SREs” - By Brendan Gregg at SREcon16 . YouTube link (1hr) Resource list for Beginner to Pro in Python. Link Navigation in IntelliJ IDEA - this could save so much time once all short cuts are know. YouTube link (8mins) Monitoring…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : Amazon S3 on it’s 15th Birthday – It is Still Day 1 after 5,475 Days & 100 Trillion Objects. An article here . A detailed Performance comparison of different programming languages / command-lines. Link here . ( If you can’t read full article, go through…
Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : An overview of iftop - a great network traffic visual tool. - here , also man page here Rust is becoming one of the most loved languages. Here is an Illustrated Note about WTF is Rust - link “ How They SRE ” - best practices, tools, techniques, and culture of…