The response to our launch has been incredible — 30K views and 400+ love in 24 hours. A ton of folks booked our calendar for a demo and a ton more signed up for the waitlist; we couldn’t wait to deliver the product to everyone 🙂 Sending all the love to everyone who showed interest [ ]
Zero to One Peter Thiel If every one of India s hundreds of millions of households were to live the way Americans already do using only today s tools the result would be environmentally catastrophic. Building a Digital ID system for India would definitely have to be different than for the US. If you [ ]
Event: Bengaluru Systems Meetup #6 Wrapping up 2024! Many underestimate the outcomes of building a centralised config system with deployment semantics that are quite widely used for code like canarying, slow rollouts, rollbacks on errors etc,.. IMO, A solid config management system is one of the highest leverage tools for improving an organization’s productivity and [ ]
Event: 2024 Wrap-Up: Grafana Friends x Bangalore SRE Year end connect This talk is something that s pretty close to my heart, given the fact that I spent a significant amount of my career handling production incidents and building systems to improve the effectiveness of the same. While there are the usual parts like the [ ]
Event: Rootconf Mini 2024 This talk is a repeat of the one I presented at the Systems Meetup a few weeks ago. Overall, I feel the Systems Meetup presentation went significantly better than the Rootconf one ¯\_(ツ)_/¯. Caching is one of the fundamental concepts many of us encounter early in our careers. It gives these [ ]
Reducing Deployments Would Reduce Incidents? This is definitely an intriguing insight As expected, extremes like elite and low-performing organisations show predictable trends: low-performing orgs have higher failure rates, while elite-performing orgs have fewer. It’s important to note that performance here refers to software delivery frequency, not business outcomes. What s interesting, however,…
Event: Bengaluru Systems Meetup #4 Had an incredible time presenting at this meetup! Interestingly, this topic was actually one of the first I presented almost 5 years ago during the SRE Learning Sessions at Zomato. Distributed Systems are born out of the need to support scale beyond a single node. Consequently, balancing the load among [ ]
Event: Bangalore Systems Meetup #3 Caching is probably the most deceptive fundamental concept that most of us come across. If caching could be imagined as a movie, it would be a thriller with most exciting surprises. You would see life changing wins like performance optimisations saving 99% of the costs, so many different ways of [ ]
Event: Bangalore SRE meetup After seeing too many posts on how a vendor solution can cut down 90% of the kafka cost, I thought its important to share how to understand the cost of your kafka cluster before trusting whether a vendor solution would actually help in cutting down your cost. For example implementing a [ ]
Optimizing Aurora Cost using Aurora Local Write Forwarding, and the Risks of this feature AWS recently released a rather very interesting feature that solves a really important problem that has been there from a long time i.e Read Write splitting, most solutions are implemented as a proxy layer on top of the mysql cluster for [ ]
This release is definitely the most feature packed after the generics one, a lot of experimental generics packages got matured and turned into standard library (slog, slices, maps, cmp), and now you can provide reasons for context cancellation. Initialisation Order Changes This definitely is going to break some programs in production, the first step of [ ]
Systems could fail due to several unexpected failures, degraded hardware suddenly showing higher latency, partial network disruption to some specific instances, or network cable cut-off etc.,… Health checking, in principle, handles such unexpected localised failures that happen to particular containers or VMs among the entire cluster by removing them from the workload. This way total [ ]
If you are working on a go service, it’s essential to properly shut down without causing errors to clients and not losing any information while shutting down. Without a proper graceful shutdown, the clients would get errors whenever a container is shutting down on the server side. In the worst case, it could lead to [ ]
Most systems have fixed-size thread pools, apache httpd, php-fpm, gunicorn, gocraft, asynq, and machinery. And those systems work great and help quickly scale without solving the problem from the ground up. Still, with enough scenarios, you would start noticing minor issues like using only about 25% of the CPU, and still, your queue lag is [ ]
Contract Validation issues are one of the highest code change-based incidents I have seen in my experience. They come in many shapes and forms and across different kinds of systems. Let s begin with the shapes and forms of these issues and then go onto different systems where they appear. Schema Based Issues Most contract issues [ ]
At zomato we used statsd exporter for envoy metrics initially and later when we had the usecase to report application metrics a couple of years ago, we added a core module in our monolith php codebase to send stats to the statsd exporter sidecar that we are already using for envoy metrics. But while transitioning [ ]
our elasticsearch cluster(part of our elk stack) is a pretty standard cluster hot/cold cluster. where logstash writes today’s logs data to hot nodes and the next day early morning those indices get moved to the cold nodes, we do this by keeping our index pattern as {index-name}-yyyy.mm.dd the yyyy.mm.dd is derived from now() in the [ ]
Thanos is great to query from multiple prometheus instances, and after setting up needs almost very little maintenance, but is still not horizontally scalable atleast not out of the box. Although the querier and store components are scalable, a significant amount of efforts are spent on the setting up individual prometheus instances to scrap the [ ]
Often applications need to install it’s dependencies through a package manager, and most langauge don’t use system package manager to do this job. ex: python uses pip, php uses composer, ruby uses gem. and all of these package managers install packages for a project based on a package list file. and it is a common [ ]