Thoughts on software and data
I’ve been working on improving how we schedule maintenance in Akamai’s cloud infrastructure, especially disruptive maintenance on hypervisor hosts serving hundreds of thousands of guest VMs. The problem is fairly complex, with competing constraints such as capacity, customer disruption SLAs, and concurrency limits across hosts, racks, and datacenters.
(This blog post later got published on Akamai Blog)
Soon after I started working on mixed integer linear programs (MILPs), it became obvious that the representation of the problem is very important for the mathematical solver to be able to find a solution fast without using too many resources. There are several ways to represent a problem, and it is not always clear which one is the best. It takes a great amount of experience and knowledge about…
I started the new year with a very puzzling bug. One of the users of my internal dashboard tool told me that he doesn’t see some of the rows he expected to see. I checked the query that populated the view, which had various filters but none of them looked suspicious, expect one of them. There was an IFNULL looking at me in the query which smelled awful.
About 6 months ago, I started working on a very interesting problem At Akamai: optimizing the allocation of virtual machines to hosts. We decided to approach this problem as a dynamic multi-dimensional bin packing problem and started working on a Mixed Integer Program to solve it. I’ve learned a lot of new techniques while developing the optimization model and wanted to share some tips in a blog…
I am using the docker-compose.yml provided by Airflow to run Airflow on Docker. It is a fairly complete setup with a CeleryExecutor so you are not limited to 1 DAG run at a time.
I recently got into deep learning and went over MIT’s 6.S191 to understand the fundamentals. I absolutely recommend at least doing their labs as they teach you the fundamentals pretty quickly and give you some hands-on experience.