Focus on AI, Cloud Infrastructure, Linux, Maker Stuff
In context learning seems to largely just work, we have utilized RLHF and RL based on verifiable rewards to the limits of economic viability such that Yann LeCun gets a billion dollars seed funding on the thesis that further scaling has diminishing returns. While they are great coders now, Karpathy coins LLMs as having “jagged” boundaries. They are simultaneously superhuman and make mistakes a…
The following are loose notes while going through the papers in The 2025 AI Engineer Reading list They are personal notes and are not refined for an external audience really. Still they might be useful to someone :). Foundational Models GPT-1 Uses two step process of first doing unsupervised pretraining and then does supervised finetuning. Finetuning happens not just via a more curated dataset…
k-d trees are a widely-used data structure for organizing points in a k-dimensional space, enabling efficient nearest-neighbor search and range queries. They work by recursively partitioning the space along the axes of the dimensions, with each node associated with a hyperplane perpendicular to its selected axis. While axis-aligned splitting is convenient, it may not always align well with the…
This blog post is ChatGPTs rewrite of my own blog post. You can read the original here . Lately, I’ve been exploring the potential of Large Language Models (LLMs) like GPT-4 to enhance productivity, particularly in the realm of blog post writing. While SEO spam is commonly regarded as the first major commercial application for LLMs, I decided to experiment with generating blog posts using ChatGPT…
I wrote this blog post on my own. I let ChatGPT create an “improved” version of this blog post. You can read the rewritten version here . I’ve been trying to use LLMs(Large language models like GPT4) for productivity increases lately. One area of interest is writing blog posts. SEO spam is the area that everyone seems to be thinking of as the first to see wide commercial application of LLMs. So I…
I am greatly dependent on Google Maps. Whenever I go someplace new or want to navigate, I turn to Google Maps. I don’t like being dependent on FAANG companies, so here we are building our own tile server. Running your own tile server is quite intimidating, as it comes with quite a few moving parts (apache, mod_tile, renderd, osm2psql, postgres, mapnik). Also the hardware requirements are quite big…
Wouldn’t it be great if your shopping list would write itself? Wouldn’t it be great to have automated statistics about your grocery buying/consumption? Reducing Shopping / Starting Stocking When Corona hit about one year ago, there were empty shelves in stores everywhere. No matter where you looked: Shanghai, Moscow, Berlin, Paris, New York - panic buying everywhere. In an effort to become…
One of my services was running into timeouts, because connections were being reset by clients. I had assumed that it couldn’t be the Network Load Balancers (NLBs) fault, a battle-tested piece in the cloud stack. The following is about NLBs, cross-zone-loadbalancing and multiple NLBs pointing to the same set of EC2 instances and how that revealed unexpected behaviour in the NLB. TLDR : If you don’t…
While daily news is all over the coronavirus, I want to try a long-term view on what changes will happen because of Coronavirus for the next decade. Individual behaviour More food stock is kept even in urban environments -> average supermarket cart size increases Handshakes become less common, hugs/kisses reserved for close friends Yearly flue shots become more popular even in healthy young people…
On my cycle-commute there are a couple of places where better lights would come in handy. A year ago another biker drove into me, because I had to emergency brake. So when a friend gifted me his left-over addressable LED stripe I was intrigued in how well this would work, as I had previously poured ours of work into making my own addressable LEDs using shift registers see my first iron man heart…