I have learned to conceptualize the pace of progress in the sciences as follows. There are three factors you can trade off against each other while keeping the pace of progress constant: Dataset size Dataset signal-to-noise ratio Intelligence of the experimentalist If your dataset is small and low signal-to-noise, you have to be really clever to make a discovery! In 1801, Giuseppe Piazzi spotted a…
Recently, I was working on a Kubernetes workload that ingests time-series data at a high frequency, analyzes it and accordingly routes packets to downstream pods. In particular, 64 streams (or "channels") of data, where each data point is a decimal value, are produced by 64 sources and sent to the workload as individual UDP packets. When the workload detects a sudden spike in the EWMA…
For the last two years, I've had the privilege of working in a scientific R&D lab that's pushing the frontier of electrochemistry. At the same time, I've witnessed AI systems acquire capabilities at breakneck speed. The resulting productivity gains have so far been primarily limited to the virtual world . While I expect this alone to eventually compound into significant increases in total factor…