Notes on applied LLMs, agents, and machine learning, by Hamidreza Saghir. Principal Applied Scientist at Microsoft. Author of Looplet.
A follow-up. The under-specification problem in coding agents does not close by writing better specs upfront. Models do not stick to instructions anyway. The gap closes by building better detection: catching deviation cheaply, often, and at the right level of abstraction. The lever is the loop, not the prompt.
Coding agents write impressive first drafts. But under the surface, corners are cut, details are missing, and technical debt accumulates with every change. The problem is not the model. It is that what we ask it to do is fundamentally under-specified.
Agent frameworks hide the loop behind agent.run() and a graph DSL. But the loop is where every interesting decision happens: what the model sees, whether a tool call proceeds, when to stop, what to record. What if you owned the loop and the framework just made it composable?
Offense asks 'does a bug exist?' Defense asks 'are all bugs gone?' One is an existential claim you can check with a single example. The other is a universal claim nobody can check. This asymmetry, not model capability, is what determines where AI agents work in security.
Both fit a distribution over outputs conditioned on an input. Both minimize a KL divergence between their model and an optimal target. The only differences are which distribution you sample from and which direction of the KL. Entropy regularization bridges them.
From spectral clustering to Gaussian processes to transformer attention, the same primitive, a similarity matrix between points, keeps showing up as the load-bearing piece of very different models.
Back after a long hiatus, what's changed and what's coming.
BFS, Dijkstra, and A* differ by one line: the data structure you pop the next node from. A worked maze example that converts each into the next.
So PyTorch is the new popular framework for deep learners and many new papers release code in PyTorch that one might want to inspect. Here is my understanding of it narrowed down…
PCA, factor analysis, ICA, Gaussian mixtures, vector quantization, HMMs, and Kalman filters are the same two equations with different restrictions on the latent variables. One EM recipe fits all of them.
Backpropagation, belief propagation, the Viterbi algorithm, and matrix-chain multiplication all solve the same problem: summing over exponentially many paths in a graph by reusing work.
I love the simplicity of autoencoders as a very intuitive unsupervised learning method. They are in the simplest case, a three layer neural network. In the first layer the data…
PCA, factor analysis, logistic regression, Gaussian mixtures, HMMs, and Kalman filters are the same probabilistic graphical model with different independence assumptions. Seeing this gives you one inference recipe that handles all of them.
What is a neural network? To get started, it's beneficial to keep in mind that modern neural network started as an attempt to model the way that brain performs computations. We…
I typically use my computers at home to connect to my work computer. I setup xRDP to remote desktop into my work computer(Linux) which is OK but slow at times depending on the…
Theano might look intimidating, but there are a few concepts that if understood, would make the engineering involved in deep learning more tangible. The first is the concept of…
Deep learning is hot! Mostly due to significantly improved results that you might have heard about. The use of graphical processing units (GPUs) that can perform many calculations…
Barbara Minto's pyramid: put the answer at the top, let the reader's questions drive the hierarchy, and choose deduction or induction at each branch.
I have been blogging about the qualifications of advanced degree holders and how they should be approaching a job search. However, job rejection is commonplace and it can be a very frustrating experience.
Recently, I read a post on the Chronicle where the author had listed all the excuses she could come up with, to justify her decision not to pursue a fulfilling and rewarding…
A PhD has traditionally been the path to a career in academia. However, recent job trends have led to less than 1% placement rate of STEM PhD graduates in tenure positions.…
An old note on Elon Musk's first-principles reasoning, updated for 2026. The politics and the personality have not aged well; the reasoning technique still has.
One of the most exhilarating observations of physics and mathematics for me, comes from understanding the concept of nonlinearity. i.e. inputs don't necessarily need to be…
The world is nonlinear. Most outcomes worth wanting do not cost proportionally more effort, they cost different effort. And once you accept that, Richard Hamming's 1986 lecture on doing important research stops sounding like advice and starts sounding like a corollary.