A Whole New Dimension of Optimization
A Whole New Dimension of Optimization
Trivial or Wrong
Machine Learning, Data Science, Programming, Mathematics and Algorithms.
Latest posts
Take the Rough With the Smooth
The starting point of the previous post was optimization of differentiable functions, but trying to utilize Fermat's theorem led us eventually to bracketing algorithms, which make no use of the derivative and are applicable for non-differentiable functions.
The Roots of No Evil
“…nothing at all takes place in the universe in which some rule of maximum or minimum does not appear…” -Leonhard Euler
Random Integers
The need to sample from a discrete distribution given by a probability vector $\vec{p}=(p_1,p_2,...,p_n)$ comes up very often. For example, when taking a bootstrapped sample from a weighted dataset, or when selecting parents from a population based on their realtive fitness in genetic algorithms, or when implementing an agent that applies a mixed strategy, or... well, I made my point: often.
Machine Learning. Literally.
What’s the most successful application of machine learning?
Laws, Sausages and ConvNets
Laws, like sausages, cease to inspire respect in proportion as we know how they are made.
The Generative-Discriminative Fallacy
1. The Generative-Discriminative Dichotomy It is customary to categorize predictive algorithms as either "generative" or "discriminative" . Denoting by $Y$ the target and by $X$ the observable, discriminative algorithms are those that model the conditional distribution $P(Y|X;\theta)$ and generative algorithms are those that model the joint distribution $P(X,Y|\theta)$) (note that this entire…
Learning Dynamical Systems
1. Motivational Speech
The Name of The Rose
Many ideas in machine learning came up independently in different contexts, and it's not uncommon to have multiple terms for the same concept. Those instances are usually dismissed as nuisance; after all, a rose by any other name, et cetera. Indeed, often this is just a matter of nomenclature, and with time conventions form, and some terms disappear while others acquire universal meaning. But…
Pointless Topology via Abstract Nonsense
In 1940. G.H. Hardy published his essay "A Mathematician's Apology" in which he used number theory as a prominent example for pure and inapplicable mathematics. He meant it seriously, not as a joke. To the defense of his apology (pun a bit intended), at the time only a handful of programmers thought about hash-tables, cryptosystems were yet to be widely implemented in E-commerce platforms, and not…