Understanding Biology through AI
Until recently, generative models were a bit like neural nets pre-2012 when AlexNet came out. People knew about them but kept asking what you could really use them for. Text-to-image models DALL-E and StableDiffusion , and the language model ChatGPT changed this–these models mark the AlexNet moment for generative modeling. The best part? These models are publicly available. So you can ask ChatGPT…
Masked-image modeling (MIM) is about inpainting; that is, covering parts of an image and then trying to recover what was hidden from what is left. Recently, it has led to state-of-the-art representation learning in images 1 . In this blog, I will dive into why masked images deliver such a powerful learning signal, think about what may constitute a good mask, and discuss my recent paper ( ADIOS )…
In machine learning, we typically work with input pairs (x, y), and we try to figure out how x and y depend on each other. To do so, we gather many such pairs and hope that the dependence will reveal itself if a) we have enough data, b) our model is expressive enough to approximate this dependency, and c) we get the hyperparameters right. In the simplest case, both x and y are just scalar values…
Objects play a central role in computer vision and, increasingly, machine learning research. With many applications depending on object detection in images and videos, the demand for accurate and efficient algorithms is high. More generally, knowing about objects is essential for understanding and interacting with our environments. Usually, object detection is posed as a supervised learning…
Every time I begin a machine learning (ML) project, I go through more or less the same steps. I start by quickly hacking a model prototype and a training script. After a few days, the codebase grows unruly and any modification is starting to take unreasonably long time due to badly-handled dependencies and the general lack of structure. At this point, I decide that some refactoring is needed:…
Machine learning is all about probability. To train a model, we typically tune its parameters to maximise the probability of the training dataset under the model. To do so, we have to assume some probability distribution as the output of our model. The two distributions most commonly used are Categorical for classification and Gaussian for regression. The latter case can be problematic, as the…
Latent Variable Models Suppose you would like to model the world in terms of the probability distribution over its possible states \(p(\mathbf{x})\) with \(\mathbf{x} \in \mathcal{R}^D\). The world may be complicated and we do not know what form \(p(\mathbf{x})\) should have. To account for it, we introduce another variable \(\mathbf{z} \in \mathcal{R}^d\), which describes, or explains the content…
Attention mechanisms in neural networks, otherwise known as neural attention or just attention , have recently attracted a lot of attention (pun intended). In this post, I will try to find a common denominator for different mechanisms and use-cases and I will describe (and implement!) two mechanisms of soft visual attention. What is Attention? Informally, a neural attention mechanism equips a…
Inference is hard and often computationally expensive. Variational Autoencoders (VAE) lead to an efficient amortised inference scheme, where amortised means that once the model is trained (which can take a long time), the inference has constant computational complexity. Variational Autoencoders (VAE) learn the approximate posterior distribution \(q(z\mid x)\) over some latent variables \(z\) by…
Variational Autoencoders (VAEs) are getting more and more popular in the Machine Learning community. While the formulation is more involved then that of a typical feed-forward neural network, VAEs have a lot of added benefits. I’ve been recently playing with one of the more complicated VAE models: Attend, Infer, Repeat (AIR) by Ali Eslami et. al. from DeepMind , and I must say it’s really cool. In…