Madiyar's Page
Why are two random vectors near orthogonal in high dimensions? Introduction High-dimensional embedding vectors are fundamental building blocks in Machine Learning, particularly in transformers or word2vec . Typically, two vectors that are semantically similar point in roughly the same direction; if they are entirely dissimilar, they point in opposite directions; and if they’re nearly orthogonal,…
Implementation of ROC AUC Score Introduction This post is a continuation of the ROC and AUC Interpretation . Please make sure that you understand that post before reading this one. In this post, we will implement a ROC AUC Score in Python with runtime complexity. Subscribe to get a notification about future posts. Explanation Similar to the previous post we have: A dataset with positive and…
ROC and AUC Interpretation Introduction A binary classification is a machine learning model that classifies input data into two classes. We need different metrics to train or evaluate the performance of ML models. The Area Under the Receiver Operating Characteristic Curve ( ROC AUC ) score is a popular metric for evaluating binary classification models. In this post, we will try to understand the…
Linear Equation Intuition Introduction A linear equation is an equation in the form . In two dimensions, it’s an equation of a line , while in three dimensions, it’s an equation of a plane . A linear equation is the foundation of linear models in machine learning, such as linear regression and logistic regression . In this post, let’s understand the intuitions behind the linear equation, such as:…
Dot Product Intuition Introduction The dot product (or scalar product ) is a simple yet powerful operation that is used in many places in Machine Learning and other fields. In this post, I will explain the geometric intuition behind the dot product. You need to have a basic grasp of trigonometry and vector algebra to follow this post. I will need this explanation for my future posts. Feel free to…
Geometric Intuition for Jensen’s Inequality Introduction Jensen’s inequality is fundamental in many fields, including machine learning and statistics. For example, it is useful in the diffusion models paper for understanding the variational lower bound. In this post, I will give a simple geometric intuition for Jensen’s inequality. Subscribe to get a notification about future posts. Feel free to…
Why does L1 regularization encourage coefficients to shrink to zero? Introduction Regularization is a common method for dealing with overfitting in Machine Learning (ML). The simplest and most widely used methods are L1 (Lasso) and L2 (Ridge). The L1 and L2 regularizations are well covered in numerous tutorials and books. However, I could not find any good geometric or intuitive explanation of why…
Notes for “High-Resolution Image Synthesis with Latent Diffusion Models” My notes for the “High-Resolution Image Synthesis with Latent Diffusion Models” paper. Feel free to ask questions on my telegram channel
Notes for “Auto-Encoding Variational Bayes” My notes for the “Auto-Encoding Variational Bayes” paper. Feel free to ask questions on my telegram channel Method Problem scenario Let us consider some independently and identically distributed (i.i.d) dataset . We assume that the data involves an unobserved random variable . Then, the process consists of two steps: is generated from some prior…
Notes for “Denoising Diffusion Probabilistic Models” My notes for the “Denoising Diffusion Probabilistic Models” paper. Feel free to ask questions on my telegram channel Background Forward Process We have latent variables of the same dimensionality as the image . The forward process or diffusion process is fixed to a Markov chain that gradually adds Gaussian noise to the image according to a…