Data Science and Math Blog
I believe that the way mathematicians conduct research will be fundamentally different in the near future. Below, I ll describe my unique perspective, developed from working as a mathematician and also in big tech developing AI solutions. Whilst AI will play a pivotal role, I believe the key next step is the adoption of software best-practices. Continue reading The (near) Future of AI in Math
In this post we will see that Log loss and AUC for binary classification problems are closely related. Suppose we have a set of probabilities that are meant to predict whether a true label is 0 or 1. That is, let be the predicted and true labels for the -th example, . Recall the AUC Continue reading AUC and Cross-entropy
Principal Component Analysis, or PCA, is a fundamental dimensionality reduction technique using in Machine Learning. The general goal is to transform a -dimensional problem into a -dimensional one, where is smaller than . This can be used, for instance, to plot large dimensional data on a 2D or 3D plot, or to reduce the number Continue reading PCA like a Mathematician
We all have some problem we are trying to solve. First off, if you can convert the problem to text, or increasingly pdfs, images, video, etc., then you might be able to use AI for some or all of the solution. Let s focus on text for now. In this case, we have Large Language Models Continue reading Breaking Down Problems for LLMs
In a recent YouTube Video I discuss how a cool feature from Amazon could be implemented with Large Language Models. The Customer Says feature, as picture here: allows one to In the video above, I explain how this feature can be broken down into a bunch of sub-problems, each of which can be solved with Continue reading Amazon s Customer Summary Feature
In this somewhat different post, I am hosting the long-running Carnival of Mathematics. First I ll talk about 223 (the issue number) and then I ll round up some mathematical posts from December 2023. It s primetime we talk about 223. First of all, it is a lucky prime, to which it is unknown if there are infinitely Continue reading Carnival Of Mathematics
In Machine Learning, and in particular Generative AI, temperature is a useful hyperparameter for tuning model outputs. In this post, we will discuss the following. Let s start with an example (the code I used is at the end of this post). Consider the following prompt, taken from Google s Minerva paper. A line parallel to passes Continue reading Temperature in Natural Language Processing
Principal Component Analysis (PCA) is a popular technique in machine learning for dimension reduction. It can be derived from Singular Value Decomposition (SVD) which we will discuss in this post. We will cover the math, an example in python, and finally some intuition. The Math SVD asserts that any matrix can be written as where Continue reading Singular Value Decomposition and PCA
In the first equation of the popular Attention is all you need paper (see also this blog post), the authors write In this post we are going to discuss where the comes from, leading us to some classical Probability Theory. We will first talk about the math with some examples and then quickly make the Continue reading The Square Root Cancellation Heuristic
How would ChatGPT read the infamous Hello, World! Does it see each character, sequentially H e l l o , W o r l d ! Or maybe it sees each word as well as the punctuation: Hello , World ! By the end of this post we will have a full understanding of this. Continue reading How Does ChatGPT read?