Everybody's Favorite Data Blog
My friend Andrew is an advocate of the spaced repetition technique for memorization of a great many facts [1]. The ideas behind this are two-fold: When one first learns a new fact, it needs to be reviewed frequently in order to not forget it. However, with each additional review, the …
There are four ways to make change for \(N=10\) cents: \(\{\) 10 pennies; 1 nickel and 5 pennies; 2 nickels; 1 dime \(\}\) . How many ways are there to make change for one trillion dollars using just pennies, nickels, dimes, and quarters? To answer this, we present here a hybrid dynamic …
In this note, I consider a generalization of Dollar Cost Averaging a popular investing strategy that involves gradually building up one s holding in a stock over a pre-specified period of time. The generalization I consider can guarantee better prices paid per share relative to the standard approach but its …
Textbook proofs of the duality theorem often apply abstract arguments that offer little tangible insight into the relationship between a linear program and its dual. Here, we map the general linear program onto a simple mechanics problem. In this context, the significance of the theorem is relatively clear. The general …
In this post, I explore some basic math behind the World Wandering Dudes framework previously introduced in this post . To briefly reintroduce the system: imagine a field, a 2-D square lattice of \(M\) sites with food distributed randomly at a density \(\rho_{food}\) . Creatures wander across the field taking \(N …
We review how one can price a dividend-bearing stock by simply discounting its dividend stream using the value suggested by the Capital Asset Pricing Model ( CAPM ). As an example, we consider the price of AT &T common stock. The model result matches the current market price quite well. Varying inputs …
A person s happiness does not depend only on their current lot in life, but also on the rate of change of their lot. This is because a person s prior history informs their expectations. Here, we build a model that highlights this emotional path-dependence quality of utility. Interestingly, we …
I was struck the other day by the following: The cooking instructions on my Bob s tri-colored quinoa package said to combine 2 cups of water with 1 cup of dried quinoa, which would ultimately create 4 cups of cooked quinoa. See image above. My first reaction was to believe …
[ TOC ] Summary Standard reinforcement learning algorithms struggle with poor sample efficiency in the presence of sparse rewards with long temporal delays between action and effect. To address the long term credit assignment problem, we build on the work of [1] to use “temporal reward transport” ( TRT ) to augment the immediate …
Deep reinforcement learning algorithms can be hard to debug, so it helps to visualize as much as possible in the absence of a stack trace [1]. How do we know if the learned policy and value functions make sense? Seeing these quantities plotted in real time as an agent is …
Here, we examine the statistics behind discrete random walks on square lattices in \(M\) dimensions, with focus on two metrics (see figure below for an example in 2-D): 1. \(R\) , the final distance traveled from origin (measured by the Euclidean norm) and 2. \(N_{unique}\) , the number of unique locations …
[ TOC ] Q-learning is a reinforcement learning ( RL ) algorithm that is the basis for deep Q networks ( DQN ), the algorithm by Google DeepMind that achieved human-level performance for a range of Atari games and kicked off the deep RL revolution starting in 2013-2015. We begin with some historical context, then provide …
Pooling of test samples can be used to reduce the mean number of test counts required to determine who in a set of subjects carries a disease. E.g., if the blood samples of a set of office workers are combined and tested, and the test comes back negative, then …
Background We discuss how to use dynamic programming ( DP ) to solve reinforcement learning ( RL ) problems where we have a perfect model of the environment. DP is a general approach to solving problems by breaking them into subproblems that can be solved separately, cached, then combined to solve the overall problem …
We take a top-down approach to introducing reinforcement learning ( RL ) by starting with a toy example: a student going through college. In order to frame the problem from the RL point-of-view, we ll walk through the following steps: Setting up a model of the problem as a Markov Decision Process …
In this post, I want to briefly introduce a new side project for the blog with applications to understanding population dynamics, natural selection, game theory, and probably more. World Wandering Dudes is a simulation framework in which you initiate a “world” which consists of a “field” and a set of …
Reinforcement Learning: An Introduction by Sutton and Barto[1] is a book that is universally recommended to beginners in their RL studies. The first chapter is an extended text-heavy introduction. The second chapter deals with multiarmed bandits, i.e. slot machines with multiple arms, and is the subject of today …
Two weeks ago, I started at the OpenAI Scholars program, which provides the opportunity to study and work full time on a project in an area of deep learning over 4 months. I’m having a blast! It’s been a joy focusing 100% on learning and challenging myself in …
CPPI * is a risk management tactic that can be applied to any investment portfolio. The approach entails banking a percentage of profits whenever a new all time high wealth is achieved, thereby ensuring that a portfolio s drawdown never goes below some maximum percentage. Here, I review CPPI and then …
We consider the equilibrium drawdown distribution for a biased random walk in the context of a repeated investment game, the drawdown at a given time is how much has been lost relative to the maximum capital held up to that time. We show that in the tail, this is exponential …
We give a simple class for marking the time at different points in a code block and then printing out the time gaps between adjacent marked points. This is useful for identifying slow spots in code. The TimeMarker class In the past, whenever I needed to speed up a block …
Overview We give a short introduction to neural networks and the backpropagation algorithm for training neural networks. Our overview is brief because we assume familiarity with partial derivatives, the chain rule, and matrix multiplication. We also hope this post will be a quick reference for those already familiar with the …
We evaluate an integral having to do with vector averages over all orientations in an n-dimensional space. Problem definition Let \(\hat{v}\) be a unit vector in \(n\) -dimensions and consider the orientation average of \begin{eqnarray} \tag{1} \label{1} J \equiv \langle \hat{v} \cdot \vec{a}_1 …
Here, we highlight one of the most important benefits of tax protected accounts (eg Traditional and Roth IRAs and 401ks). Specifically, we review the fact that not having to pay taxes on any investment growth that occurs while the money is held in the account results in compounding / exponential growth …
We consider how the GDP or utility output of a city depends on the number of people living within it. From this, we derive some interesting consequences that can inform both government and individual attitudes towards newcomers. Edit 9/2022: The model here can t be complete because it doesn …
We use a simple argument to estimate the speed of traffic on a highway as a function of the density of cars. The idea is to simply calculate the maximum speed that traffic could go without supporting a growing traffic jam. Jam dissipation argument To estimate the speed of traffic …
We illustrate the application of two linear compression algorithms in python: Principal component analysis ( PCA ) and least-squares feature selection. Both can be used to compress a passed array, and they both work by stripping out redundant columns from the array. The two differ in that PCA operates in a particular …
This is a tutorial post relating to our python feature selection package, linselect . The package allows one to easily identify minimal, informative feature subsets within a given data set. Here, we demonstrate linselect s basic API by exploring the relationship between the daily percentage lifts of 50 tech stocks over …
It has been quite awhile since I have posted, largely because soon after I started my job at Square I had a child! I hope to have some newer blog post soon. But along those lines I want to share a blog post I did with a coworker ( Juan Hernandez …
This note addresses the typical applied problem of estimating from data how a target conversion rate function varies with some available scalar score function e.g., estimating conversion rates from some marketing campaign as a function of a targeting model score. The idea centers around estimating the integral of the …
We review the math and code needed to fit a Gaussian Process ( GP ) regressor to data. We conclude with a demo of a popular application, fast function minimization through GP -guided search. The gif below illustrates this approach in action the red points are samples from the hidden red curve …
Here, I give a quick review of the concept of a Martingale. A Martingale is a sequence of random variables satisfying a specific expectation conservation law. If one can identify a Martingale relating to some other sequence of random variables, its use can sometimes make quick work of certain expectation …
We review binary logistic regression. In particular, we derive a) the equations needed to fit the algorithm via gradient descent, b) the maximum likelihood fit s asymptotic coefficient covariance matrix, and c) expressions for model test point class membership probability confidence intervals. We also provide python code implementing a minimal …
I review and provide derivations for some basic properties of Normal distributions. Topics currently covered: (i) Their normalization, (ii) Samples from a univariate Normal, (iii) Multivariate Normal distributions, (iv) Central limit theorem. Introduction This post contains a running list of properties (with derivations) relating to Normal (Gaussian) distributions. Normal distributions …
The AUC score is a popular summary statistic that is often used to communicate the performance of a classifier. However, we illustrate here that this score depends not only on the quality of the model in question, but also on the difficulty of the test set considered: If samples are …
We demo a script that converts python numerical commands to LaTeX format. A notebook available on our GitHub page will take this and pretty print the result. Introduction Here, we provide a simple script that accepts numerical python commands in string format and converts them into LaTeX markup. An example …
Here, we provide a brief introduction to reinforcement learning ( RL ) a general technique for training programs to play games efficiently. Our aim is to explain its practical implementation: We cover some basic theory and then walk through a minimal python program that trains a neural network to play the game …
There are many tutorials with directions for how to use your Nvidia graphics card for GPU -accelerated Theano and Keras for Linux, but there is only limited information out there for you if you want to set everything up with Windows and the current CUDA toolkit. This is a shame …
Here, we briefly review a subtlety associated with machine-learning model selection: the fact that the optimal hyperparameters for a model can vary with training set size, \(N.\) To illustrate this point, we derive expressions for the optimal strength for both \(L_1\) and \(L_2\) regularization in single-variable models. We find that …
We review the Metropolis algorithm a simple Markov Chain Monte Carlo ( MCMC ) sampling method and its application to estimating posteriors in Bayesian statistics. A simple python example is provided. Introduction One of the central aims of statistics is to identify good methods for fitting models to data. One way to …
Our last post showed how to obtain the least-squares solution for linear regression and discussed the idea of sampling variability in the best estimates for the coefficients. In this post, we continue the discussion about uncertainty in linear regression both in the estimates of individual linear regression coefficients and the …
We review classical linear regression using vector-matrix notation. In particular, we derive a) the least-squares solution, b) the fit s coefficient covariance matrix showing that the coefficient estimates are most precise along directions that have been sampled over a large range of values (the high variance directions, a la PCA …
Queries ping a certain computer server at random times, on average \(\lambda\) arriving per second. The server can respond to one per second and those that can t be serviced immediately are queued up. What is the average wait time per query? Clearly if \(\lambda \ll 1\) , the average wait …
A common task in applied statistics is the pairwise comparison of the responses of \(N\) treatment groups in some statistical test the goal being to decide which pairs exhibit differences that are statistically significant. Now, because there is one comparison being made for each pairing, a naive application of the …
A previous post showed beginners how to try out deep learning libraries by using an Amazon Machine Image ( AMI ) pre-installed with deep learning libraries setting up a Jupyter notebook server to play with said libraries If you have VirtualBox and Vagrant , you can follow a similar procedure on your own …
Want a quick and easy way to play around with deep learning libraries? Puny GPU got you down? Thanks to Amazon Web Services ( AWS ) specifically, AWS Elastic Compute Cloud ( EC2 ) no data scientist need be left behind. Jupyter/IPython notebooks are indispensable tools for learning and tinkering. This post shows …
Reinstalling software and configuring settings on a new computer is a pain. After my latest hard drive failure set the stage for yet another round of download-extract-install and configuration file twiddling, it was time to overhaul my approach. Enough is enough! This post walks through how to back up and …
Two microphones are placed in a room where two conversations are taking place simultaneously. Given these two recordings, can one remix them in some prescribed way to isolate the individual conversations? Yes! In this post, we review one simple approach to solving this type of problem, Independent Component Analysis ( ICA …
In this post, we review two facts about maximum-likelihood estimators: 1) They are consistent, meaning that they converge to the correct values given a large number of samples, \(N\) , and 2) They satisfy the Cramer-Rao lower bound for unbiased parameter estimates in this same limit that is, they have the …
We review the two essentials of principal component analysis ( PCA ): 1) The principal components of a set of data points are the eigenvectors of the correlation matrix of these points in feature space. 2) Projecting the data onto the subspace spanned by the first \(k\) of these listed in descending …