Personal website of Andy Timm
My new R package regrake is now live! Regularized raking enables more flexible functional forms in raking to population targets, leading to more expressive and efficient survey weights. You can install the package from r-universe or github: install.packages ( "regrake" , repos = "https://andytimm.r-universe.dev" ) The rest of this post gives a light overview of the problem regularized raking…
If you wish to make an apple pie LLM from scratch, you must first invent the universe systems considerations. — Carl Sagan, probably In December, I finished working through Stanford NLP’s CS336: Language Models From Scratch . This post is a short review of the class, and a bunch of practical tips/considerations if you’re considering auditing it as well. The basic premise of the course is that:…
Ahead of starting my current gig last February, I was reviewing a bit about multilevel models, and came across this fascinating footnote in Richard McElreath’s Statistical Rethinking : See Jaynes (1986) for an entertaining example concerning the beer preferences of left-handed kangaroos. There is an updated 1996 version of this paper available online. I’m not sure what it is with Bayesians and…
Jim Savage calls the MaxDiff model of discrete choice a “mathematically incorrect model of a psychologically incoherent concept” 1 . Despite this lovely dunk, and some wonderful notes explaining why MaxDiff’s not great, the model remains frequently used, most prominently in market research, as implemented in Sawtooth . Why is this? Beyond the most obvious explanations like inertia, it’s also…
This is a draft of a paper I wrote in grad school on whether formation of voting habits exhibits any heterogeneity by race, party, or gender, using data from the Florida voter file. Putting it online since I’ve shown the draft to a few folks at this point and someone mentioned it’d be nice to have it publicly available. TL;DR: It doesn’t seem like there is any strong heterogeneity by race, party,…
I recently gave a talk on Regularized Raking at the New York Open Statistical Programming Meetup . Here is the abstract: Raking is among the most common algorithms for producing survey weights, but it is often opaque what qualities of the resulting weights set are prioritized by the method. This is especially true when practitioners turn to heuristic methods like trimming to improve weights. After…
Note: Since writing this post, Pathfinder (described below as well in Section 5) has become available in CmdStan, which poses a problem for this concluding post to the series. More specifically, the running example (a simple-ish MRP model), was chosen to be too complex to get a good approximation with a simple mean-field or full-rank approximation as implemented by rstanarm , but Pathfinder is…
This is section 6 in my series on using Variational Inference to speed up relatively complex Bayesian models like Multilevel Regression and Poststratification without the approximation being of disastrously poor quality. The general structure for this post and the posts around it will be to describe a problem with VI, and then describe how that problem can be fixed to some degree. Collectively,…
This is section 5 in my series on using Variational Inference to speed up relatively complex Bayesian models like Multilevel Regression and Poststratification without the approximation being of disastrously poor quality. The general structure for this post and the posts around it will be to describe a problem with VI, and then describe how that problem can be fixed to some degree. Collectively,…
This is section 4 in my series on using Variational Inference to speed up relatively complex Bayesian models like Multilevel Regression and Poststratification without the approximation being of disastrously poor quality. The general structure for this post and the ones after it will be to describe a problem with VI, and then describe how that problem can be fixed to some degree. Collectively, all…
Note: I’ve gotten a lot more pessimistic about how generally useful the alternatives to simple KL-Divergence are on their own since writing this post. I still think these are really useful ideas to think to build intuition about VI, and techniques like CHIVI are useful for some lower dimensional problems or as part of an ensemble of techniques for high dimensional ones. However, this paper from…
This is the second post in my series on using Variational Inference to speed up relatively complex Bayesian models like Multilevel Regression and Poststratification without the approximation being of disastrously poor quality. In the last post, I laid out why such reformulating the Bayesian inference problem as optimization might be desirable, but previewed why this might be quite hard to find…
This post introduces a series I intend to write, exploring using Variational Inference to massively speed up running complex survey estimation models like variants of Multilevel Regression and Poststratification while aiming to keep approximation error from completely ruining the model. The rough plan for the series is as follows: (This post) Introducing the Problem- Why is VI useful, why VI can…
Race is a defining part of political identity in the United States, and so it should be no surprise that accurately modeling race can be beneficial for many political campaign activities. For instance, many organizations work to improve turnout in specific communities of color, or want to target persuasion on a given issue to certain racial group. Alternatively, race and ethnicity might be desired…