"Models Demystified" is a blog about data science and machine learning.
Introduction It’s been nearly four years since I first summarized the state of deep learning (DL) for tabular data, and about three years since my follow-up post . Back then, the verdict was clear: for most tabular scenarios, gradient boosting methods like XGBoost, LightGBM, and CatBoost were the pragmatic choice. Complex DL architectures, like TabNet and even the relatively newer…
My book, Models Demystified: A Practical Guide from Linear Regression to Deep Learning , co-authored with Seth Berry, is now published! It was a long journey, but we’re excited to share it with you. We started in mid-2023 with the goal of ending by the end of that year. That naivety was quickly dashed, but we ended up with a much more comprehensive book than we originally planned when all was said…
As with the last post on class imbalance , this is a resurrection of sorts. I began this post a little after that one had gotten an initial draft completed. Since then, conformal prediction has caught on quite a bit, but there wasn’t much at the time in terms of tools. I was focused on MAPIE as that’s a good package in Python, and as I was writing this, probably finally offered something more…
This post began around two years ago and was initially intended for the Strong Analytics blog, but for various reasons never came to fruition. We still thought the spirit of the content was valuable, so we did what we could to get it out one way or another. Both the intended audience and focus have changed over time, but hopefully the end result is cohesive enough to be useful to a wide audience.…
Update Among many things that have happened recently, I’ve become a father to a beautiful baby girl, and written a book to come out this year (2025). Fun stuff! Elsewhere, my employer Strong Analytics merged with OneSix last summer, which has gone well so far. It’s been great to expand our capabilities and personnel, and I’m excited to see what we can do in the future. Not so fun - I’ve migrated…
Book in progess TLDR: https://m-clark.github.io/book-of-models Been a long time since I posted. Part of this was due to the fact that I had almost completely transferred my site to quarto, but then never got around to finishing it. That will happen eventually, and I will hopefully start posting again at that point. But the real news is that I am working on a new book. It is a book on exploring…
It’s been a bit so thought I’d force myself to post a couple things I’ve played around with, or that aren’t ready yet for a full post, or won’t be one. Football players still don’t know penalty kick basics Did a quick and dirty Bayesian analysis to get posterior probabilities for location, controlling for various factors. As a side note, I won the office world cup challenge with a fancy model of…
Introduction NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . This post gives a by-hand example of a linear model using pytorch . A good question to ask right off the bat would be- why would anyone do this? We use deep learning typically…
NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . Intro This post was mostly complete around May 2021, but for various reasons not actually posted until August of 2022. I haven’t changed much aside from adding a section on boosting, and…
NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . Introduction Oftentimes I’m looking to gain speed/memory advantages, or maybe just exploring how to do the same thing differently in case it becomes useful later on. I thought I’d start…
Introduction In a previous post, I offered a summary of several articles that came out over the summer of 2021 regarding the application of deep learning (DL) methods to tabular data. DL has shown astounding success in the natural language processing, computer vision, and other fields, but when it comes to the sorts of data common in other situations, especially where data is usually smaller and…
A co-worker passed along a recent article (Dar, Muthukumar, and Baraniuk 2021) on the topic of double descent in machine learning. I figured I’d summarize some key points I came across while perusing it and some referenced articles. In addition, I’ll provide an accessible example demonstrating the phenomenon. What is double descent? Bias-variance trade-off To understand double descent you have to…
Motivation I’ve been a little perplexed at the lack of attention of deep learning (DL) toward what I consider to be ‘default’ data in my world, often referred to as tabular data , where typically we have a two dimensional input of observations (rows) and features (columns) and inputs are of varying type, scale and source. Despite the ubiquity of such data in data science generally, and despite…
NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . Overview Bayesian analysis takes some getting used to, but offers great advantages once you get into it. While it can be difficult to get started, it typically should not take much to repeat…
NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . Overview In Part I , we talked about the basics one can do to run a Bayesian model with a high-level Stan package like brms , and what to do if there is a problem. But it might be nice if we…
New Book I’ve completed a new bookdown document, Models by Example , that converts most of the code from my Miscellaneous R repo. I initially just wanted to update the code, but decided to use a more formal approach to make it cleaner and more accessible. It’s mostly complete, though may be added to on rare occasion, and further cleaned as I find annoying bits here and there. Each topic contains…
NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . Introduction Every once in a while, it comes up that someone has clustered data, with covariates that vary at different levels, and where mixed models or similar would normally be…
Introduction In R there are many tools available to help you dive in and explore your data. However, in consulting I still see a lot of people using base R’s table and summary functions, followed by a lot of work to get the result into a more presentable format. My own frustrations led to me creating a package ( tidyext ) for personal use in this area. While that suits me fine, there are tools…
NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . Also for this particular post, Stata is not something I’ve used in years nor have access to. So for those interested who do have access, I can only show you the code, since at the time, it…
NB: This post was revisited when updating the website early 2025, and some changes were required. Attempts to keep things consistent were made, but if you feel you’ve found an issue, please post it at GitHub . Last updated January 02, 2025. Prerequisites: familiarity with factor analysis Introduction The psych package is a great tool for assessing underlying latent structure. It can provide…