Recent content on A. Solomon Kurz
This is the first in a new series discussing causal inference with experimental data using multilevel models. Welcome to the next beginning In 2023 I released a 9-part series on causal inference for randomized experiments. We focused on real-world data from several published experiments, and practices fitting models with a variety of likelihood functions from the GLM. In this series, we expand to…
Center the \(x\) In the last post, we fit a model with a single continuous predictor using the very non-default brms syntax bf(y ~ 1 + x, center = FALSE). We also had some difficulty settling on a sensible prior for the intercept in this context, given how all our predictor values were far above zero. In this post we’ll explore different ways to fit the model with a mean-centered predictor, what…
We can predict In the first post of this series, we focused on a single-level intercept-only model. Even in that context, the Stan code used by brm() included several sophisticated features. In this post we expand the model to include a single continuous predictor. Along the way, we review matrix notation, model matrices, and a new class of distribution functions for Stan. Load the primary R…
Stan code can be hard Though I’m an ardent brms user, I’ve been learning more about Stan for another project. One of the ways you can learn more about Stan is by reviewing the Stan code underlying your brms models. But I’m guessing that if you’ve ever tried that, you discovered Paul Bürkner’s Stan code can be challenging to read. The purpose of this post, and the next couple posts to come, is to…
Prerequisites We assume familiarity with R and RStudio. Outcomes After completing this course, the participants will: have a basic professional website, built primarily with Quarto, with files stored on GitHub, and hosted by Quarto Pub. Supporting materials You can find supporting materials from the workshop on the OSF at https://osf.io/es8bv/.
Context I have a project for work where the goal is to compare two non-randomized groups. One group received an experimental intervention, and we’d like to compare those folks to their peers who were not offered that same intervention. We have easy access to data from folks from the broader population during the same time period, so the goal isn’t so bad as far as quasi-experiments go.1 The team…
Sometimes in the methodological literature, models for continuous outcomes are presumed to use the Gaussian likelihood. In the sixth post of this series, we saw the gamma likelihood is a great alternative when your continuous data are restricted to positive values, such as in reaction times and bodyweight. In this ninth post, we practice making causal inferences with the beta likelihood for…
So far in this series, we have used the posttreatment scores as the dependent variables in our analyses. However, it’s not uncommon for researchers to frame their questions in terms of change from baseline with a change-score (aka gain score) analysis. The goal of this post is to investigate whether and when we can use change scores or change from baseline to make causal inferences. Spoiler: Yes,…
Books At present, all of my books share a similar format and goal. I am a fan of applied Bayesian statistics. In recent years, scholars have released user-friendly Bayesian software packages and have published reasonably-accessible introductory books on applied Bayesian analysis. These have been boons for us all. In my experience, Bayesian methods are easiest to use within the R computing…
We social scientists love collecting ordinal data, such as those from questionnaires using Likert-type items.1 Sometimes we’re lazy and analyze these data as if they were continuous, but we all know they’re not, and the evidence suggests things can go terribly horribly wrong when you do ( Liddell & Kruschke, 2018). Happily, our friends the statisticians and quantitative methodologists have built…
So far the difficulties we have seen with covaraites, causal inference, and the GLM have all been restricted to discrete models (e.g., binomial, Poisson, negative binomial). In this sixth post of the series, we’ll see this issue can extend to models for continuous data, too. As it turns out, it may have less to do with the likelihood function, and more to do with the choice of link function. To…
In the third post in this series, we extended out counterfactual causal-inference framework to binary outcome data. We saw how logistic regression complicated the approach, particularly when using baseline covariates. In this post, we’ll practice causal inference with unbounded count data, using the Poisson and negative-binomial likelihoods. We need data We’ll be working with a subset of the…
In the first two posts of this series, we relied on ordinary least squares (OLS). In the third post, we expanded to maximum likelihood for a couple logistic regression models. In all cases, we approached inference from a frequentist perspective. In this fourth post, we’re finally ready to make causal inferences as Bayesians. We’ll do so by refitting the Gaussian and binomial models from the…
So far in this series, we’ve been been using ordinary least squares (OLS) to analyze and make causal inferences from our experimental data. Though OLS is an applied statistics workhorse and performs admirably in some cases, there are many contexts in which it’s just not appropriate. In medical trials, for example, many of the outcome variables are binary. Some typical examples are whether a…
One of the nice things about the simple OLS models we fit in the last post is they’re easy to interpret. The various \(\beta\) parameters were valid estimates of the population effects for one treatment group relative to the wait-list control.1 However, this nice property won’t hold in many cases where the nature of our dependent variables and/or research design requires us to fit other kinds of…
Welcome to the beginning This is the first post in a series on causal inference. Our ultimate goal is to learn how to analyze data from true experiments, such as RCT’s, with various likelihoods from the generalized linear model (GLM), and with techniques from the contemporary causal inference literature. We’ll do so both as frequentists and as Bayesians. I’m writing this series because even though…
⚠️ This workshop has already come and gone. You can find some of the original marketing descriptions, along with a link to the online supporting materials, below. Overview We are entering the Golden Age of Bayesian statistics. Thanks to fast personal computers and powerful free software (e.g., Stan), working scientists can fit an array of Bayesian models tailored to their specific needs. Recent…
Courses 2022 Fall semester Statistical Analysis (PA 652) For: The Chicago School of Professional Psychology, Los Angeles Where: Online Goal: This course is designed to introduce the generalized linear mixed model (GLMM) to doctoral students in behavior analysis. We start the semester with simple regression via ordinary least squares, introduce the generalized linear model with maximum likelihood…
Purpose Once again, it was time to update my website. This time I am switching to the Hugo Apéro (a-pay-ROH) theme, by the great Alison Hill. The purpose of this post is to highlight some of the steps I took to rebuild my website. At a minimum, I’m hoping this post will help me better understand how to set up my website the next time it needs an overhaul. Perhaps it will be of some help to you,…
Support Yes, you can support me Every so often, people ask how they might support the work I do with my tutorial blog posts and ebooks. Here are some options: If you like my material, give it a shout out on social media! You can find me on Twitter or Bluesky. If my material directly helped your scientific work, consider citing them. You can find tips on how to cite blog posts in APA 7 style here.
What? We psychologists analyze a lot of sum-score data. Even though it’s not the best, we usually use the Gaussian likelihood1 for these analyses. I was recently in a situation were I wanted to model sum-scores from a new questionnaire and there was no good prior research on the distribution of the sum-scores. Like, there wasn’t a single published paper reporting the sample statistics. Crazy, I…
What? All the players know there are three major ways to handle missing data: full-information maximum likelihood, multiple imputation, and one-step full-luxury1 Bayesian imputation. In an earlier post, we walked through method for plotting the fitted lines from models fit with multiply-imputed data. In this post, we’ll discuss another neglected topic: How might one compute standardized regression…
What/why? This is a follow-up to my earlier post, Notes on the Bayesian cumulative probit. If you haven’t browsed through that post or if you aren’t at least familiar with Bayesian cumulative probit models, you’ll want to go there, first. Comparatively speaking, this post will be short and focused. The topic we’re addressing is: After you fit a full multilevel Bayesian cumulative probit model of…
Abstract Background: The USA is undergoing a suicide epidemic for its youngest Veterans (18-to-34-years-old) as their suicide rate has almost doubled since 2001. Veterans are at the highest risk during their first-year post-discharge, thus creating a “deadly gap.” In response, the nation has developed strategies that emphasize a preventive, universal, and public health approach and…
Version 1.1.0 Edited on January 2, 2023, to acknowledge the related simulation study by O’Connell et al. ( 2017). See footnote #3 for details. What? If you’d like to study the effectiveness of a clinical treatment, one of the simplest and most widely used approaches it to recruit participants from the target population, measure the outcome variable during a pre-treatment assessment, randomly…
Abstract Research on moral elevation has steadily increased and identified several psychosocial benefits that bear relevance to both the general population and people with psychological distress. However, elevation measurement is inconsistent, and few state-level measures have been created and critically evaluated to date. To address this gap, the State Moral Elevation Scale (SMES) was developed…
Preamble In an earlier post, I gave an example of what a power analysis report could look like for a multilevel model. At my day job, I was recently asked for a rush-job power analysis that required a multilevel model of a different kind and it seemed like a good opportunity to share another example. For the sake of confidentiality, some of the original content will be omitted or slightly altered.
Abstract Lifestyle behaviors such as exercise, sleep, smoking, diet, and social interaction are associated with depression. This study aimed to model the complex relationships between lifestyle behaviors and depression and among the lifestyle behaviors. Data from three waves of the Midlife in the United States study were used, involving 6898 adults. Network models revealed associations between the…
Abstract Objective: Posttraumatic stress disorder (PTSD) is a common problem for veterans. Resilience, the tendency to bounce back from difficult circumstances, is negatively associated with posttraumatic cognitions (PTCs) among individuals with a history of trauma, and thus it may be important to understand responses to trauma reminders. Methods: Using a quasi-experimental design, we examined the…
What/why? Prompted by a couple of my research projects, I’ve been fitting a lot of ordinal models, lately. Because of its nice interpretive properties, I’m fond of using the cumulative probit. Though I’ve written about cumulative logit ( Kurz, 2020b, sec. 11.1) and probit models ( Kurz, 2020a, Chapter 23) before, I still didn’t feel grounded enough to make rational decisions about priors and…
Scenario You’re an R ( R Core Team, 2022) user and just fit a nice multilevel model to some grouped data and you’d like to showcase the results in a plot. In your plots, it would be ideal to express the model uncertainty with 95% interval bands. If you’re a Bayesian working with Stan-based software, such as brms ( Bürkner, 2017, 2018, 2022), this is pretty trivial. But if you’re a frequentist and…
Version 1.1.0 Edited on December 12, 2022, to use the new as_draws_df() workflow. Preamble After tremendous help from Henrik Singmann and Mattan Ben-Shachar, I finally have two (!) workflows for conditional logistic models with brms. These workflows are on track to make it into the next update of my ebook translation of Kruschke’s text (see here). But these models are new to me and I’m not…
Version 1.1.0 Edited on October 9, 2022. Doctoral candidate Reinier van Linschoten kindly pointed out a mistake in my R code for \(V_B\), the between imputation variance. The blog post now includes the corrected workflow. What? If you’re in the know, you know there are three major ways to handle missing data: full-information maximum likelihood, multiple imputation, and one-step full-luxury1…
What When you fit a logistic regression model, there are a lot of ways to display the results. One of the least inspiring ways is to report a summary of the coefficients in prose or within a table. A more artistic approach is to show the fitted line in a plot, which often looks nice due to the curvy nature of logistic regression lines. The major shortcoming in typical logistic regression line…
Abstract Although there is growing interest in the neural foundations of aesthetic experience, it remains unclear how particular mental subsystems (e.g. perceptual, affective and cognitive) are involved in different types of aesthetic judgements. Here, we use fMRI to investigate the involvement of different neural networks during aesthetic judgements of visual artworks with implied motion cues.…
Version 1.1.0 Edited on December 12, 2022, to use the new as_draws_df() workflow. Preamble Suppose you’ve got data from a randomized controlled trial (RCT) where participants received either treatment or control. Further suppose you only collected data at two time points, pre- and post-treatment. Even in the best of scenarios, you’ll probably have some dropout in those post-treatment data. To get…
Abstract A starting point of many digital health interventions informed by the Stages of Change Model of behavior change is assessing a person’s readiness to change. In this paper, we use the concept of readiness to develop and validate a prediction model of health-seeking behavior in the context of family planning. We conducted a secondary analysis of routinely collected, anonymized health…
Version 1.1.0 Edited on December 12, 2022, to use the new as_draws_df() workflow. What? One of Tristan Mahr’s recent Twitter threads almost broke my brain.1 It turns out that you can use random effects on cross-sectional count data. Yes, that’s right. Each count gets its own random effect. Some people call this observation-level random effects and it can be a tricky way to handle overdispersion.…
Context In one of my recent Twitter posts, I got pissy and complained about a vague power-analysis statement I saw while reviewing a manuscript submitted to a scientific journal. If you submit a manuscript for publication that involves HLMs and SEMs of longitudinal data and you vaguely summarize your power analysis in one sentence, I, as your friendly neighborhood Reviewer #2, am requesting a full…
Version 1.1.0 Edited on December 12, 2022, to use the new as_draws_df() workflow. Context Someone recently posted a thread on the Stan forums asking how one might make item-characteristic curve (ICC) and item-information curve (IIC) plots for an item-response theory (IRT) model fit with brms. People were slow to provide answers and I came up disappointingly empty handed after a quick web search.…
tl;dr When your MCMC chains look a mess, you might have to manually set your initial values. If you’re a fancy pants, you can use a custom function. Context A collaborator asked me to help model some reaction-time data. One of the first steps was to decide on a reasonable likelihood function. You can see a productive Twitter thread on that process here. Although I’ve settled on the…
Purpose Just this past week, I learned that, Yes, you can fit an exploratory factor analysis (EFA) with lavaan ( Rosseel, 2012; Rosseel & Jorgensen, 2019). At the moment, this functionality is only unofficially supported, which is likely why many don’t know about it, yet. You can get the [un]official details at issue #112 on the lavaan GitHub repository ( https://github.com/yrosseel/lavaan). The…
Purpose A few weeks ago, I was preparing to release the second blog post in a two-part series (you can find that post here). During the editing process, I had rendered the files into HTML and tried posting the draft to my website. Everything looked fine except that the figures wouldn’t render. I hadn’t seen this behavior before and I figured it had to do with some software update. When I checked,…
Version 1.1.0 Edited on December 12, 2022, to use the new as_draws_df() workflow. Orientation This post is the second and final installment of a two-part series. In the first post, we explored how one might compute an effect size for two-group experimental data with only \(2\) time points. In this second post, we fulfill our goal to show how to generalize this framework to experimental data…
Background This post is the first installment of a two-part series. The impetus is a project at work. A colleague had longitudinal data for participants in two experimental groups, which they examined with a multilevel growth model of the kind we’ll explore in the next post. My colleague then summarized the difference in growth for the two conditions with a standardized mean difference they called…
Version 1.1.0 Edited on December 16, 2022, to use the new as_draws_df() workflow. Purpose In the contemporary longitudinal data analysis literature, 2-timepoint data (a.k.a. pre/post data) get a bad wrap. Singer and Willett ( 2003, p. 10) described 2-timepoint data as only “marginally better” than cross-sectional data and Rogosa et al. ( 1982) give a technical overview on the limitations of…
Version 1.1.0 Edited on December 16, 2022, to use the new as_draws_df() workflow. The set-up PhD candidate Huaiyu Liu recently reached out with a question about how to analyze clustered data. Liu’s basic setup was an experiment with four conditions. The dependent variable was binary, where success = 1, fail = 0. Each participant completed multiple trials under each of the four conditions. The…
Abstract Research on moral elevation has steadily increased and identified several psychosocial benefits that bear relevance to both the general population and people with psychological distress. However, elevation measurement is inconsistent, and few state-level measures have been created and critically evaluated to date. To address this gap, the State Moral Elevation Scale (SMES) was developed…
Version 1.1.0 Edited on December 12, 2022, to use the new as_draws_df() workflow. Preamble In Section 14.3 of my ( 2020b) translation of the first edition of McElreath’s ( 2015) Statistical rethinking, I included a bonus section covering Bayesian meta-analysis. For my ( 2020a) translation of the second edition of the text ( McElreath, 2020), I’d like to include another section on the topic, but…
Abstract Coping with food cravings is crucial for weight management. Individuals tend to use avoidance strategies to resist food cravings and prevent overeating, but such strategies may not result in the benefits sought. This study compared the effects of two cognitive techniques (Restructuring vs. Defusion) for dealing with food cravings in terms of their impact on healthy vs. unhealthy eating…