Psychopathology structure changes with severity

A new preprint by the amazing Ashley Watts (and gang) argues that the structure of higher-order psychopathology domains such as internalizing and externalizing — that is, that such domains are correlated — arises from what they call nomorbidity, not comorbidity.

The authors demonstrate this by applying Bayesian change point models to four waves of the Adolescent Brain Cognitive Development Study data in a large sample of over 10k youths, and analyzing associations among four higher-order psychopathology dimensions, with a focus on two: externalizing and internalizing. If you’re not familiar with mental health research, you can think of these as something akin to big personality facets or facets of cognitive abilities (e.g. verbal vs mathematical): they are large summaries of a lot of different types of psychopathology. Internalizing disorders contain diagnoses such as Major Depressive Disorder, Generalized Anxiety Disorder, panic disorders, phobias, and eating disorders. Externalizing disorders, on the other hand, include Attention-Deficit/Hyperactivity Disorder (ADHD), Oppositional Defiant Disorder (ODD), Conduct Disorder, Antisocial Personality Disorder, and substance use disorders.

Correlations everywhere

The correlation between internalizing and externalizing disorders of around r=0.5 is among the most robust phenomena in mental health science, and thus requires explanation. Unsurprisingly, entire research programs and classification frameworks (such as HiTOP) have been built to explain it. A common view is that the correlation reflects comorbidity: people have both types of conditions.

And this explanation of comorbidity is usually argued to have one of two root causes. A lot of folks believe that there are shared underlying mechanisms that cause both types of conditions, with the mysterious p-factor among the most prominent explanations. You can find out more about this thing here:

Specifically, as we write in the paper, the p-factor has been argued to reflect a number of different processes:

“Researchers have proposed numerous substantive explanations of the p-factor, including: unspecified causal mechanisms, deficits in intellectual functioning, disordered thought, negative emotionality and emotion dysregulation. We consider each of these explanations interesting and expand on them below, but we acknowledge that they are, at best, weak theories. Each of the following explanations is fledgling, arguably underspecified, is difficult to falsify and has not yet been subjected to risky or otherwise scrutinous tests.”

The alternative explanation is that problems attract problems — domains are correlated because one problem causes other problems, according to the network or systems conceptualization — which has its own problems, of course.

As so often, the truth is in the middle: there are common mechanisms that make people more vulnerable, and problems cause other problems. But these explanations are not the focus of the current post, so let’s leave them behind for now.

But is there a correlation in the first place?

Watts et al. question whether there is a correlation in the first place, that is, whether there is a comorbidity phenomenon which requires explanations such as the p-factor. They argue that the correlation is largely derived from the combination of two things: (1) around 70% of general population samples meet criteria for 0 mental disorders, which they call nomorbidity, and (2) nearly all work on the structure of psychopathology is based on such general population samples with heavily zero-inflated symptom counts. We’ve actually shown this in a nice 2016 paper, which I’ll return to later.

Here, instead of reiterating their methods and results, I’ll take a different path: a quick simulation study.

Let’s simulate

The simulation below shows that even in a world where there is zero comorbidity, externalizing and internalizing domains would be correlated around 0.5.

Like Watts et al., I assume that general population samples that most of the literature is based on are a mixture of two populations: people with issues, and those without. This is a simplification, but not an outrageous one. We now create both groups separately, and they differ from each other only in the amount of symptoms they have: high vs low counts. And then we decide that there are fewer people in the mental disorder group, and more in the healthy group. You can adjust all these, but we start with what I consider reasonable assumptions.

Next, we create a scenario in which there is no comorbidity by making the two domains in the smaller psychopathology subsample uncorrelated: internalizing and externalizing are orthogonal. And in the asymptomatic group, domains are also uncorrelated, because most people simply have zero. But overall, due to the larger number of low observations, a correlation around 0.5 is induced in the whole population.

The simulation is intentionally simple, but it illustrates why the population mixture matters.

Open the simulation in a new page

Our 2016 precursor paper on the topic

During my first postdoc, I actually wrote a paper in which we highlight this phenomenon in some detail. I was practicing network analyses at the time, and realized that when you take depression clinical trial data, the networks look really different at the start of the clinical trial (when symptoms are high) and the trial endpoint. In other words, the correlations among depression symptoms change dramatically in a population that is first fairly ill, and then gets better over time.

We ended up documenting this in two large prospective studies (total N=3,509) in which overall depression levels decreased, examining 4 common depression rating scales (1 self-report, 3 clinician-report) with different time intervals between assessments (between 6 weeks and 2 years), and the phenomenon emerged very robustly.

I’ll first show you the distributions of the total scores for the four scales, where you can see that they decrease:

And next, here are the psychometric scale properties. You can see inter-item correlations go up, and the scales become more unidimensional (Cronbach’s alpha goes up):

We also show that the questionnaires are much more multifactorial at the first time point (models with 3-5 factors fit best), whereas fewer factors are required at the exit timepoint where psychopathology is better.

This is highly consistent with the phenomenon Watts et al. observe in their newest preprint. Core summary:

“Specifically, we found that the means of the total scores systematically decreased while their variances increased, the factor structure of the depression scales became less multifactorial (but not unidimensional), and reliability increased as the intercorrelation among items increased. Of note, the results were consistent across two different samples, various time frames ranging from 6 weeks to 2 years, and across four self-report and clinician-rated scales.”

When I dug into the literature back then, I discovered the most amazing thing: this phenomenon was widely known to scale developers. And I know that because if you look at the original scale development papers that often took place in clinical trials, you will find that authors usually only report Cronbach’s alpha at the exit timepoint of the clinical trial, without justification. Now we know why: because it was much worse at the entry timepoint. Page 9 of the paper has a detailed overview of the literature on this, and the phenomenon, while not previously written up as such, is very clear looking at the depression literature. We conclude the paper by describing a number of potential mechanisms that could explain this change of inter-item correlations over time, including changes in variances, response shift bias, restriction of range, selection effects, regression to the mean, and more, but conclude that none of these likely sufficiently explain the phenomenon and call for further work: “While we have provided a thorough description of the crime scene, we have no good idea who the main suspect may be”.

I’m very excited Watts et al. now provide a transdiagnostic version of this phenomenon: when symptom severity changes, the structure of psychopathology changes, too. I hope the quick simulation above helps.

---