Imputing and prestidigitation

A polygenic risk score (PRS) is an estimate of an individual’s relative genetic liability to a specific trait or complex disease, made by summing the effects of multiple genetic variants across their genome. It uses genomic information alone—without incorporating lifestyle or environmental factors—to show whether a person has a higher or lower genetic predisposition compared to the general population.

The score is calculated by identifying thousands of common genetic-disease variants, known as single-nucleotide polymorphisms (SNPs), and weighting them on how strongly each variant impacts the condition. A polygenic risk score (PRS) provides a much more comprehensive view of complex health traits than looking at individual single-nucleotide polymorphisms or simple combinations of them.

While a higher score places an individual on the upper end of a risk distribution curve, it only reflects statistical probability rather than a definitive diagnosis or a timeline for when a disease will occur. Yet it is one of our most powerful predictive tools.

The math that underlies a polygenic risk score fits on a napkin. You take a list of genetic variants, each with a weight that came out of some large study; you look up which version of each variant a person carries; you multiply; and you add. That is the whole thing: a weighted sum. Sigma of beta times dosage. A bright high schooler could compute one by hand given an afternoon and enough coffee.

I spent the better part of a week getting my platform to produce a single one of these scores for a single disease, and I want to tell you why, because the distance between the napkin and the working feature turns out to be the entire story of modern bioinformatics. The science is the easy part. The plumbing is where you actually live.

The variants that matter are not on the chip

Opus23, the platform I build for clinicians, already scored consumer genetic data. If a physician uploaded a client’s 23andMe or Ancestry file, the system could match it against the public catalog of published risk scores and hand back a result. For a lot of traits this worked beautifully. And then I started paying attention to which traits, and a pattern emerged that I found genuinely humbling.

How to conjure something

Consumer genotyping chips do not read your whole genome. They read a curated few hundred thousand positions, chosen years ago, mostly common variants. A published risk score, meanwhile, was often built on imputed or sequenced data and leans on SNP variants the chip never measures. So a score can look perfect on paper (small, tidy, clinically validated) and be completely hollow against real consumer data, because the one or two variants carrying most of its predictive weight simply are not there.

Idiopathic pulmonary fibrosis (an area I still find fascinating) was the cleanest example. Its genetics are dominated by a single promoter variant in a gene called MUC5B, a variant so influential it outweighs everything else in the score by a factor of four. On the consumer chips my clients actually use, that variant is absent. It sits in a stretch of DNA that arrays do not read. So the ‘IPF score’ my engine could compute was covering thirteen percent of the score’s information and none of the part that counted. It was a number without relevance, and the honest thing to do is refuse to even show it.

Kidney disease failed the same way (its signal lives in a variant called UMOD that is also off-chip). So did telomere length. I built a little vetting tool to make this judgment automatic, rating each candidate score PASS, MARGINAL, or FAIL based not on how many variants it had but on whether the heavy ones landed on real chips. That tool confirmed what I was starting to feel: a whole tier of the most clinically interesting scores was locked away, and no amount of cleverness with the data I had would open it.

The escape hatch that is free but not cheap

The technical answer to missing variants is imputation (Latin putare: ‘to reckon, clear up, or consider’). You take the few hundred thousand positions a chip does read, and you statistically infer the millions it does not, by comparing the person’s DNA to an enormous reference panel of fully sequenced genomes. Where their chip goes quiet, the reference fills in the most probable genotype. MUC5B, invisible on the chip, comes roaring back through imputation.

There are public servers that do this. The TOPMed Imputation Server, run on NIH infrastructure, will impute a genome against a panel of nearly a hundred thousand people, and it will do it for free. When I first read that, I assumed the catch was money, and there wasn’t one. The catch, I learned, is three other things: consent (a client’s genome now leaves my server for someone else’s cluster, which for a physician tool is a real governance decision and not a checkbox), asynchronicity (imputation is a batch job that takes hours, if not days, not a web request that takes a second), and conversion. That last one nearly ended me.

Share

Building the ends before the middle

I built the payoff first, deliberately, because I wanted proof before I invested in plumbing. I wrote the piece that scores an imputed genome, then fed it a synthetic imputed file containing MUC5B, and watched idiopathic pulmonary fibrosis go from thirteen percent coverage to one hundred. The dominant variant resolved. The dead trait came alive. That was the moment I knew the rest was worth building.

Then came the middle: a converter to turn a raw consumer file into the format the server wants, an orchestrator to submit jobs and fetch results over the server’s API, and a “glue” script to unzip the encrypted results, throw away the low-quality guesses, score every trait, and file the answer where the report could find it. Each piece tested, each piece behaving. I felt good. I was, it turned out, about to meet the conversion problem in person.

The text file that isn’t a text conversion

Here is the thing nobody warns you about. A consumer genotype file and the VCF format that imputation servers require look like the same kind of thing: rows of variants, tab-separated, human-readable. Converting one to the other feels like it should be a five minute reformat. It is not, and the reason is quietly profound.

A genotype file records what you have. It says, at this position, you carry two copies of C (CC). A VCF records how you differ from the reference human genome. It needs to know, at that position, which letter the reference has and which is the variant. And your file cannot tell you that. If you are homozygous C (CC), you know you have C, but you have no idea whether C is the reference base or the change from it. The only thing that knows what that C means is the reference genome itself, all three billion letters of it. So the simple text conversion is actually a lookup against a nine hundred megabyte reference file, done for every one of six hundred thousand positions, and there is no honest shortcut around it. This is why the imputation servers increasingly just say ‘give us a VCF’ and hand the hard part back to you.

The standard tool for this lookup is called bcftools. I did not have it. I did not have a laptop to install it on either. What I had was a locked down web server: no administrator rights, no package manager, no ability to install anything the normal way.

The locked box

This is the part I am oddly proud of. The server was not as sealed as it looked. It had outbound internet, a personal folder that programs would search for commands, and, gloriously, four hundred and eighty gigabytes of free space. That was enough. I bootstrapped a self contained package manager into my home directory (no root required), used it to install bcftools and its friends into a private corner of my own account, downloaded the three gigabyte human reference genome, and indexed it. A machine that half an hour earlier could not convert a file was now a fully equipped bioinformatics workstation, all of it living inside one unprivileged user account.

Then I converted the wrong file.

I pointed the tool at one of my internal master files and got an empty result, because those files, in my own storage format, had long ago discarded the chromosome positions a VCF needs. That cost a few hours. The positions were the thing. I found the original raw download, the real one with its plus strand build 38 coordinates intact, and ran it again.

Six hundred seventy five thousand variants converted. The genotype counts came back exactly as a real genome should look: most positions matching the reference, a healthy fraction heterozygous, fewer homozygous for the variant. That distribution was the proof. If the reference lookup had failed, those numbers would have been garbage. They were pristine.

The spinner

I uploaded the converted file to TOPMed. The same server that, days earlier, had rejected my raw genotype file in six seconds with a curt “no VCF detected,” now showed a small rotating icon. Working. Past the gate. Actually imputing.

That spinner represents a weighted sum that fits on a napkin, wrapped in a week of file formats, missing reference alleles, free-but-async infrastructure, a governance decision, and a package manager smuggled into a locked account. None of it was the science. All of it was the difference between an algorithm and a thing your physician can actually use.