Writing on biology and computers by Alex Carlin
I really enjoyed this Stanford course on deep generative models taught by Stefano Ermon. The teaching was super engaging, the material was interesting and deep, and I learned a lot about the field from the super clear explication and focused presentation. There was a fantastic lecture at the very end by Aaron Lou, a PhD student in Stefano's lab, regarding Aaron's work on diffusion models for…
Evaluating discriminative models is relatively straightforward. In contrast, evaluating generative models is difficult. We can't just hold out a test set and predict labels and calculate accuracy, we need more sophisticated means to tell if our models are any good. We need to be able to focus in on things we really care about. In the case of generative models for protein structure that seek to…
I've been inspired by many great teachers over the years, and one of my favorite courses recently has been Andrej Karpathy's Neural Networks: Zero to Hero series. In this detailed, hands-on series, Andrej shows how to build up a GPT-2 model from scratch in Python code. But what's really cool is that the course starts at the very beginning, by building an autograd engine and then working from there…
If you were a biologist before the combination of new architectures and much better compute made deep learning approaches useful in biology, you might wonder about the vast array of different educational resources out there to help you master the concepts and application of deep learning. You might wonder which resources are really worth your time as you seek to understand classical ML approaches…
Protein language models are large transformers trained on datasets of protein sequences. Where natural language models use a vocabulary made up of "tokens," protein language models use the small, simple vocabulary of the 20 amino acids, plus a couple more characters. You can prompt a natural language model like GPT-2, say, with some tokens "I'm a protein like" and it will complete the text ( "a…
Enzyme design is a bit of a grand challenge in protein science. Let’s just put it this way: enzymes have proven to be incredibly difficult to design effectively. Mayo and coworkers proposed an automated method for de novo enzyme design in 1997 , and more and more smart people have been dealing with the insane challenges posed by the problem of designing proficient catalysts ever since. In 2008,…
So just to start off, there is no problem with ProteinGym, and this post is not to hate on ProteinGym. ProteinGym is awesome. It was a super good idea, it is well-executed, and it’s truly a tremendous diversity of amazing datasets (each of which, of course, was itself a ton of work to produce), from binding affinities, to catalytic rate, to growth rate, and many other kinds of readouts. And that…
Summary We pose the problem of protein sequence design given a backbone structure as a node labeling problem and solve it with graph attention networks (GATs). We use a dataset of structurally non-redundant protein structures and implement graph representation and graph attention networks using PyTorch Geometric. In our experiments, GATs achieve perplexities All code is available on GitHub:…
Over the past few years, we've seen a huge explosion in use of and interest in machine learning for understanding biology. As someone with six years of industry experience running hundreds of campaigns where ML models are used to design proteins, I've developed some mental models around using ML for protein design. Here's a big one. Use ML models for what they’re good at, instead of what they’re…
Today I'm releasing a set of commands that will help anyone be more productive when using PyMOL. As an almost daily user of PyMOL, I have come to deeply appreciate it, but there are some commands that I always wished for, so I made them. Power up your PyMOL with PyMOL PowerPack 🚀 I'm making the code for PyMOL PowerPack available freely on GitHub with just a few of my most-used commands to start…