Recent content on Stephen Malina
Earlier today (on August 7th), I realized that I was about to miss a golden opportunity to quickly test how calibrated I was about AI progress by predicting some GPT-5 benchmark scores. Upon realizing this, I dashed off some quick predictions based on a combination of then-current top benchmark scores and my intuition about how big a jump GPT-5 would be over o3, Grok 4, Claude Opus 4, and Gemini…
In September 2022, I wrote a post describing a set of “omens” that, if observed, would indicate faster-than-expected AI progress. I framed many of these in the context of 0-5 years, but obviously AI moves too fast to wait until 2027 to review these. Consider this post an arbitrarily timed intermediate check in triggered by my sense that many of my omens had come to pass. The rest of…
Introduction # In the spirit of writing for the LLMs , I thought I’d write up some of the software debugging heuristics and tacit knowledge I have gotten good mileage out of in my career so far. How I would approach this post as a human reader # Part of the reason people don’t write about tacit knowledge is because it’s by definition the knowledge that’s impossible to fully…
A shortlist of my favorite posts, organized by topic. AI & Technology # AI Omens - Reflecting on early signs of AI’s rapid progress from 2022. GPT-4 Predictions - My predictions about GPT-4 before its release. Decentralization of Atoms is Underrated - Why physical decentralization matters for the future Biology & Biotech # Biologizing the Stack - Imagining a future where biology replaces…
I’m experimenting with writing some shorter, less polished posts. These come with the disclaimer that I’m optimizing for getting thoughts out there while they’re still crystallizing rather than achieving maximum rigor. For this post in particular, I’m not arguing that this direction is definitely worth pursuing so much as exploring a frame/idea to see where it gets me. A…
What are continuous glucose monitors good for? More broadly, what are continuous measures of physiology ( CxM ) good for? Similar to Elliot and I’s dialogue , this post has an unconventional format. It’s both a synthesis of Willy ’s, Eryney’s , and my opinions on continuous monitoring combined with a debate in places where we disagree. To try and make it clear which views…
Summary # Gene editing has made a lot of progress over the past few decades. CRISPR-Cas9 sparked the recent fervor and won some of its early pioneers a Nobel Prize , and more recently Prime Editing and Cytosine/Adenine Base Editors (CBEs/ABEs) have further expanded the range of editing targets and types of edits that are possible while increasing efficiency and reducing the rate of off-target…
Introduction # A few months ago, Elliot Hershberg, one of my favorite Substack writers, wrote a post laying out his personal mission. I really enjoyed his post and, as is customary in blogging, the best way I could think of to compliment it was to share some (hopefully constructively) critical thoughts on it. After sending my post to Elliot, he proposed the great idea of us having a discussion of…
2022 was another exciting, fun year. To round it out, I wrote up some scattered reflections. Enjoy, or if you aren’t a fan of this genre of post, don’t! Also, each section is mostly self-contained and they’re not organized in any special order, so don’t hesitate to skip a section if it sounds boring. Writing # 2022 felt like a productive year for blogging even though my…
This post was co-authored with Austin Vernon . Summary # While the internet often focuses its furious debate energy on decentralization in the realm of bits, in the realm of atoms, decentralization is currently underrated in terms of its feasibility and desirability. New technologies in areas like aviation, freight, and life sciences can and are shifting the scales towards decentralized models…
As a way to keep myself honest, I’m going to be recording GPT4 predictions here and archiving versions of the post so I can’t go back and change it without people knowing. Not that I would do this anyway, but it’s been a tough week for “trust me, I’m telling the truth” so I’m doing this in case readers want the receipts. I predict GPT4 won’t be able…
A friend recently asked me what my “ AI timelines ” were. Especially with recent progress in image generation , language modeling , code generation , robotics , protein folding , and other areas, this has become an increasingly popular question amongst those who are paying attention. However, I find trying to answer this question pretty frustrating. There already exist long, thorough…
Since I’ve been going around telling everyone how Top Gun: Maverick (TG:M) is one of the best action movies of our generation, I’d figured I’d try and write down some of my thoughts while they’re still fresh. The core reason I love TG:M is that it’s an unapologetic paean to excellence, Captain Pete “Maverick” Mitchell (Maverick) embodies excellence as a…
Introduction # Generally when someone tells you they’ve invented an “art of mind-using” that involves lots of made-up words and comes packaged with a spiritual philosophy, you should tell them “no thanks” and walk away. But, what if that same would-be crackpot was also an incredibly successful inventor whose contemporaries considered a genius? Moreover, what if that…
Introduction # Along with a few internet blogger friends, inspired by Erik Hoel’s post (parts of which I disagree with, including the headline), I’ve decided to make predictions for 2050. The rest of this post discusses some meta aspects of making these predictions and then lists out the predictions. In case you’re waiting for my (hopefully) less speculative 2022 annual…
At the beginning of last year, I made predictions for the year. Now it’s time to score them (before I make my 2022 predictions). Data & code # Here is a spreadsheet containing my scored predictions. If you want to see my full analysis, you can look at my Jupyter notebook which contains Brier score calculations plus plotting. High level analysis # The key metrics I looked at were my overall…
Note: This post was co-written with Uri Bram of The Browser , who deserves all credit for the good parts and no blame for the bad parts and inevitable mistakes (which things fall into which category is at your, the reader’s, discretion). A recurring problem I, and I think people similar to me, face is letting my excitement about something new transform from being based on curiosity and a sense of…
One thing I like about the synthetic biology community is that, from the early days, its participants have been willing to entertain “wild” ideas like building dragons . While this willingness comes with all of the risks associated with building a lot of hype around a nascent technology, against the backdrop of cultural indefinite pessimism, I still prefer it. In that vein, inspired by…
Reading biographies and observing friends, family, and colleagues has led me to become interested in what factors drive the variance in cognitive stamina and observed levels of energy between individuals. Identifying the biological, environmental, or motivational factors which produce this difference seems important and neglected. Understanding this is a research agenda’s worth of work, so…
A World Without Email (AWWE) is Cal Newport’s new book arguing for an overhaul in our workplace communication and project management style. This topic is a natural follow up to Newport’s previous two books – Deep Work and Digital Minimalism. These books focused on the individual benefits of hard focused effort and deliberate technology reduction respectively. This book applies a…
I recently had someone ask me what I felt helped me grow the most in my first few years working as a software engineer. Upon them asking me this, I initially drafted an answer that looked something like the following: A combination of building systems and seeing how they failed and then reading lots of code and books including: (list of programming books I’ve read). However, upon writing…
Originally posted on Alexey’s blog. By Stephen Malina , Alexey Guzey , Leopold Aschenbrenner . Introduction # Conventional startup and business wisdom has become “there are plenty of good ideas, all that matters is execution.” We don’t buy it. Empirically, all our aspiring founder friends are desperate for startup ideas, with few managing to land on something worthwhile, and all our…
I just went on a short nighttime mid-snow storm walk and it unexpectedly provoked three separate observations along with a desire to capture them. Reveling in the beauty of our creations # Everyone loves to hate on Ayn Rand but noone else can give me ASMR writing about skyscrapers. I would give the greatest sunset in the world for one sight of New York’s skyline. Particularly when one…
This is a review of The Art of Learning: A Journey in the Pursuit of Excellence by Joshua Waitzkin. I’m sort of surprised how much I liked this book. At times, it bordered on self-help, spiritual BS but Waitzkin’s undeniable competitive success and tempering of the more spiritually flavored advice with insightful musing on how to master a domain or understand one’s opponent…
As someone who has and continues to derive tremendous value from writing on the internet, I’ve recently felt an urge to share some of my favorite semi-esoteric internet writing in part so that I have a post to refer back to instead of resending the same articles to friends over and over again. Since many of these articles resist easy categorization, I’m organizing this in the form of a…
For the past two years, my friends have had a tradition of making predictions and scoring them at the end of the year. This year, I decided why not just post my predictions on my blog and score them publicly (with a few redacted)? This will hopefully incentivize me to be more thoughtful about them and also will help direct my endless desire for internet acclaim towards something that has a…
Previously: Upcoming Maniac Week . Summary # Inspired by Nick Winter , Bethany Soule , Danny Reeves , Brent Yorgey , and Alex Strick , I undertook a “maniac week” from Friday, October 9th (9:00 AM) through Thursday, October 15th (12:00 PM). During my maniac week, I worked 11+ hours a day on schoolwork and my primary research project while avoiding social media and other internet…
Abstract # Guy on internet decides to follow lead of other people on internet and pre-commits to working much more than usual . Plans to document whole thing under illusion people care about some guy on the internet working a more-than-average number of hours relative to his baseline. Summary # From Friday, October 9th (9:00 AM) through Thursday, October 15th (2:00 PM), I’ll attempt to work…
Note: Ignore the weird Markdown formatting, it’s because I copied this from the blog source code. Inspiration : Understanding by Nabeel Qureshi, David Deutsch’s The Beginning of Infinity and Fun Criterion . Meta : This is an experimental post in which I dissect something about myself rather than write about some technical topic. Feedback and criticism encouraged as always! One commonality…
This semester, I had the pleasure of taking a course on Causal Inference with Professor Elias Bareinboim , one of Judea Pearl’s former students and author of some seminal results in Causal Inference. I feel like this course highlighted not only the content Causal Inference researchers consider important but also the types of understanding they value. In this review, I’ll focus on the…
Intro / Motivation # I know I’m a walking millenial trope, but I’m starting a deliberate experiment to dramatically reduce my internet engagement. Especially while I’m quarantined for who knows how long, I want to almost entirely stop: checking Twitter; reading about the virus throughout the day ; checking email; reading blogs frequently; and constantly texting with people…
Attention conservation notice: Narrow target audience - will only make sense to people somewhat familiar with causal inference who don’t find the result entirely boring. The Front-Door Criterion # Suppose we have a causal graphical model that looks like the following. Assume $ U $ is unmeasured whereas $ X, M, Y $ can be measured. Notice that: All directed paths from $ X $ to $ Y $ flow…
Abstract # I conducted a two-week blinded, randomized experiment to test whether drinking regular vs. decaf coffee had a detectable effect on my mood and alertness. Each day, I drank one cup of coffee in the morning and immediately took a Quantified Mind test. I also tracked hours of sleep each night and measured subjective factors such as mood and alertness a few times throughout the day.…
Selected Exercises # 1. Suppose we play a game where we start with $ c $ dollars. On each play of the game you either double or halve your money, with equal probability. What is your expected fortune after $ n $ trials? I’m pretty sure we can use a trick that I’ve seen a lot in machine learning to solve this using plain old expectations. If I’m right, we let $ X \sim…
In which I record my thoughts on Network Mendelian Randomization by Burgess et al. What is this paper about? # This paper describes a method for doing Mendelian Randomization (MR) in the presence of a potential mediating variable. In the typical MR setting, we have an instrumental variable which “instruments” an exposure that we believe causally influences the outcome we care about.…
Causal Inference in Statistics # Questions # Why is the causal effect identifiable in an IV DAG when the dependencies are linear (from an SCM perspective)? I think this may relate to factor models. Advanced Data Analysis from an Elementary Point of View (Ch. 18-23) # Chapter 18 # Exercises # 18.2. Proof that every path must go through a collider: Observe that every path between two exogenous…
Summary # Detailed Notes # 3 IV Requirements # IV must have a direct influence on the treatment IV must not covary with the unmeasured confounding that impacts the outcome IV must not have a direct influence on the outcome Relevant considerations for deciding whether to do an IV analysis # Is there any unmeasured confounding? If not, is the answer that we can just use a normal regression? When is…
Matrix Potpourri # As part of reviewing Linear Algebra for my Machine Learning class, I’ve noticed there’s a bunch of matrix terminology that I didn’t encounter during my proof-based self-study of LA from Linear Algebra Done Right . This post is mostly intended to consolidate my own understanding and to act as a reference to future me, but if it also helps others in a similar…
In which I record my thoughts on DeepSEA . Terminology # ChIP-seq (Chromatin Immunoprecipitation Sequencing): Expression Quantitative Trait Locis (eQTLs) Cofactor Binding Sequences Histone Marks: modifications to histone proteins in the nucleosome that impact the shape of chromatin. gkm-SVMs: The gkm-SVM is the previous SOTA model for predicting transcription factor binding based on ChIP-seq data.…
In which I record my thoughts on Basset . Bio Background # The genome consists of (broadly) two types of genes, coding genes and noncoding genes. Coding genes get translated into proteins (as laid down by the Central Dogma) and are what most of us learned about in bio class. Non-coding genes… don’t. As I understand it, noncoding genes can do a bunch of different things, but part of…
In which I record my thoughts on DeepBind . What is this paper about? # The authors of this paper designed a conv net model to predict how well different proteins will bind to sequences of DNA or RNA. They train one model per protein for many different sequences and show that these models can predict binding affinities for sequences well enough to produce insights regarding the impact of single…
Selected Exercises # 6.B # 1. (a) Suppose \( \theta \in \mathbf{R} \). Show that $ (\cos \theta, \sin \theta), (-\sin \theta, \cos \theta) $ and $ (\cos \theta, \sin \theta), (\sin \theta, -\cos \theta) $ are orthonormal bases of $ \mathbf{R}^2 $. (b) Show that each orthonormal basis of $ \mathbf{R}^2 $ is of the form given by one of the two possibilities of part (a). (a) First, we show that both…
Meta: While I originally set out to write this essay about why I no longer “endorse” Stoicism as an operating framework, re-reading a bunch of Stoic stuff led me to realize that I still agree with ~90% of Stoic principles. Peter Thiel recently talked about (in German) why he’s not a Stoic, Stoics, for example, are obsessed with death. However, I am the opposite of a stoic, I…
Selected Exercises # 5.A # 12. Define $ T \in \mathcal L(\mathcal P_4(\mathbf{R})) $ by $$ (Tp)(x) = xp’(x) $$ for all $ x \in \mathbf{R} $. Find all eigenvalues and eigenvectors of $ T $. Observe that, if $ p = a_0 + a_1 x + a_2 x^2 + a_3 x^3 + a_4 x^4 $, then $$ x p’(x) = a_1 x + 2 a_2 x^2 + 3 a_3 x^3 + 4 a_4 x^4. $$
Selected Exercises # 4. Suppose $m$ and $n$ are positive integers with $ m \leq n $, and suppose $ \lambda_1, \dots, \lambda_m \in F $. Prove that there exists a polynomial $ p \in \mathcal P(\mathbf{F}) $ with $ \deg p = n $ such that $ 0 = p(\lambda_1) = \cdots = p(\lambda_m) $ and such that $ p $ has no other zeros. First, we can show that the polynomial $p’(z) = (z - \lambda_1) \cdots…
Selected Exercises # 3.D # 7. Suppose $ V $ & $ W $ are finite-dimensional. Let $ v \in V $. Let $$ E = \{ T \in \mathcal L(V, W): T(v) = 0 \}. $$ (a) Show that $ E $ is a subspace of $ \mathcal L(V, W) $. (b) Suppose $ v \neq 0 $. What is $\dim E$? For (a), to show $ E $ is a subspace of $ \mathcal L(V, W) $, we need to show that $ E $ contains zero and is closed under both addition and…
As I understand it, much of Coase’s work focuses on why companies’ internal structures don’t resemble markets all that much. Instead, most firms have feudal-ish structures. The CEO has, at the discretion of the board, final decision making power on nearly all decisions, and then their power trickles down recursively to their subordinates for decisions in which they’re not…
Intro # I’ve recently been making my way through Axler’s Linear Algebra Done Right and, as a way to motivate myself to continue, have decided to blog my notes and solutions for exercises as I go. Insights # Section 2.A # You can convert any linearly dependent list to a linearly independent list with the same span. # By the linear dependence lemma, if you have a list that’s…
Babbling About Babble # In a series of posts on his blog (or LessWrong if you prefer), alkjash writes about “babble” and “prune”, the two components of an adversarial model of knowledge production in humans. Alkjash describes babble and prune like this: Here’s a simplistic model of how this works. I try to build a coherent sentence. At each step, to pick the next word, I…
(Cross-posted from Facebook .) This is the first time I’ve posted on Facebook in … long. However, at the encouragement of a few friends, I’ve decided to leverage my no-shave November stunt as an opportunity to urge people to donate to one of the many worthwhile causes out there. Technically, my friends said I should do “the Movember thing” but this long-winded post…