John Wu's personal website
Deep neural networks can be thought of as compositions of many simple transformations, each represented by a layer with trainable parameters. When the number of layers is large, the effect of multiplying many random matrices becomes exponentially unstable, i.e. they can grow or shrink exponentially. This is the primary reason that naive initialization leads to exploding or vanishing signals for…
AI is here, and its impacts on education cannot be overstated. Let’s put aside the issues of cheating; I assume that you want to learn, perhaps with the assistance of LLMs if they are actually helpful. But how do you know you’re not using AI as a crutch, versus using it to augment learning? The former setting outsources your thinking to AI, whereas the latter can help you reveal gaps in your…
With the Euclid and Roman Space Telescope missions ready to image billions of galaxies, we’ll need data-driven methods to find new, rare phenomena that exist outside human-defined taxonomies! Sparse Autoencoders (SAEs) can be that discovery engine, surfacing interpretable features in modern galaxy surveys. This blog post highlights some preliminary results from our tiny NeurIPS ML4PS workshop…
This post discusses how graph neural networks (GNNs) can model the galaxy–halo connection within its large-scale surroundings. Dark matter structures, which seem to account for most of the mass in the Universe, can be represented as nodes in a cosmic graph. But dark matter—which solely interacts via gravitation—is also much easier to simulate than the messy baryons, whose magnetohydrodynamics are…
The Eleven Laws of Showrunning by Javier Grillo-Marxuach is full of useful advice for management and operations. Nominally, it’s about how to deliver a television show, from ideation to writing to production to postproduction, but there’s a ton of guidance that’s surprisingly relevant for working with large language models (LLMs).
In the book Impro: Improvisation and the Theatre, Keith Johnstone recounts a moment between a teacher and a special needs student. The teacher holds up a flower and says, “Look at the pretty flower.” The girl responds, “All of the flowers are beautiful.” Then the teacher gently says, “but this flower is especially beautiful.” The girl proceeds to scream and thrash about violently.
Many physical phenomena exhibit relational inductive biases and can be represented as mathematical graphs. In recent years, graph neural networks (GNNs) have been successfully used to model and learn from astronomical data. This post provides an introductory review to GNNs for astrophysics.
If you’re a blogger or researcher sharing your work online, you’ve probably wondered: is social media actually useful for disseminating your writing? I’ve been asking myself this question since returning to blogging just over a month ago.
I am a self-confessed productivity junkie. I hate wasting time. And if you scroll through social media, or even my blog posts, you might think that the typical research or learning process is just a happy, monotonic hill climb, capped off with regular announcements of new discoveries or gained expertise. But what if the most important lessons emerge not from unencumbered progress, but rather from…
Here’s a casual introduction to foundation models and how they might impact astronomy research in the coming years. I’m writing this on the train back from New York to Baltimore, having just wrapped up the Foundation Models in Astronomy workshop at the Flatiron Institute Center for Computational Astrophysics. My co-organizers and I are planning to write up a more comprehensive blog post based on…