Unified factored representations

One of the most interesting papers I’ve read in the last couple of years is Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis (2025).

I don't know what to do with the ideas in the paper, and that's a frustration of my own making.

The paper suggests:

1. what many machine learning systems learn—“the representation”—is a tangled mess;
2. better representations can be discovered;
3. you don’t need more data to do this; and
4. better representations matter.

The observations in the paper go like this

You can train a neural network to reproduce a particular image, such as a butterfly or a skull. When you look at what’s been learned, it’ll be whatever it needs to get the job done. Fine.

Now, as an alternative: ask people to pick what they like from abstract images, and use evolutionary computation to “breed” neural networks that will generate new images for the next generation. Repeat that, and perhaps because we’re drawn to symmetry and interesting-looking things, you do end up with butterflies, skulls, and other artefacts. 

What have these two approaches learned? In a sense, the way they represent the world (or in this simple case, one image) is through the connections between nodes. The weights.

The fun part is this: when you look into the representations found by the evolutionary system, you can vary the weights between connections and you get systematic changes in the image. Larger or smaller wing size, or an eye socket in a skull that opens and closes. 

Try that with a more traditionally trained neural network, and you get noise and blotches, and useful properties like symmetry are not preserved. Or as the paper calls it, these are the result of “fractured entangled representation”.

Look at the image at the top of this post to see that in action (images from figure 16 of the paper).

Who cares?

The paper is very clearly not proposing an improved machine learning framework. Instead, it’s saying that we might be able to learn more modular and useful representations of the world, rather than living with spaghetti code. 

And if that’s true, throwing more compute at the current machine learning isn’t going to be enough to move the field on.

As I said, I don’t know what to do with this information, but it’s compelling.