The Last Architecture Designed by Hand

This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.

I bet there is another new architecture to find that is gonna be as big of a gain as transformers were over LSTMs. Sam Altman, the CEO of the company most invested in the transformer is telling a room of students it isn’t the final form. So what comes after the transformer? He’s probably right that something will, and the evidence is no longer anecdotal. Several recent papers have…

Read on philippdubach.com ↗