Reflecting on 2025 and my one year back at Google Deepmind’s Gemini team. From being back to research to winning the historic IMO gold medal, along with starting GDM Singapore, I reflect on what a wild year it has been.
Returning to Google and recounting my experiences as a startup co-founder.
A Blogpost series about Model Architectures Part 1: What happened to BERT and T5? Thoughts on Transformer Encoders, PrefixLM and Denoising objectives
Chronicles of training strong LLMs from scratch in the wild
Documenting my 3.3 years at Google Research and Brain.
Releasing the new open source Flan-UL2 20B model.
Here are some of the best language AI / NLP papers of 2022!
Some thoughts on emergent abilities and scaling language models.