Regular analyses of advances in natural language processing and machine learning.
Throughout recent years, LLM capabilities have outpaced evaluation benchmarks. This is not a new development. What is new is that the set of standard LLM evals has further narrowed and there are questions regarding the reliability of even this small set of benchmarks.
The Top Open-Weights LLM + RAG and Multilingual Support
Teaching Machines a New Language Like Humans
And Why I Joined Cohere
A Workshop Retrospective
5 Key Research Directions Without Much Compute
In this newsletter, I ll discuss a selection of exciting papers and workshops I m looking forward to at EMNLP 2023 and the trends I observed.
A Round-up of 20 Exciting LLM-related Papers
We cover the latest generation of instruction-tuning datasets that are much closer to real-world use cases but still have their limitations!
In this post, we ll take a closer look at the question Do LMs represent space and time? inspired by a recent paper.
The most popular instruction datasets
In this edition, we ll explore tool use arguably one of the hottest new capabilities of LLMs. We ll look at types of tools, benefits of tool use, recent developments, and future directions.
This newsletter discusses components for building generative agents and publication venues for large language models (LLMs).
This newsletter discusses attention that enables modeling long sequences and simple but surprisingly competitive classifiers such as based on gzip compression.
Hi all,
Hi all,This newsletter covers PaLM, DALL-E 2, and Chinchilla, chain-of-thought prompting, and the role of values and culture in NLP.This edition is somewhat delayed as I've been busy with planning a move (I'll be flying to Germany tomorrow; say hi if you're in Berlin) and exhausted by current events. I hope that you are all staying safe in these trying times .I really appreciate your feedback, so…
Hi all,I hope you've had a good start to the new year. This newsletter covers my and others' highlights of 2021. I also discuss recent pre-trained models that put more emphasis on safety and recent text similarity models where large is not always better. I really appreciate your feedback, so let me know what you love and hate about this edition. Simply hit reply on the issue.Click here to view the…
Hi all, First off, some personal news: I've moved from DeepMind to Google Research this week. Because of this move, the past months have been quite busy. In light of this, I decided to pause the newsletter over the last couple of months. I plan to continue with it in a more sustainable manner.I'll be continuing to work on multilingual NLP, with a focus on under-represented languages, particularly…
Hi all,This newsletter covers some of my favourite papers from ICML 2021, a discussion of open collaboration, art generated by the CLIP model, how to leverage information from the Internet in your models, and new benchmarks in the style of GLUE.FYI, I'll be at ACL 2021 virtually this week. Ping me on Gathertown or send me an email if you would like to chat. I'm co-author on two papers on…
Hi all,This newsletter is a bit delayed. I had to skip the last one as I had to take a break after a busy period in the end of May (EMNLP and NeurIPS deadlines). At the same time, there's so much happening that I've found it hard to catch up. Now I'm back, feeling more energized, and updating myself (and you) on what's new.I'll discuss the biggest advances over the last months including GitHub…