A Simple Way to Switch Audio Codecs in Audio Language Models

When working with language models for audio tasks, switching between different audio representations can be challenging. Let’s say you have a text-to-speech model using a particular audio codec, and a new codec comes along with better quality. The obvious approach would be to retrain your model with the tokens from this new codec. However, language models are typically large and training them from…