For the past two weeks, an AI agent ran autonomous architecture research on BlaGPT , my GPT-2-scale playground for testing LLM techniques. It ran about 100 short proxy experiments and 22 full training runs, implemented techniques from papers, kept its own ledger, and pushed everything to the repo. Validation loss went from 3.2354 to 3.1965. The agent is Keche, my personal AI assistant that lives…
Yann LeCun gave a talk at ETH Zürich last week, “World Models: Enabling the next AI revolution”. I went through the whole thing and checked his claims against the actual papers. Here is where he is right, where he overreaches. The talk is his standard world-models pitch, updated with new results from 2025: LeJEPA/SIGReg, V-JEPA intuitive physics, and the launch of his startup AMI Labs. It is also,…
Efficient large language models have driven various architectural innovations—from mixture-of-experts to quantization techniques. Attention mechanisms remain the core computational bottleneck in transformers, and optimization typically degrades output quality. Most sparse attention approaches make coarse-grained trade-offs, sacrificing model capability for speed. DeepSeek-V3.2-Exp uses a…
MiMo-Audio is Xiaomi’s 7B parameter model that processes speech and text through a unified architecture. Trained on 100+ million hours of audio data—10x larger than existing open-source models— that results in emergent capabilities like voice conversion, speech translation, and cross-modal reasoning through few-shot learning, demonstrating speech scaling laws similar to text language models.…
VibeVoice: Next-Token Diffusion Meets Long-Form Speech Generation TL;DR for the Busy Reader: VibeVoice generates up to 90 minutes of multi-speaker conversational audio using next-token diffusion Introduces ultra-efficient speech tokenizers operating at 7.5 Hz (3200× compression vs. 24kHz audio) Combines Qwen2.5 language models with token-level diffusion heads for streaming synthesis Outperforms…
The race for more capable large language models has largely been about scaling—bigger models, more parameters, more compute. NVIDIA’s Nemotron Nano 2 challenges this paradigm with a hybrid architecture that achieves competitive performance through architectural innovation rather than brute-force scaling. The Architectural Revolution Nemotron Nano 2 introduces a hybrid layer pattern that combines…
Text-to-Speech (TTS) systems have traditionally struggled with the trade-off between quality and latency. Most high-quality systems require processing the entire text before generating audio, while streaming approaches often sacrifice naturalness. KyutaiTTS breaks this paradigm with a novel approach that delivers high-quality, streaming audio generation with unprecedented low latency. The…
Speech synthesis has evolved rapidly, but most systems still struggle with natural expressiveness and multi-modal understanding. Higgs Audio V2 addresses these limitations through a unified architecture that treats audio as a language, enabling emergent capabilities like multi-speaker dialogues and prosody adaptation without explicit training. The Challenge Traditional text-to-speech systems…
Here are two papers that target: Paper1: improving pre-training performance of an LLM by merging model checkpoints along the training trajectory and Paper2: improving post-training RL efficiency by avoiding the low-probability token dominance. Model Merging in Pre-training of Large Language Models What’s new Pre-trained Model Average (PMA) strategy for model merging during pre-training of large…
Recently, I’ve been testing different architectural modifications in my BlaGPT benchmark to see how they affect performance. I wanted to share what I’ve learned about three interesting techniques: Softpick, Canon Layers, and Parallel Transformer blocks. Softpick: A Different Approach to Attention Softpick offers an interesting alternative to the standard softmax function in attention blocks. It…
When working with language models for audio tasks, switching between different audio representations can be challenging. Let’s say you have a text-to-speech model using a particular audio codec, and a new codec comes along with better quality. The obvious approach would be to retrain your model with the tokens from this new codec. However, language models are typically large and training them from…
Study the Problem Describe the problem in detail Identify inputs and outputs of the system Identify the business context and goals. Identify the constraints and assumptions. Identify the stakeholders and ensure that everyone is on the same page about all the above. Define the Metric Define a single value performance metric for evaluation. Ensure that the metric is aligned with your goals. When…
Having dedicated the last half-dozen years to an open-source AI, here are my insights on the pros and cons of operating as an open-source organization. This article is a reflection of my experience. I welcome you to participate in the conversation, regardless of your agreement or disagreement with any points. Feel free to reach out at any time. Goods Easier to reach your users Cool models and…
Here I wrote a short note on my experiment with training XTTS for Voice Conversion. Briefly here is what I tried: Use quantized Hubert as content encoder to extract speaker-independent content features. Replace the text conditioning in the XTTS with the content features. Train the model with speech to speech manner. Source speech is quantized with Hubert and used to condition the model along side…
🐸 Github 🤗 Demo 🤖 Model card 💬 Discord We recently released XTTSv2 with 🐸TTS v0.20, and here I go over the relevant details of the model. XTTSv2 uses the same backbone as XTTSv1. It is a GPT2 model that predicts audio tokens computed by a pre-trained Discrete VAE model. The core update is changing the way we condition the model on the speaker information with a Perceiver model. In our model,…
🎮 XTTS Demo 👨💻 XTTS Code 💬 The XTTS v1.1 version now includes Japanese, along with some model enhancements and a fresh vocoder. As a result, this iteration of XTTS can now speak in 14 languages with improved quality, One of the common complains with XTTS v1 revolved around the recurrence of prompts in the output audio. The principal reason for this was that the model was trained by…
In this article, I will explore various alternatives to transformers, considering their architectural improvements, computational efficiency, and performance results across different benchmarks. I intend to continually update this post with new models in the future. If you believe there are any models or important points that should be included or any corrections that need to be made, please feel…
"Smart electric cars in French, Rococo style, classical style, oil on canvas" XTTS v1 technical notes 🎮 XTTS Demo 👨💻 XTTS Code 💬 XTTS is a versatile Text-to-speech model that offers natural-sounding voices in 13 different languages. One of its unique features is the ability to clone voices across languages using just a 3-second audio sample. Currently, XTTS-v1 supports the following 13…
"Smart electric cars in French, Rococo style, classical style, oil on canvas" We recently open-sourced XTTS, the best open-source text-to-speech model we’ve released so far. XTTS uses the latest generative AI techniques to deliver faster, higher-quality speech in 13 different languages. It is now accessible with 🐸TTS. XTTS offers… Voice cloning with just a 3-second audio clip. Emotion and style…
Generated by Stable Diffusion" 👉 Subscribe to my Substack to get the latest news and articles. I recently dedicated some time to watching the new course on fine-tuning large language models offered by Deep Learning AI. I took notes during my viewing and would like to share them here. What is fine-tuning? Specializing LLMs, teaching a new skill instead of expanding its knowledge base. GPT4 is…
Generated by Stable Diffusion with “a robot handshaking with a human. (You can also see this on my substack .) Privacy violation Stealing private information (credit cards, passwords). Simply ask ChatGPT for bank login credentials, credit card numbers, social security numbers, etc. 🔗 News The advent of AI technologies like ChatGPT has raised significant privacy concerns. As these models interact…
Generated by Midjourner - "A group of robots working together in a robot factory, French, Rococo style, classical style, oil on canvas" TLDL, Don’t create APIs for ML, just copy&paste. Test every line. If you aren’t sure about the design, test first. Always keep your experiments reproducible (lineage, data, code, baseline). Document everything. Be clear, and avoid abbreviations. Copy & paste ML,…
YourTTS The recent surge of new end-to-end deep learning models has enabled new and exciting Text-to-Speech (TTS) use-cases with impressive natural-sounding results. However, most of these models are trained on massive datasets (20-40 hours) recorded with a single speaker in a professional environment. In this setting, expanding your solution to multiple languages and speakers is not feasible for…
git show-branch \ | sed "s/].*//" \ | grep "\*" \ | grep -v "$ ( git rev-parse --abbrev-ref HEAD ) " \ | head -n1 \ | sed "s/^.*\[//"
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. This is one of the things I always need but I forget. So here is a piece of mind to check back. pip install ipython pip install ipdb export PYTHONBREAKPOINT = ipdb . set_trace # this is to use…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Original paper : https://arxiv.org/abs/2005.14165 It uses the same architecture as GPT-2. The largest model uses 170B parameters and trained with a batch size of 3.2 million . (Wow!). Training…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Model Samples: https://erogol.github.io/ddc-samples/ Colab Notebook (PyTorch): link Colab Notebook (Tensorflow): link Despite the success of the latest attention based end2end text2speech (TTS)…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. In this post, I like to introduce two methods that worked well in my experience for better attention alignment in Tacotron models. If you like to try your own you can visit Mozilla TTS. The first…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Tacotron is a commonly used Text-to-Speech architecture. It is a very flexible alternative over traditional solutions. It only requires text and corresponding voice clips to train the model. It…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. After a while of using tmux, you might see that you cannot reconnect it from another terminal windows with the error message error connecting to /tmp/tmux-1000/default (No such file or directory…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Mixup - https://arxiv.org/abs/1710.09412.pdf Shake and Shake - https://arxiv.org/pdf/1705.07485.pdf Mix Feat - https://openreview.net/forum?id=HygT9oRqFX Speed perturbation (ASR) -…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. To explain briefly, WSL enables you to run Linux on Win10 and you can use your favorite Linux tools (bash, zsh, vim) for your development cycle and you can enjoy Win10 for the rest. It obviates…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Small Intro. and Background Recently, I started at Mozilla Research. I am really excited to be a part of a small but a great team working hard to solve important ML problems. And everything is…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Small Intro. and Background Recently, I started at Mozilla Research. I am really excited to be a part of a small but a great team working hard to solve important ML problems with a great deal of…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Selenium is a great tool for Internet scraping or automated testing for websites. I personally use it for scrapping on dynamic content website in which the content is created by JavaScript…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. I started to work on time series on stock market for fun. I like to see if an AI bot trading without a manual help is possible or is just a luring dream. Lately, I read a lot about the topic…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Online Hard Example Mining (OHEM) is a way to pick hard examples with reduced computation cost to improve your network performance on borderline cases which generalize to the general performance.…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Let’s directly dive in. The thing here is to use Tensorboard to plot your PyTorch trainings. For this, I use TensorboardX which is a nice interface communicating Tensorboard avoiding Tensorflow…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. paper: https://arxiv.org/pdf/1709.07634.pdf ReLU is defined as a way to train an ensemble of exponential number of linear models due to its zeroing effect. Each iteration means a random set of…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. There are numerous on-line and off-line technical resources about deep learning. Everyday people publish new papers and write new things. However, it is rare to see resources teaching practical…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Lately, we ( TwentyBN ) took a part in Activity Net trimmed action recognition challenge. The dataset is called Kinetics and recently released. It is a collection of 10 second YouTube videos.…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Sometimes, It is really a mess to try installing OpenCV to your system. Nevertheless, it is really great library for any case of vision and you are obliged to use it. (No complain, just C++). I…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. One of the main problems of neural networks is to tame layer activations so that one is able to obtain stable gradients to learn faster without any confining factor. Batch Normalization shows us…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. I owned a Raspberry Pi long ago and it was just sitting in my tech wash box. After watching a Youtube session of creative Raspberry applications, with envy , I decided to try something by myself.…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. paper: https://arxiv.org/abs/1702.08591 The whole heading of the paper is “The Shattered Gradients Problem: If resnets are the answer, then what is the question?”. It is really interesting work…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Quora recently announced the first public dataset that they ever released. It includes 404351 question pairs with a label column indicating if they are duplicate or not. In this post, I like to…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. In simple terms, dilated convolution is just a convolution applied to input with defined gaps. With this definitions, given our input is an 2D image, dilation rate k=1 is normal convolution and…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. What is Adversarial? Machine learning is everywhere and we are amazed with capabilities of these algorithms. However, they are not great and sometimes they behave so dumb. For instance, let’s…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Note : I regularly update this list. Machine Learning 101: I. Introduction to Machine Learning http://homepages.inf.ed.ac.uk/rbf/IAPR/researchers/MLPAGES/mltut.htm…
This post was originally published on erogol.com (hosted on DigitalOcean) and recovered from the Wayback Machine after the original server was lost. Some formatting or images may differ from the original. Paper : https://arxiv.org/abs/1312.6199 This paper studies description of semantic information with higher level units of an network and blind spot of the network models againt adversarial…