BitDance: Scaling Autoregressive Generative Models with Binary Tokens

Yuang Ai*, Jiaming Han*, Shaobin Zhuang*, Weijia Mao, Xuefeng Hu, Ziyan Yang, Zhenheng Yang, Yali Wang, Huaibo Huang†, Xiangyu Yue†, Hao Chen*†‡

^* Equal Contribution  ^† Corresponding Author  ^‡ Project Lead

For visual generation, discrete autoregressive models often struggle with poor tokenizer reconstruction, difficulties in sampling from large vocabularies, and slow token-by-token generation speeds. We present BitDance, which addresses these challenges via a large-vocabulary binary tokenizer, a binary diffusion head for sampling in large discrete space, and a next-patch diffusion paradigm that enables efficient multitoken prediction. BitDance is an open-source discrete autoregressive foundation model with 14B parameters, trained on large-scale multimodal tokens. While maintaining the standard language modeling paradigm for text tokens, BitDance employs a next-patch diffusion paradigm for visual tokens to predict multiple tokens in parallel—up to 64 per step. This unified multimodal framework is simple, efficient, scalable, and capable of efficiently generating high-resolution, photorealistic images.

🔥 News

• 2026.2.18: Special thanks to @Bili-Sakura for providing the diffusers version for all BitDance models!
• 2026.2.17: We release UniWeTok, An Unified Binary Tokenizer with Codebook Size