中文 ｜ English

A unified representation for text and vision is a natural pursuit, as it enables simpler multimodal modeling and more efficient training. However, representing images as discrete signals in the same way as text inevitably introduces severe information loss: reconstruction-oriented representations often lack semantics, whereas semantically stronger features typically suffer from loss of detail.

ViQ (Visual Quantized Representations) is a framework designed to balance semantics and details in discrete visual representations while supporting inputs at native resolutions — serving as a unified, general discrete representation for arbitrary visual inputs. Built on a SigLIP2 vision tower with a position-aware, head-wise FSQ (Finite Scalar Quantization) head, ViQ turns an image at any resolution into a sequence of discrete codes that can feed either an MLLM (Qwen2.5 backbone) for understanding or a decoder for high-fidelity reconstruction.

📢 News

• [2026/06] 🔥 We release ViQ — both training and inference code. For training we provide a simple single-stage example that directly trains the 16k-FSQ ViQ; the paper's two-stage recipe (2-1 Proximal Representation Learning and 2-2 Quantization Training) can be reproduced by toggling the flags in the training script.

🌟 Overview

Approach of ViQ representation learning. ViQ structures quantization learning into two stages. (Stage 1) Text-Aligned Pre-training aligns the ViQ encoder with semantic-rich supervision from a pretrained language model, while resized positional embedding and native patchify enable any-resolution inputs; a self-distillation loss against a fixed-resolution teacher preserves the foundational language-image knowledge. Stage 2 discretizes the continuous features progressively: (2-1) Proximal Representation Learning compacts the latent space with an