X-Token: Distilling Knowledge Across Tokenizers That Don’t Speak the Same Language

You’ve got a small student model you want to make smarter. Somewhere out there, a bigger, wiser teacher model has all the answers. Standard knowledge distillation says: have the student match the teacher’s output distribution, position by position, via KL divergence. Elegant. Clean. Works great.