Separating Simulacra: tags for smarts and safety

I’ve seen multiple papers where LLM training data included tags providing metadata, which improved the models’ downstream performance. However, as far as I can tell, they always end training on pure text data so that the models can be used without tags. I don’t understand why that should be desireable.