MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Introduces a text-heavy visual instruction-tuning strategy that transfers instruction-following ability and domain knowledge across modalities, matching vision-heavy mixtures across 12 benchmarks with as little as half the training tokens.