On the Viability of Fine-Tuning SLMs

Until recently, I held the opinion that training custom language models was inadvisable except in relatively rare cases. Simply requiring marginally better performance on your task was insufficient justification for customization; subsequent generations of models from foundation labs would inevitably catch up. Only when you had extreme latency constraints, specific tasks with low drift, and/or…