Shared Backbones: Loading Weights Once, Serving Many Models

I keep running into the same pattern when trying to self-host models (which is a lot of fun): we run several big models side by side, all of them valuable, all of them slightly different, and all of them wasting VRAM by reloading nearly the same weights. &#xA; This post is my attempt to explore a specific idea: &#xA; Can we load a shared backbone of weights once on a GPU, then load only the small,…