8 On-Device Inference Challenges

One of the trends I find the most exciting these days, is the local inference of LLMs on mobile. That’s for sure related to my time as a mobile- (or, rather iOS-) developer. Yet, the domain brings so many compelling technical challenges that it’s hard not be fascinated about. As I wrote in a previous LinkedIn post, local inference offers many benefits for AI, especially generative AI, primarily in…