Running LLMs on AMD Ryzen AI NPU under Linux — A First

I just got Llama 3.2 1B running at 60 tokens/sec on my nucbox mini PC&rsquo;s NPU (Neural Processing Unit) (AMD Ryzen AI 9 HX 370) under Ubuntu Linux — something that, as far as I can tell, hasn&rsquo;t been publicly demonstrated before. The problem AMD shipped the XDNA2 NPU in Ryzen AI processors (Strix Point, Strix Halo, Kraken Point) with up to 50 TOPS of AI compute. On Windows, tools like…