The quagmire of LoRA training on MPS

I’ve accumulated some knowledge and it’s time to give back to the community. The knowledge was earned the old-fashioned way, but ramming my head into different machine learning walls whilst trying to adapt a Gemma3 model on my MBP M2 Max (96GB). First wall: MLX is amazing, but has very sharp edges MLX runs noticeably faster than Pytorch on Apple Silicon, which is why apps like LM Studio use it as…