Getting SAC to Work on a Massive Parallel Simulator: Tuning for Speed (Part II)

This second post details how I tuned the Soft-Actor Critic (SAC) algorithm to learn as fast as PPO in the context of a massively parallel simulator (thousands of robots simulated in parallel). If you read along, you will learn how to automatically tune SAC for speed (i.e., minimize wall clock time), how to find better action boundaries, and what I tried that didn&rsquo;t work. Part I analyzes why…