Gamifying Machine Learning

A game that plays a game in a game…

Reward based Machine Learning simulations are games for AI algorythms to play.

Setup your simulation, setup your rewards, let the machine learning network learn how to play it and get it’s best score, constantly trying to improve. I’ve setup plenty of machine learning simulations in Unity3d previously.

First I needed to remove the dependancy on the Python ML stack, I wrote about some of the hurdles in achieving native machine learning in Unity. Sincce then I’ve created a robust custom C# training scripts, here’s a ball balance solver from June 2022.

I love the experimentation and free play of sandbox video games, I also like casual cozy games, they strike me as a good fit for a machine learning game where you can train your characters or bots in-game on-device. Nothing pre-scripted just training your own bots from a blank slate.

I’ve mocked up an approach that looks like it would be fun and interesting. It focuses on allowing the player agency to train the AI by tweaking the rewards for various parts of the task, this is the essencce of simulation based training, setup the environment and then set up the reward parameters.

I set up a simple kart racer which seems to fit well with a racing mechanic.

I really enjoyed watching the training sessions from this AI teaching itself to drive in Trackmania video

Its a setup that I think can be optimised to allow learning to take place rapidly and the user can watch as it learns. Here are a few rough screen grabs of early prototypes July 2022

Bringing this up to date, here’s a recent July 2023 training run

I’ve optimised the setup for machine learning in Unity removing the need for any python libraries by creating the whole reward machine learning algorythms in C#, there were a few complications as the C# Math.net libraries weren’t originally meant to be used in game loops.

There are some knock on effects of running machine learning on device, firstly you want to speed up the simuation to the absolute maximum and traditional math library matrix operations need to be rewritten to allocate and reuse their memory.

Digging into the Math.net code I found the scripts for the matrix operations and modified the implementations to create a fixed piece of memory at startup and reuse that memory. I was seeing around 200k of allocations per learning frame, and managed to reduce this to under 1k. While memory allocations aren’t all completely gone, it’s enough to allow an iPad air to run the training at about 10x realtime, which allows a simple training of an agent to be able to drive competitively in about 5 mins.

You can find out more about the project on it’s own page, see below

Racelab: Robot Race Training

·

July 26, 2023

RaceLab: a machine learning racing game You are in charge of training your robot racers however you want. You can create fast racers that aim for an optimal racing line or aggressive brains that attempt to block a…