Hirshbot is a simple game I developed in Unreal Engine 4 for CS 4701: Practicum in Artificial Intelligence, a class I took in Spring 2020 that was—if it wasn’t obvious—taught by Professor Haym Hirsh. I was a part of a 3-person team, though I developed the base Hirshbot game myself since I was the only one familiar with Unreal Engine at the time.
Our original idea was to run a reinforcement learning algorithm on the popular browser game QWOP, but given the time constraints we ended up deciding it would be simpler to create our own 3D version of QWOP in Unreal, since it would be much easier that way to figure out a way for our algorithm to interface with the game.
The resulting simple game I developed features a four-legged robot that can be controlled with QALP, each input corresponding to a limb.
To make things easier when running our AI, Hirshbot has no lose condition but instead a 30-second timer, and the goal is to maneuver Hirshbot as far from the starting point as possible within that time. In addition, because there is no balancing element like in QWOP, I also added static obstacles—columns that stay in the same place for each run—so that instead of learning how to stay balanced, our AI would have to learn how to navigate around the obstacles in the most effective way.
Our reinforcement learning algorithm was written in C++ and made to interface with Hirshbot using UE4 Blueprints. We tested multiple different variations of a genetic algorithm on Hirshbot via a different game mode I set up, which allowed us to automatically run a simulation that used a chosen algorithm over many runs to try and maximize score.
We created a 16-letter “alphabet” where each letter corresponded to a possible combination of the four QALP inputs, and the “chromosome” used for a run was composed of a combination of those 16 letters, creating a string of inputs that would be fed into Hirshbot at equal intervals. Over the course of a simulation, a set of chromosomes forming a “generation” would be run, and the next generation’s chromosomes would be built based on the most successful chromosomes of the previous generation. All decisions made by the AI were documented as data for us to later analyze.
Our final report, which more thoroughly documents our process and shows our results, can be accessed below.