NEAT: A-Level Coursework
For my A level computer science project I have decided to dive deep into the world of AI.
Inspired by many Youtube videos of genetic algorithms at work I started my research and came across a paper called "Evolving Neural Networks through Augmenting Topologies" (NEAT) published by the EPlex group at the University of Central Florida.
I've picked Unity as my development environment as I wanted to test the networks on real-time physics control problems and Unity comes with a great physics engine. Given my past experience with Unity, I was able to focus on the task at hand instead of spending time on learning the platform. I was usually evaluating around 100 networks per generation, so I used multi-threading and Unity co-routines to parallelise the process where possible.
As this was my first hands-on encounter with neural networks, I decided that instead of importing ready-made libraries and applying them to my simulations, I would write everything from scratch myself in order to gain a deeper understanding. Therefore although C# NEAT libraries existed, my only resources were the original NEAT papers.
The method starts with a connected network with no hidden layers and then adds neurons and connections as it evolves. Due to natural selection, every significant modification to the network is justified since if it underperformed, it'd be discarded. In order to allow for novel modifications to prove themselves, the networks compete within their 'specie' (other networks that are similar) as opposed to competing with the entire population.
Finally, once NEAT was implemented, I tested it on a few problems. Namely: getting a worm to jump over obstacles, landing a rocket and playing tic-tac-toe. I was very impressed by how well the networks were able to perform even after a short training time on a low power laptop.
Inspired by the YouTube video: Genetic algorithm. Learning to jump over ball.
I decided to put NEAT to the test and see if it could learn how to jump over an incoming ball. The objective was to minimize the contact time with the ball. The agent was able to control 2 actuators that make either the left or the right hand side jump.
I also wanted to try out the system on something similar to OpenAI's Lunar Lander. I decided to recreate a similar environment in Unity where a rocket has to land on the red landing pad. The agents were trying to maximise the score function of: final distance to lanuch pad + bonus points if landing was successful.
Finally, I wanted to see how the system would perform on a simple 'board game' such as Tic-Tac-Toe. Against a random opponent it was able to win with a very significant margin. However, under closer inspection when playing against humans, it was clear that the agents were simply learning a static strategy that mostly works and wouldn't take into consideration the actual geometrical relations of the cells of the baord and their states. Hyper-NEAT should be more suited for such a task since by its design it already has a geometrical representation embedded due to it's use of Compositional Pattern Producing Networks (CPPNs).