Implementing Reinforcement Learning in Unreal Engine 4 with Blueprint

Abstract

With the availability of modern sophisticated game engines, it has never been easier to create a game for implementing and analyzing machine learning (ML) algorithms. Game engines are useful for academic research because they can produce ideal environments for rapid simulation and provide ways to implement Artificial Intelligence (AI) in Non-Player Characters (NPCs). Unreal Engine 4 (UE4) is a great choice for ML simulation as it contains many useful tools. These tools include Blueprint Visual Scripting that can be converted into performant C++ code, simple-to-use Behavior Trees (BT) for setting up traditional AI, and more complex tools such as AIComponents and the Environment Query System (EQS) for giving an agent the ability to perceive its environment. These built-in tools were used to create a simple, extensible, and open-source environment for implementing ML algorithms in hopes that it will reduce the barrier of entry for using these algorithms in academic and industry-focused research. Experimental results indicate that reinforcement learning (RL) algorithms implemented in Blueprint can lead to learning a successful policy in very short training episodes.