Video Game with Artificial Intelligence Elements Based on Reinforced Machine Learning

Video games industry came still a short, but at the same time a very productive way from 1971’s considered by many first commercial arcade video game called “Pong”, which was replicating an already existing and widespread game of tennis, to 2020s and beyond with countless examples of what a man’s thought can achieve in technology, storytelling, design and arts by presenting work in, unseen using any other media before, highly detailed digital interactive 2D and 3D environments, which recently, with modern technology, slowly but steadily moving to virtual reality and machine learning, and produce more than $120 billion of revenue per year. This paper explores the possibility of using machine learning technologies (reinforced machine learning in particular) in artificial intelligence of video games opponents (often times referred to as NPCs – non-player characters) to shorten resources spent on game development by minimizing amount of scripted events in their behavior and increasing potential outcome of project in terms of sales, by achieving broader, more diverse and meaningful gameplay scenarios, based on non-scripted non-player characters’ behaviors (which on their behalf can be based on player’s actions, decisions and different states of environment set by game designers), such that a player can achieve overall better game experience by receiving unique responses from game’s world and characters and having interactions with complex systems which can produce wide range of events, rather than scripted pre-determined scenarios. Given work proposes usage of the “WanderAI” system for non-player characters’ behaviors – a merge between reinforced machine learning and classic approaches to artificial intelligence programming in video games, such as: pathfinding algorithms (breadth first search, depth first search, greedy, Dijkstra’s and A*) for character’s navigation and finite state machines, queues, or behavior trees for character’s behavior states changes. Work includes overview of video game engines, pathfinding algorithms and state-space representation models. Also, introduction to key concepts and processes of video game development is given with explanation of their implementation, such as: game objects, game loop, environment setup (including scene setup, light sources setup, colliders and triggers setup, and objects placement) and scripting (including various things, such as: collision detection, runtime object editing, timers and etc.). This work also provides custom implementations of different game systems (such as: walkable waypoints system, “WanderAI” non-player character AI and vision system, player input and camera control system, objects and goal system, locations and objects visibility system, smooth object fading system, game reset and spawning system, and auto-play system) with code examples and comments. In the field of machine learning this work presents study and implementation of reinforced machine learning custom reward function and hyperparameters setup for given environment. Paper’s end goal and result is a video game prototype, working on OSX, Windows PC and Linux operation systems. Needed descriptions and comparisons regarding software tools and approaches used during development also given. Keywords: video game development; video game engines; artificial intelligence; machine learning; pathfinding algorithms; state-space representation models.