The NPC-Behavior Subsystem Based on Reinforcement Learning

Continuous development of the Artificial Intelligence theory and simulated robotics contributed to the emergence of various real-world applications based on general learning models both from the “software” and “physical” perspectives prominent in the domain of reinforcement learning. In order to overcome rigidness and non-idempotency of the real-world, an idea of environment simulation was introduced, where computer-generated visuals and physics become the ground truth for target models, providing both rich control, stability and potential for further replication in real scenarios. Resulting complexity of the simulation topic lead to studies being bounded by the availability of conceptual models for simulation-driven reinforcement learning, leading to niche solutions. In the last decade, domain of game simulations became the main platform for reinforcement learning research, further driven by systems like AlphaGo and AlphaStar. Game environments essentially allowed us to remove bounds of research by bringing interpretability of simulation scenarios, accessibility and rich instruments for virtual world creation in the face of 3D-Engines. In this paper we provide the conceptual and architectural model of the framework for simulated reinforcement learning based on the novel notion of bilateral dynamics, which imposes modularity of reinforcement learning components, and extensive means of environment and agent control. Proposed framework is focused on supporting the subsequent development of an accessible toolkit for both research and application of RL-based solutions by its integration with Unreal Engine 4 as NPC behavior modification subsystem. Proposed bilateral framework is evaluated through showcasing of several RL models applications both from the simulation perspective and integration capabilities with external instruments. The study itself is done as a part of the research of the HSE University International Laboratory for Intelligent Systems and Structural Analysis of sandboxed simulation of robot agents controlled by RL algorithms.