Applying a Neural Network Architecture with Spatio-Temporal Connections to the Maze Exploration

The paper reviews a Reinforcement Learning model, which uses a modified neural network architecture with spatio-temporal weights also well-known as Temporal Hebbian Self-Organizing Map (THSOM). We claim that the states clustering can be successfully applied to such kind of environment as mazes what leads to usage of relatively small neural networks, as a result of which the training time and computational costs can be significantly reduced. We conducted many experiments where we were trying the model with the solving labyrinths of different complexity. In the most cases the model demonstrated sustainable learning, building close to optimal path. In this work we are considering in details how different parameters of the algorithm affect the speed and convergence of the agent's learning process, as well as the time of maze solving. At the chapter 3 we provide a critics of the model, consider some non-trivial for an agent cases and give advice on how to overcome them. Overall, based on the obtained results, it can be judged that the experiment considered in this work was successful and we succeeded in developing an algorithm of the agent's behavior that is applicable in real conditions.