Sim2Real for Autonomous Driving Using the Latent Space

The field of autonomous driving algorithms has been actively developing recently. Existing industrial autopilots for motorway driving, parking, and general city traffic are drawing particular attention. Such algorithms are very time-consuming to implement, maintain, and adapt. To simplify such algorithms, it is possible to use machine learning methods, namely end-to-end learning. However, this approach also has challenges, such as the requirement for a large amount of collected data in the real world. Sim2Real tries to address these challenges by using a simulator to train and transfer the driving algorithm. In this work, I develop a machine learning model that learns from control commands in a simulator and is tested in the real world. I propose a novel use of temporal order to improve the baseline approach. I demonstrate the benefits of the method and the temporal order constraints using the Duckietown platform as an example. Under experimental conditions, the basic Sim2Real model improves the transfer efficiency by 3-4 times relative to the trivial transfer, while the addition of temporal constraints – by 4-6 times. The practical significance of this work lies in the way to use temporal order, which is inseparable from continuous driving data, and, as a result, to improve the quality of simulated models in the real world. Keywords: autonomous driving, simulator, transfer from the simulator to the real world, temporal order.