Reinforcement Learning for Knowledge Graph-based Question Answering

In this paper, we are solving the problem of answering questions by walking around a knowledge graph (KG). More specifically, we formulate the problem in terms of reinforcement learning and train an agent to find a path in KG that leads from starting entity to the target, according to the given relation between them. Our agent is a transformer-based model trained with REINFORCE algorithm to sample the most promising edge and walk over it. All prior reinforcement learning approaches used recurrent networks for generating embeddings and simple MLPs for an action prediction. We noticed that a trajectory in the knowledge graph can be seen as a sequence of entities and relations. In this case the problem can be formulated as a sequence labeling taks and transformer-based model is a perfect solution to it. We also propose two modifications of the reward function: path lenght penalty and positive reward for the wrong answer. Experimental results on knowledge graph dataset WN18RR show that our approach outperforms stateof-the-art methods by a large margin.