Reinforecement Learning for DSGE Models

Dynamic Stochastic General Equilibrium (DSGE) models are a class of macroeconomic models used by central banks to understand business cycles, shocks, and the effects of policy decisions on the economy. Each model consists of a number of economic agents with utility functions that interact with one another, reflecting the complex processes of the real economy. The interactions among agents can be characterized as dynamic and adaptive, allowing them to adjust their strategies based on external conditions and internal motivations. We propose a perfectly scalable reformulation of DSGE models in the language of agents and markets. Agents hold balances of different types of resources and exchange them with each other in the markets. Each agent aims to construct an optimal exchange chain that maximizes his utility function. To replicate perfect competition, some markets have a special type of agent—market makers. Their task is to set prices so as to balance supply and demand. To build a DSGE model within this framework, one needs to create agents, markets, and connect agents with arrows to the markets they have access to. To obtain equilibrium trajectories, we apply multi-agent reinforcement learning algorithms for both regular agents and market makers. To demonstrate that our reformulation is equivalent to the standard understanding of DSGE models, we compare the trajectories of macroeconomic variables implied by our environment with the theoretical trajectories in simple DSGE models. As reference models, we use the Real Business Cycle Model and the Extended Real Business Cycle Model with two types of households. We replicate both models within our framework, generate trajectories of macroeconomic variables using reinforcement learning, and demonstrate that these trajectories closely match the theoretical predictions.