Gradient Boosted Decision Tree Net

We explore a new approach to the construction of neural networks, based on the compositions of decision trees used as layers. The tasks are to research and implement algorithms that allow both replacing some layers in the network with tree compositions, and implementing tree networks purely from from scratch. Implemented components should be trained as part of the network by the method of back propagation, but, in turn, within itself, implement the gradient boosted composition. The boostnet library was implemented, using which all the experiments and measurements were carried out. The library allows you to create and train usual neural networks, and use tree layers in them. Before learning, a calculation graph is constructed and used to distribute calculations between several threads, which makes it possible to more fully utilize available computing resources. With the help of the library, the task of constructing an auto-encoder was solved. We develop a whole range of optimizations, affecting both existing approaches, and developed new ones. The task of quickly tree differentiating, the task of existing tree tuning, pruning of compositions was solved.