Construction of Neural Network Frame Based on Formal Concept Lattice

In this paper we consider the problem of constructing a neural network model, in case when we haven’t a priori information about the topology. The selection of optimal network architecture is crucial, as it has high impact on the results of classification. Indeed, if the structure is too small, then we have only few parameters to fit the model, so the resulting accuracy will not be high. On the other hand, with large architecture neural network is more likely to over-fit the training data. This work attempts to propose an approach based on covering relation on closed subsets in a data set. It is based on the assumption that there is a connection between the complexity of optimal network architecture and the depth of corresponding formal concept lattice. The main advantage of such approach lies in the possibility of eliminating redundant nodes and edges, by choosing only qualitative formal concepts, and thus obtaining straightforward architecture. Another advantage is in the possibility to pre-train the network, using the interpretation of nodes as formal concepts. There are two main types of architecture models are studied in the research. The first one is based on the ordinary definition of formal concept, and the second one is constructed with use of monotone Galois connections. For every type we have made a comparative empirical analysis of various metrics and algorithms, and as a result we have obtained the optimal structure for further experiments. In addition, the difference between pre-training a neural network and initializing the random weights was studied. In the final section we have compared the constructed models with other classification methods. In order to make comparison complete, we chose 7 different real datasets with various size, density, and the number of classes. A practical comparison has proven the possibility to use the mentioned models on real datasets, revealed pros and cons of the developed methods.