Comparative Analysis of Short Domain-specific Texts On-Device Classification Approaches Using Neural Projections

The development of the smartphone market and the rapid expansion of machine learning has led to more companies starting to use machine learning models in their mobile applications. This is evidenced by speech recognition on mobile phones, Smart Reply and Spoken Language Understanding. Additionally, the use of typical neural network models on mobile devices is often unfeasible, since huge models cannot fit into the limited memory available on such compact devices. We are proceeding from the fact that there is a solution to this problem - namely, it is ProjectionNet framework. It introduces a novel architecture that jointly trains the cumbersome ‘trainer’ neural network combined with the compact ‘projection’ model. However, only a few papers studying the behavior of combinations of ‘projection’ networks with various large neural networks for short text classification in the context of the specific subject-oriented domain. Therefore, the present paper aims at comparing models obtained using different deep learning models compressed with neural projection framework on several short text classification datasets. In order to do it, we implement various combinations of state-of-the-art text classification approaches and projection networks. Thereon, we jointly train them on the selected datasets. Finally, comparative analysis of models obtained is provided. The paper contains 50 pages, 11 figures, 12 tables, 43 sources and 1 appendix.