Distributed System for Document Storage and Clustering

<div>Graduate work &quot;Distributed System for Document Storage and Clustering&quot; has a volume of 43 pages, contains 5 figures and 4 tables. It contains 47 items in a list of sources.</div><div>&nbsp;</div><div>The structure of the work includes: introduction, review of sources and similar decisions, 2 main chapters, the chapter about experimental testing of the developed system and the conclusion.</div><div>&nbsp;</div><div>Graduate work is devoted to design and experimental testing of the distributed system for processing of collections of textual documents.&nbsp;</div><div>In the work, the methods of clustering and document retrieval based on PLSI and LDA models are implemented. These methods allow to make special clustering of documents in collections, extract themes from collections and used for document retrieval in the work. Also new method for document retrieval based on abstract suffix trees (ASD) was designed and implemented. It includes special preliminary processing of document collections, based on selection of particular features for constructing of feature index. All implemented methods were included in the distributed system for document collections processing. An experimental comparison of the implemented methods was performed using &quot;Ozon&quot; web store collections of goods, which are available for download. Experimental results have shown author&#39;s modifications can improve document retrieval quality, especially in the case of incomplete search queries or queries which contains errors. Distributed processing of collections can reduce the time spent collections processing. Preliminary processing of document collection reduces the time spent query performing and allows to use Abstract Suffix Trees (ASD) as a powerful mathod for document retrieval.&nbsp;</div><div>&nbsp;</div><div><strong>Keywords: </strong>Latent Dirichlet Allocation (LDA), Probabilistic Latent Semantic Analysis (PLSI), Abstract Suffix Trees (ASD), Text Documents Clustering, Document Retrieval, Distributed Text Processing Systems</div>