Unsupervised Wep Page Information Extraction Using CNN

This master thesis focuses on the problem of structured content extraction from HTML documents. The relevance of this paper lies with the fact that of today there are no open source projects on the topic that give admissible results given the ongoing rapid development of web pages. Old classical methods lose their functionality due dynamic loading, log-in requirements, CAPTCAs, IP blocking and Honeypots. Most of the existing methods were developed before wide application of neural networks. Since then both hardware technological development and machine learning algorithms improved drastically and allowed better quality of problem solving in related areas. Specifically, machine learning algorithms based on multilayer neural networks have recently led to significant improvements in such areas as computer vision and natural language processing. Considering this the assumption of this thesis is that applying these methods to structured information extraction ought to yield good results. The aim of this work is to explore the possibility to apply Convolutional Neural Networks architectures to the task structured information extraction. In this work a comprehensive research of the existing methods was conducted, based on which the new method of content extraction was proposed and implemented on the collected dataset. Keywords. Convolutional neural networks, HTML, content extraction.