• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Student
Title
Supervisor
Faculty
Educational Programme
Final Grade
Year of Graduation
Mikhail Sadov
The Research of Methods for Texts Classification for Unbalanced Data
Fundamental and Applied Linguistics
(Bachelor’s programme)
10
2016
This research was dedicated to analyzing methods for texts classification for unbalanced data, also the most effective of them was defined and implemented taking into account the pecularities of the Russian language. It was found that the most effective way to work with unbalanced data is to change the output threshold or classes weights rather than changing the distribution of the training data set randomly or synthetically. Thus, it was defined that the most effective approach is to use cost-sensitive methods.

For empirical verification of theoretical conclusions was selected the task of classifying statements in Russian depending on whether they are correct or contain any offensive content. As a result, it was found that the usage of cost-sensitive methods did not help to improve the results of the texts classification compared to training model on moderately unbalanced data for this task. However, after conducting such experiment using strongly skewed data, it was found that the working with weighted models indeed leads to an improved classification result.

Student Theses at HSE must be completed in accordance with the University Rules and regulations specified by each educational programme.

Summaries of all theses must be published and made freely available on the HSE website.

The full text of a thesis can be published in open access on the HSE website only if the authoring student (copyright holder) agrees, or, if the thesis was written by a team of students, if all the co-authors (copyright holders) agree. After a thesis is published on the HSE website, it obtains the status of an online publication.

Student theses are objects of copyright and their use is subject to limitations in accordance with the Russian Federation’s law on intellectual property.

In the event that a thesis is quoted or otherwise used, reference to the author’s name and the source of quotation is required.

Search all student theses