Approach to Detecting Anomalies in Website Usage Data

Search engines have become a major tool to retrieve information and, thus, an important part of people’s daily life. Thereby, there is a huge turnover of money in search engine industry, which leads to the emergence of people trying to profit. Apart from honest users, there are examples of fraudulent behavior aiming to abuse search engine and/or other users. Consequently, it is essential to somehow detect these malicious users. Nevertheless, there are usually few examples of fraud that let us consider dishonest users as anomalies and apply anomaly detection techniques. The paper proposes a novel approach of detecting malicious users by detecting malicious entities, in particular, websites accessed, and comparison of existing anomaly detection algorithms. The transition from users to entities has become possible due to modeling the Web as bipartite user-entity graph. In order to develop the approach, we first introduce the definition of the term “anomaly”, conduct brief theoretical comparative analysis of different anomaly detection methods and select a few candidates taking into consideration specifics of dataset in use. Then, we implement selected methods and perform their empirical evaluation on real-world website usage data containing approximately one million objects. Finally, based on experimental results, we choose the most suitable for a given task and dataset in use method. The paper contains 85 pages, 3 sections, 33 figures, 2 tables, 143 references and 4 appendices.