Scientometrics at HSE

HSE is an ambitious research university, which means that apart from producing high-quality work, it also needs to measure its success. Scientometrics, broadly speaking, is a branch of knowledge concerned with measuring various characteristics of research, especially communication in research. Bibliometrics, as a sub-branch of scientometrics, has become highly accessible to the research and general public, and seemingly brought on a simple way to evaluate research productivity. However, this simplicity is deceptive, and led to the so-called 'Metric Tide': an overwhelming wave of poorly thought through indicators and target goals which are used to make decisions in research management. In order to use bibliometrics reasonably and purposefully HSE created a Scientometrics Centre, which is headed by Ivan Sterligov and focuses on the applied use of scientometrics.

Best practices

Scientometrics Centre uses the best practices and approaches developed by leading centers around the world, such as Centre for Science and Technology Studies (CWTS) at Leiden University (the Netherlands). It’s famous for producing Leiden Ranking, which gives information about 750 major world universities. It’s also widely known for developing SNIP (Source-Normalized Impact per Paper) metric which provides a more nuanced measure of publications’ impact, especially through providing corrections for differences in various scientific fields.

HSE actively uses this indicator for analyzing publications of its researchers and faculty members in Scopus journals. It is especially useful to compare publications in different fields of HSE’s research, as well as to assess the level of “fresh” publications which have not accumulated a lot of citations yet. The Faculty of Mathematics has been leading by this indicator for several years, with the Faculty of Economics taking the leading position only recently.

HSE also uses the tools developed by colleagues in Canada (Science-Metrix), France (Observatoire des sciences et des techniques  -OST) and Germany (AG Bibliometrie). There are barely more than a dozen similar centers around the world, and no direct counterparts in Russia, but instead there is a small but dedicated professional community with an interest in scientometrics. Scientometrics Centre was established in late 2015 and is headed by Ivan Sterligov; its team has been working on these issues at HSE for a longer time, as a part of the Office of Research Evaluation.

Scientometrics Centre: applied instruments for research evaluation

The centre works predominantly on analyzing citations of HSE publications indexed by Web of Science and Scopus, but it also analyzes other resources related to production of knowledge and research (research networks, personal pages and CVs of scientists, grant applications etc.) and does qualitative research together with Russian and international colleagues. It is not scientometrics in the strict sense, but it is part of the current understanding of how indicators and peer review make for good research evaluation tool when combined.

Last year HSE’s scientometrics team finished a large-scale two-year project on ranking Russian research journals: around 700 experts were involved in assessing journals in 14 thematic areas which HSE focuses its research on. The resulting lists of high-quality journals will be used by various HSE commissions for review, hiring or research. The centre also updates the “blacklist” of journals which do not adhere to the desired academic standards, so that HSE researchers can make informed decisions when choosing a publication venue.

The Centre also does analytical work for clients outside HSE, including other Russian universities, office of the Russian Academic Excellence Project 5-100, and international clients. It also works on rankings performance predictions jointly with colleagues from Ranking Information Centre. Due to lack of information about the methodology of rankings calculations, it is difficult to predict how the year’s results will be interpreted, and Scientometrics Centre helps to build predictions on analysis of past evaluations.

Resources for HSE researchers

In addition to the list of recommended journals and their “blacklist” the Centre also consults researchers about how best to choose publication venues and how sceintometrics and its tools can help with that. Another useful instrument developed by the Centre is a database of research grants and competitions for which Russian universities are eligible, and its’ possible to subscribe to the newsletter about opportunities in the fields which interest you by writing to Alfiya Enikeeva.  Scientometrics Centre also consults on how to best use SciVal database which offers an overview and analysis of research results in 220 countries and 4600 research institution. It can be used for refining the focus of research projects and finding collaborators. 

Based on Measuring Science 1(1), 2016, a supplement to the university bulletin Okna Rosta. English text prepared by the HSE Look, English-language HSE bulletin