Beauty in Details: HSE University and AIRI Scientists Develop a Method for High-Quality Image Editing
Researchers from the HSE AI Research Centre, AIRI, and the University of Bremen have developed a new image editing method based on deep learning—StyleFeatureEditor. This tool allows for precise reproduction of even the smallest details in an image while preserving them during the editing process. With its help, users can easily change hair colour or facial expressions without sacrificing image quality. The results of this three-party collaboration were published at the highly-cited computer vision conference CVPR 2024.
Artificial intelligence is already able to generate and edit images using generative adversarial networks (GANs). The architecture consists of two independent networks: a generator that creates images and a discriminator that distinguishes between real and generated samples. These networks compete with each other, and a new stage in their development is the StyleGAN model. This model can generate images and modify specific parts based on user requests, but it has not been able to work with real photos or images before.
Researchers from the HSE AI Research Centre, the Artificial Intelligence Research Institute (AIRI), and the University of Bremen have proposed a method to quickly and efficiently edit real images. This StyleFeatureEditor approach consists of two modules: the first inverts (reconstructs) the original image, and the second edits this reconstruction. The results of these two steps are passed to StyleGAN, which generates the edited image based on the internal representations. The developers addressed some challenges that had been encountered in previous research. With a small set of representations, the network could edit the image well, but it lost some details from the original. However, with a larger set, all the details were preserved, but the network had difficulty transforming them correctly according to the task.
To solve this, the researchers proposed a new solution: the first module finds both large and small representations, while the second learns how to edit the larger ones using the smaller ones as reference.
However, to train these modules to accurately edit the representations, the neural network requires both real images and their edited versions.
‘We needed examples, such as the same face with different expressions, hairstyles, and details. Unfortunately, such image pairs do not exist at the moment. So, we came up with a trick: using a method that works with small representations, we created a reconstruction of a real image and an example of editing this reconstruction. Although the examples were relatively simple and without details, the model clearly understood how to make the edits,’ explains Denis Bobkov, one of the authors of the article, a research intern at the Centre of Deep Learning and Bayesian Methods of the AI and Digital Science Institute (part of the HSE Faculty of Computer Science), and a Junior Research Fellow at AIRI’s Fusion Brain Lab.
However, training only on generated (simple) examples leads to a loss of detail when working with real (complex) images. To prevent this, the researchers added real images to the training dataset, and the neural network learnt to reconstruct them in detail.
Thus, by showing the model how to edit both simple and complex images, the scientists created conditions under which the network could edit complex images more effectively. In particular, the developed approach handles adding new elements of style while preserving the details of the original image better than other existing methods.
In the case of simple reconstruction (first row), StyleFeatureEditor accurately reproduced a hat, while most other methods almost completely lost it. The developed method showed the best results with additional accessories (third row): most methods could add glasses, but only the StyleFeatureEditor retained the original eye colour.
‘Thanks to this training technique on generated data, we have obtained a model with high editing quality and a fast processing speed due to the use of relatively lightweight neural networks. The StyleFeatureEditor framework requires only 0.07 seconds to edit a single image,’ says Aibek Alanov, Head of the Centre of Deep Learning and Bayesian Methods of the AI and Digital Science Institute (part of the HSE Faculty of Computer Science), and leader of the research group ‘Controlled Generative AI’ at AIRI's Fusion Brain Lab.
The research was funded by a grant from the Analytical Centre under the Government of the Russian Federation for AI research centres.
The research results will be presented at the Fall into ML 2024 conference on artificial intelligence and machine learning, which will take place at HSE University on October 25–26, 2024. Leading AI scientists will discuss the best papers published at top-tier (A*) flagship AI conferences in 2024. A demo of the developed method can be tried out on HuggingFace, and the source code is available on GitHub.
See also:
‘The Peak of Stupidity’ and ‘The Valley of Despair’: HSE Economists Propose an Explanation for the Dunning–Kruger Effect
The Dunning–Kruger effect, which describes a sharp surge in self-confidence among beginners followed by an equally rapid decline as they gain experience, can be explained by the nature of the learning process and the acquisition of new knowledge. This conclusion was reached by Andrey Vorchik of the HSE Faculty of Economic Sciences together with independent researcher Murat Mamyshev. They developed a mathematical model of learning and demonstrated how subjective confidence is formed and changes as knowledge accumulates, as well as how teachers can reduce the ‘valley of despair’ experienced by learners.
Toffee and Risk: Scientists Discover Why People Who Crave Sweets Make More Impulsive Choices
Having a sweet tooth may be linked not only to eating habits but also to the way people make decisions. Researchers at HSE University have found that people with a preference for sweet foods tend to behave more impulsively—not because they want immediate rewards, but because they are less willing to tolerate uncertainty. These findings may help improve treatments for addiction. The study findings have been published in Frontiers in Psychology.
Physicists Find a Way to Model Ion Parameters in Plasma in Seconds
Researchers from HSE University and the Moscow Institute of Physics and Technology (MIPT) have developed a set of simple analytical methods for calculating the properties of heavy ions in helium under the influence of a strong electric field. The new approach speeds up calculations of ion mobility and ion–molecule reaction rates by thousands of times while maintaining sufficient accuracy for plasma jet modelling. The findings have been published in the journal Physica Scripta.
Two Years of Growth or Decline: How to Choose an Investment Strategy
Economists from HSE University, together with colleagues from international universities, have analysed stock market movements over almost a century and proposed an investment strategy that could have delivered returns nearly twice as high as the market average. Their research suggests following a momentum strategy during periods of sustained market growth and switching to a value strategy after prolonged market declines. The study has been published in the Journal of Banking and Finance.
Researchers Reveal Link Between Attention and Communication Difficulties in Autism
Researchers at HSE University have examined how communication difficulties in children with autism are related to brain function. The findings show that not only language networks but also attention networks play an important role. The weaker the connections involved in maintaining focus and switching attention, the more pronounced communication difficulties were. The study has been published in European Child & Adolescent Psychiatry.
Scientists Discover Why Some People Wore Masks During COVID-19 While Others Did Not
Why do some people voluntarily follow new rules while others ignore them? Researchers at HSE University have found that the answer lies not so much in people's willingness to cooperate, as previously believed, but in their ability to empathise with others. Empathy proved to be the strongest predictor of whether people chose to wear face masks voluntarily during the COVID-19 pandemic. The findings have been published in Frontiers.
Physicists at HSE University and FIAN Discover Way to 'Photograph' Sound for Testing Materials Used in 6G Communications
Researchers at HSE University, in collaboration with colleagues from the Lebedev Physical Institute of the Russian Academy of Sciences (FIAN), have developed a method for rapidly determining how firmly a film is bonded to a substrate. This is important for the creation of ultrahigh-frequency acoustic filters, which are key components of next-generation 5G and 6G communications. For the first time, researchers have succeeded in measuring the lateral rigidity of the bond between a two-dimensional material film and a substrate in this way. The study results have been published in Applied Physics Letters.
Scientists Create Open Dataset for Studying Concentration
A team of Russian researchers, including scientists from HSE University–St Petersburg, has developed the first open multimodal dataset containing recordings of brain activity, heart function, and video observations to help researchers understand what happens in the human brain during deep concentration. In the future, the dataset could accelerate the development of neural interfaces, rehabilitation technologies, and AI systems. The article has been published in Scientific Data.
Scientists Propose Method for More Efficient Resource Use in Machine Learning
An international group of researchers, including mathematicians from the AI and Digital Science Institute at the HSE Faculty of Computer Science, has provided a theoretical justification for a simple and computationally efficient method of estimating uncertainty in Stochastic Gradient Descent (SGD). The paper has been published on the scientific preprint server arXiv.org and presented at AISTATS 2026.
Team Success: Aligning Means with Objectives
In corporations, sports, and academia, people often face challenges they cannot handle alone. In such cases, selecting the right team is crucial. Tatiana Mayskaya, Associate Professor at the HSE Faculty of Economic Sciences and the International College of Economics and Finance, together with colleagues from foreign universities, examined team characteristics and found that less diverse teams are better suited to objectives where a high average performance is important, whereas more diverse teams are preferable when avoiding failure is critical. The paper has been published in Economic Theory.