Beauty in Details: HSE University and AIRI Scientists Develop a Method for High-Quality Image Editing
Researchers from the HSE AI Research Centre, AIRI, and the University of Bremen have developed a new image editing method based on deep learning—StyleFeatureEditor. This tool allows for precise reproduction of even the smallest details in an image while preserving them during the editing process. With its help, users can easily change hair colour or facial expressions without sacrificing image quality. The results of this three-party collaboration were published at the highly-cited computer vision conference CVPR 2024.
Artificial intelligence is already able to generate and edit images using generative adversarial networks (GANs). The architecture consists of two independent networks: a generator that creates images and a discriminator that distinguishes between real and generated samples. These networks compete with each other, and a new stage in their development is the StyleGAN model. This model can generate images and modify specific parts based on user requests, but it has not been able to work with real photos or images before.
Researchers from the HSE AI Research Centre, the Artificial Intelligence Research Institute (AIRI), and the University of Bremen have proposed a method to quickly and efficiently edit real images. This StyleFeatureEditor approach consists of two modules: the first inverts (reconstructs) the original image, and the second edits this reconstruction. The results of these two steps are passed to StyleGAN, which generates the edited image based on the internal representations. The developers addressed some challenges that had been encountered in previous research. With a small set of representations, the network could edit the image well, but it lost some details from the original. However, with a larger set, all the details were preserved, but the network had difficulty transforming them correctly according to the task.
To solve this, the researchers proposed a new solution: the first module finds both large and small representations, while the second learns how to edit the larger ones using the smaller ones as reference.
However, to train these modules to accurately edit the representations, the neural network requires both real images and their edited versions.
‘We needed examples, such as the same face with different expressions, hairstyles, and details. Unfortunately, such image pairs do not exist at the moment. So, we came up with a trick: using a method that works with small representations, we created a reconstruction of a real image and an example of editing this reconstruction. Although the examples were relatively simple and without details, the model clearly understood how to make the edits,’ explains Denis Bobkov, one of the authors of the article, a research intern at the Centre of Deep Learning and Bayesian Methods of the AI and Digital Science Institute (part of the HSE Faculty of Computer Science), and a Junior Research Fellow at AIRI’s Fusion Brain Lab.
However, training only on generated (simple) examples leads to a loss of detail when working with real (complex) images. To prevent this, the researchers added real images to the training dataset, and the neural network learnt to reconstruct them in detail.
Thus, by showing the model how to edit both simple and complex images, the scientists created conditions under which the network could edit complex images more effectively. In particular, the developed approach handles adding new elements of style while preserving the details of the original image better than other existing methods.
In the case of simple reconstruction (first row), StyleFeatureEditor accurately reproduced a hat, while most other methods almost completely lost it. The developed method showed the best results with additional accessories (third row): most methods could add glasses, but only the StyleFeatureEditor retained the original eye colour.
‘Thanks to this training technique on generated data, we have obtained a model with high editing quality and a fast processing speed due to the use of relatively lightweight neural networks. The StyleFeatureEditor framework requires only 0.07 seconds to edit a single image,’ says Aibek Alanov, Head of the Centre of Deep Learning and Bayesian Methods of the AI and Digital Science Institute (part of the HSE Faculty of Computer Science), and leader of the research group ‘Controlled Generative AI’ at AIRI's Fusion Brain Lab.
The research was funded by a grant from the Analytical Centre under the Government of the Russian Federation for AI research centres.
The research results will be presented at the Fall into ML 2024 conference on artificial intelligence and machine learning, which will take place at HSE University on October 25–26, 2024. Leading AI scientists will discuss the best papers published at top-tier (A*) flagship AI conferences in 2024. A demo of the developed method can be tried out on HuggingFace, and the source code is available on GitHub.
See also:
Men Behind the Wheel: Three Times More Violations and Accidents than Women
Men are three times more likely than women to commit traffic violations while driving and to be involved in accidents. Moreover, they are more likely to create situations on the road that are highly dangerous to others. Men are also twice as likely to drive under the influence and nearly one-third more likely to receive a prison sentence for reckless driving. Perhaps it comes down to cultural norms and the different attitudes men and women have toward driving. These are the conclusions reached by Anton Kazun, Assistant Professor at the HSE Faculty of Economic Sciences, and Research Assistant Mikhail Belov.
HSE Scientists Discover How to Predict Charitable Behaviour Through Physiological Reactions
Researchers at the HSE Institute for Cognitive Neuroscience have investigated how the emotional impact of advertising affects the amount people willing to donate to support animal welfare. To accomplish this, the researchers measured physiological responses such as heart rate, electrodermal activity, and facial expressions in individuals viewing various photos of dogs. The findings indicate that willingness to donate is most accurately predicted by heart rate and facial muscle activation. The study has been published in Social Psychology.
'We Are Creating the Medicine of the Future'
Dr Gerwin Schalk is a professor at Fudan University in Shanghai and a partner of the HSE Centre for Language and Brain within the framework of the strategic project 'Human Brain Resilience.' Dr Schalk is known as the creator of BCI2000, a non-commercial general-purpose brain-computer interface system. In this interview, he discusses modern neural interfaces, methods for post-stroke rehabilitation, a novel approach to neurosurgery, and shares his vision for the future of neurotechnology.
First Successful Attempt in 55 years: Physicists in Russia and Germany Confirm 1969 Experiment Results
A team of researchers, with the participation of physicists from HSE University, replicated the 1969 experiment on superconductivity and its properties. The scientists induced superconductivity by deliberately deteriorating the interfaces between the layers of superconductors and ferromagnets in the system, resulting in better performance of spin valves compared to the classical version, where the interfaces between the layers are ideal. This approach could lead to the development of more efficient devices for data storage and computing. The study findings have been published in the Beilstein Journal of Nanotechnology.
Healthy Nutrition Saves Public Funds: Strategies to Reduce Healthcare Costs in Russia
In Russia, the annual cost of treating type 2 diabetes alone exceeds 500 billion roubles. Promoting healthy nutrition programmes can ease the burden on the healthcare system and increase life expectancy. This was the conclusion reached by economists at HSE University after analysing global experiences with government involvement in promoting a healthy lifestyle.
Conscientious Individuals Live Longer
Personality traits such as conscientiousness, emotional stability, and an internal locus of control significantly influence one's lifestyle and longevity. Not only can personality traits influence health through beneficial and harmful habits but can also have a direct effect on mortality. Higher conscientiousness reduces the risk of premature death by 20 percentage points, while higher neuroticism increases it by 12 percentage points. These are the findings from a new study by Ksenia Rozhkova, Junior Research Fellow at the Laboratory for Labour Market Studies of the HSE Faculty of Economic Sciences.
Esports Players Play Better Online
In competitions, esports players, like other athletes, face stress and show worse results due to pressure. A substantial decrease takes place in the performance of esports players during overtime. This effect, however, is significantly mitigated in online competitions compared to live events—the difference can reach 30%. A study by a team of authors from HSE University’s Moscow and Perm campuses and European University Viadrina (Germany) explores the phenomenon of choking under pressure within the context of esports. The study was published in the Journal of Economic Behavior & Organization.
Analysing Genetic Information Can Help Prevent Complications after Myocardial Infarction
Researchers at HSE University have developed a machine learning (ML) model capable of predicting the risk of complications—major adverse cardiac events—in patients following a myocardial infarction. For the first time, the model incorporates genetic data, enabling a more accurate assessment of the risk of long-term complications. The study has been published in Frontiers in Medicine.
A New Tool Designed to Assess AI Ethics in Medicine Developed at HSE University
A team of researchers at the HSE AI Research Centre has created an index to evaluate the ethical standards of artificial intelligence (AI) systems used in medicine. This tool is designed to minimise potential risks and promote safer development and implementation of AI technologies in medical practice.
Russian Scientists Have Studied Marine Worm with Unique Spermatozoa
Russian scientists at Zhirmunsky National Scientific Centre of Marine Biology, HSE University, and Moscow State University have studied Phoronis embryolabi, a rare species of marine invertebrate found in the waters of the Sea of Japan. This species is notable for its unique reproductive system, which includes the development of larvae within the parent’s body and an unusual sperm structure. The study's findings contribute to our understanding of the evolutionary adaptations of marine organisms to extreme conditions. The study has been published in Zoologischer Anzeiger.