AI Helps Discover New Space Anomalies
The SNAD team, an international network of researchers including Matvey Kornilov, Associate Professor of the HSE University Faculty of Physics, has discovered 11 previously undetected space anomalies, seven of which are supernova candidates. The researchers analysed digital images of the Northern sky taken in 2018 using a k-D tree to detect anomalies through the ‘nearest neighbour’ method. Machine learning algorithms helped automate the search. The paper is published in New Astronomy.
Most astronomical discoveries have been based on observations with subsequent computations. While the total number of observations in the 20th century was still relatively small, the volumes of data drastically increased with the arrival of large-scale astronomical surveys. For example, the Zwicky Transient Facility (ZTF), which uses a wide-field view camera to survey the Northern sky, generates ∼1.4 TB of data per night of observation and its catalogue contains billions of objects. Manually processing such enormous amounts of data is both expensive and time consuming, so the SNAD team of researchers from Russia, France and the US came together to develop an automated solution.
When scientists examine astronomical objects, they observe their light curves, which show variations of an object’s brightness as a function of time. The observers first identify a flash of light in the sky and then follow its evolution to see whether the light becomes brighter or weaker over time, or goes out. In this study, the researchers examined a million real light curves from the ZTF's 2018 catalogue and seven simulated live curve models of the types of objects under study. In total, they followed some 40 parameters, including the amplitude of an object's brightness and the timeframe.
Konstantin Malanchev, co-author of the paper and postdoc at the University of Illinois at Urbana-Champaign
'We described the properties of our simulations using a set of characteristics expected to be observed in real astronomical bodies. In the dataset of approximately a million objects, we were looking for super-powerful supernovae, Type Ia supernovae, Type II supernovae, and tidal disruption events. We refer to such classes of objects as anomalies. They are either very rare, with little-known properties, or appear interesting enough to merit further study.'
The light curve data from real objects was then compared to that of simulations using the k-D tree algorithm. A k-D tree is a geometric data structure for splitting space into smaller parts by cutting it with hyperplanes, planes, lines or points. In the current research, this algorithm was used to narrow down the search range when looking for real objects with properties similar to those described in the seven simulations.
Subsequently, the team identified 15 nearest neighbours, ie real objects from the ZTF database, for each simulation—105 matches in total, which the researchers then examined visually to check for anomalies. The manual verification confirmed 11 anomalies, of which seven were supernova candidates, and four were active galactic nuclei candidates where tidal disruption events could occur.
Maria Pruzhinskaya, a co-author of the paper and research fellow at the Sternberg Astronomical Institute
'This is a very good result. In addition to the already-discovered rare objects, we were able to detect several new ones previously missed by astronomers. This means that existing search algorithms can be improved to avoid missing such objects.'
This study demonstrates that the method is highly effective, while relatively easy to apply. The proposed algorithm for detecting space phenomena of a certain type is universal and can be used to discover any interesting astronomical objects, not limited to rare types of supernovae.
'Astronomical and astrophysical phenomena which have not yet been discovered are in fact anomalies. Their observed manifestations are expected to differ from the properties of known objects. In the future, we will try using our method to discover new classes of objects.'
Over three days, more than 300 conference participants attended workshops, seminars, sections and a poster session. During panel discussions, experts deliberated on the regulation of artificial intelligence (AI) technologies and considered collaborative initiatives between academic institutions and industry to advance AI development through megaprojects.
Top development teams around the world are trying to create a neural network similar to a curious but bored three-year-old kid. IQ.HSE shares why this approach is necessary and how such methods can bring us closer to creating strong artificial intelligence.
Seungmin Jin, from South Korea, is researching the field of Explainable AI and planning to defend his PhD on ‘A Visual Analytics System for Explaining and Improving Attention-Based Traffic Forecasting Models’ at HSE University this year. In September, he passed the pre-defence procedure at the HSE Faculty of Computer Science School of Data Analysis and Artificial Intelligence. In his interview for the HSE News Service, he talks about his academic path and plans for the future.
Machine Learning (ML) is a field of AI that examines methods and algorithms that enable computers to learn based on experience and data and without explicit programming. With its help, AI can analyse data, recall information, build forecasts, and give recommendations. Machine learning algorithms have applications in medicine, stock trading, robotics, drone control and other fields.
At the end of September, the HSE University Faculty of Computer Science launched the laboratory ‘Artificial Intelligence in Mathematical Finance’, which brought together more than 70 participants. Representatives of the laboratory told students and staff about the key tasks and goals of the new research department.
'Intelligence is ten million rules,' said Douglas Lenat, one of the creators of artificial intelligence (AI). For nearly four decades, he worked to instil 'common sense' in computers, painstakingly describing hundreds of thousands of concepts and millions of relationships between them.
Today, neural networks can easily identify emotions in texts, photos and videos. The next step is modelling them—an essential component of full-fledged intelligence in people and machines alike.
HSE AI Research Centre works on achieving applied results, striving not only to develop innovative algorithms and models, but also to put them into practice to solve real-world problems and tasks. Alexey Masyutin, Head of the AI Research Centre, spoke about some of the the specifics of applying AI technologies in various fields.
Fall into ML 2023: HSE University’s Faculty of Computer Science Organises Conference on Machine Learning
On October 26–28, HSE University’s Institute of Artificial Intelligence and Digital Sciences at the Faculty of Computer Science will hold a conference titled Fall into ML 2023 withsupport from the HSE University AI Research Centre . The event is dedicated to promising directions of fundamental AI development.
On September 4, the HSE University building on Pokrovsky Bulvar hosted ARTificial Fest, an event devoted to neural network art. The festival was organised by the HSE University Faculty of Creative Industries, the HSE Career centre, and the Chisty List (‘Blank Page’) student organisation. The event was open not only to students and staff of HSE University, but also to anyone interested in the blending of machine algorithms and art.