HSE Student to Present at Association for Computational Linguistics Conference
An article by HSE Faculty of Computer Sciences senior Artyom Gadetsky was accepted for presentation at the Association for Computational Linguistics international conference, the only A* level conference on computational linguistics. According to the CORE system, which ranks large conferences in computer science, the category A* is the highest level for a conference.
Artyom prepared his paper together with Faculty of Computer Science Professor Dmitry Vetrov and Ilya Yakubovsky, a researcher at the company Joom. The paper, ‘Conditional Generators of Words Definitions,’ will be presented at the conference, which is set to take place in Melbourne, Australia, on July 15-20.
About the Article
Text oftentimes serves as data in tasks involving machine learning. Such text is usually broken up by words and then transformed into a vector – a set of numbers with which machine learning algorithms can already work comfortably. Artyom Gadetsky’s work researches this sort of vector representation of words. One vector is usually used for a single word, but since there are polysemantic words, it is not obvious that this representation stores information on all meanings of a word. The paper presents a model for generating word meanings. With the help of this model, various definitions can be generated for different meanings of the same word, with an example of word usage serving as the foundation. See below for an example of how this works with simple words such as star, sentence, and head:
|Word||Usage Example||Generated Definition|
|star||she got star treatment||a person who is very important|
|star||bright star in the sky||a small circle of a celestial object or planet that is seen in a circle|
|sentence||sentence in prison||an act of restraining someone or something|
|sentence||write up the sentence||a piece of text written to be printed|
|head||the head of a man||the upper part of a human body|
|head||the leader of organization||with the highest rank or position|
The research also showed that one such representation contains information about numerous possible meanings. In addition, there is reason to believe that separate components of the word vectors could be responsible for parts of speech and various other word properties.
Artyom Gadetsky, fourth-year student in the Applied Mathematics and Information Science programme
Building interpretable models is an important field that is still developing, as a full understanding of how such methods work allows you to find limitations and fix them. Our work is a step towards understanding how neural networks learn.
Dmitry Vetrov, Head of the Samsung-HSE Laboratory
This work is an excellent experimental confirmation that modern neural networks extract much more information on data than previously believed. A neural network not only understands the meanings of words depending on context, but is also able to formulate its own understanding in human language. This opens up new opportunities to build interpretable AI models, where a neural network, aside from decision-making, immediately generates a textual description of why it made this decision in particular. The technology proposed in our article presents both a concrete practical use in the financial sphere, for example, as well as a more fundamental use. It does this by showing that even relatively primitive (as far as fully fledged AI is concerned) machine learning technologies have unexpected properties that are traditionally attributed to higher nervous activity. This means that another step has been taken towards creating a complete artificial mind.
The 8th RuSSIR-2014 (Russian Summer School in Information Retrieval), organized by the Higher School of Economics in Nizhny Novgorod and ROMIP, has ended.