Bachelor
2023/2024
Machine Learning 2
Type:
Compulsory course (Data Science and Business Analytics)
Area of studies:
Applied Mathematics and Information Science
Delivered by:
Big Data and Information Retrieval School
Where:
Faculty of Computer Science
When:
4 year, 1, 2 module
Mode of studies:
offline
Open to:
students of one campus
Language:
English
ECTS credits:
5
Contact hours:
56
Course Syllabus
Abstract
The course "Machine learning 2" is dedicated to the introduction to deep learning and natural language processing problems at the intersection of disciplines such as machine learning, deep learning, and linguistics. The course consists of three parts: (1) introduction to deep learning, (2) the basics part which covers the main concepts, models, and (3) task formulations, and the advanced part that focuses on industrial applications and modern scientific research.
Learning Objectives
- Study what deep learning is, which spheres of AI it embraces and learn the basics of each section
- Learn to implement neural network models
- Study basic tasks and methods of natural language processing and text analysis
- Study modern neural network models for natural language processing
- Acquire knowledge of software systems and tools for text processing and analysis
Expected Learning Outcomes
- Be able to apply basic word processing and analysis techniques
- Be able to formulate and solve problems related to language modeling and specialized problems on text data
- Know the ethical aspects of word processing
Course Contents
- Neural Networks. Backpropagation. Gradient Descent. Optimization
- ‘’Special layers’’. Weights initialization. Regularization.
- Image Recognition. Convolutional Neural Networks (CNN). Augmentation.
- Transfer learning. Fine-tuning. TracIn. AutoEncoders.
- Embeddings. Recurrent Neural Networks (RNN). Transformers.
- Automatic Speech Recognition.
- Distillation. Uncertainty Estimation. Quantization. Active Learning.
- Recommender Systems.
- NLP Introduction. Statistical text analysis.
- Vector text representation models
- Texts classification
- Sequence labelling
- Language models
- Syntax parsing
- Machine translation
- Pretrained language models
- Text generation
- Text markup, active learning
- Question-answering systems
- Multimodal methods
- Multi-language models
- Information extraction & information search
- Text summarization
- Ethical issues in natural language processing
Bibliography
Recommended Core Bibliography
- Deep learning, Goodfellow, I., 2016
- Introduction to natural language processing, Eisenstein, J., 2019
Recommended Additional Bibliography
- Speech and language processing : an introduction to natural language processing, computational linguistics, and speech recognition, Jurafsky, D., 2009