Deep Learning 1

2026/2027

Учебный год

ENG

Обучение ведется на английском языке

4

Кредиты

Кто читает:

Департамент больших данных и информационного поиска

Статус:

Курс по выбору

Когда читается:

3-й курс, 1, 2 модуль

Преподаватели

Кирюхов Григорий Петрович

Шипилов Фома Александрович

Course Syllabus

Abstract

The course serves as an in-depth introduction to deep learning -- one of the most rapidly developing fields of machine learning. The studentswill learn the basic working principles of neural network architectures that are in widespread use today. he course has a clear practical focus; students are expected to implement and train neural networks in Python programming language using PyTorch framework. The course covers basic computer vision and natural language processing tasks, introduction to generative models, self-supervised learning, and model optimization and deployment.

Learning Objectives

• Understand the operating principle and be able to train the following types of neural networks: fully connected, convolutional, recurrent, and transformers.
• Proficiency in using the PyTorch framework for training neural networks.
• To understand the various tasks that can be solved using deep learning.

Expected Learning Outcomes

• The ability to process data, adapt neural network architectures, and build pipelines for training neural networks.

Course Contents

• Computation graphs, fully connected neural networks
• Loss functions, normalization, regularization
• Optimization of neural networks
• Convolutional neural networks
• Modern convolutional architectures
• Computer vision tasks
• Text processing (encodings, embeddings)
• Recurrent neural networks, text processing
• Transformer architecture
• Self-supervision, BERT
• Applications of transformers in computer vision
• Distillation, pruning, quantization
• Adversarial attacks

Assessment Elements

• BHW 1 (Big homework 1)
  Image classification, competition format
• SHW 2 (Small homework 2)
  Convolutional neural networks (10 points)
• SHW 3 (Small homework 3)
  language models (15 points)
• SHW 4 (Small homework 4)
  Unsupervised learning, 10 points
• BHW 2 (Big homework 2)
  Machine translation, competition format
• SHW 1 (Small homework 1)
  Training neural networks using NumPy, fully connected neural networks (20 points)
• Test 1
  Fully connected neural networks, optimization of neural networks
• Test 2
  Image processing, convolutional neural networks
• Exam
  Written test, in the classroom, no materials allowed, 2 hours
• Test 3
  Text processing, recurrent neural networks, transformer architecture
• Test 4
  Self-supervision, distillation, quantization, pruning, adversarial attacks, generative adversarial networks

Interim Assessment

• 2026/2027 2nd module
  Result = Rounding(0.3 * BHW + 0.25 * SHW + 0.15 Test + 0.3 * Exam), BHW — the average grade for all big homework assignments, SHW — the average grade for all small homework assignments

Bibliography

Recommended Core Bibliography

• Kelleher, J. D. (2019). Deep Learning. Cambridge: The MIT Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=2234376
• Neural Networks and Deep Learning - CCBY4_068 - Michael Nielson - 2022 - Open Educational Resources: libretexts.org - https://ibooks.ru/products/390854 - 390854 - iBOOKS

Recommended Additional Bibliography

• Ian Goodfellow, Yoshua Bengio, & Aaron Courville. (2016). Deep Learning. The MIT Press.
• Siddhartha Bhattacharyya, Vaclav Snasel, Aboul Ella Hassanien, Satadal Saha, & B. K. Tripathy. (2020). Deep Learning : Research and Applications. De Gruyter.

Authors

• SHIPILOV FOMA ALEKSANDROVICH