Master
2019/2020
Introduction to Machine Learning
Category 'Best Course for Career Development'
Category 'Best Course for Broadening Horizons and Diversity of Knowledge and Skills'
Category 'Best Course for New Knowledge and Skills'
Type:
Elective course (Internet of Things and Cyber-physical Systems)
Area of studies:
Infocommunication Technologies and Systems
Delivered by:
School of Electronic Engineering
When:
1 year, 3 module
Mode of studies:
distance learning
Instructors:
Ilya Ivanov
Master’s programme:
Internet of Things and Cyber-physical Systems
Language:
English
ECTS credits:
4
Contact hours:
2
Course Syllabus
Abstract
The course starts with a recap of linear models and discussion of stochastic optimization methods that are crucial for training deep neural networks. Learners will study all popular building blocks of neural networks including fully connected layers, convolutional and recurrent layers. Learners will use these building blocks to define complex modern architectures in Ten-sorFlow and Keras frameworks. In the course project learner will implement deep neural network for the task of image captioning which solves the problem of giving a text description for an input image. https://www.coursera.org/specializations/aml
Learning Objectives
- The goal of this course is to give learners basic understanding of modern neural networks and their applications in computer vision and natural language understanding.
Expected Learning Outcomes
- • Welcome to AML specialization! • Course intro • Linear regression • Linear classification • Gradient descent • Overfitting problem and model validation • Model regularization • Stochastic gradient descent • Gradient descent extensions
- • Multilayer perceptron (MLP) • Chain rule • Backpropagation • Efficient MLP implementation • Other matrix derivatives • What is TensorFlow • Our first model in TensorFlow • What Deep Learning is and is not • Deep learning as a language • Optional reading on matrix derivatives • TensorFlow reading • Keras reading
- • Motivation for convolutional layers • Our first CNN architecture • Training tips and tricks for deep CNNs • Overview of modern CNN architectures • Learning new tasks with pre-trained CNNs • A glimpse of other Computer Vision tasks
- • Unsupervised learning: what it is and why bother • Autoencoders 10 • Autoencoder applications • Autoencoder applications: image generation, data visualization & more • Natural language processing primer • Word embeddings • Generative models 10 • Generative Adversarial Networks • Applications of adversarial approach
- • Motivation for recurrent layers • Simple RNN and Backpropagation • The training of RNNs is not that easy • Dealing with vanishing and exploding gradients • Modern RNNs: LSTM and GRU • Practical use cases for RNNs
Course Contents
- Module 1: Introduction to optimizationWelcome to the "Introduction to Deep Learning" course! In the first week you'll learn about linear models and stochatic optimization methods. Linear models are basic building blocks for many deep architectures, and stochastic optimization is used to learn every model that we'll discuss in our course
- Module 2: Introduction to neural networksThis module is an introduction to the concept of a deep neural network. You'll begin with the linear model and finish with writing your very first deep network.
- Module 3: Deep Learning for imagesIn this week you will learn about building blocks of deep learning for image input. You will learn how to build Convolutional Neural Network (CNN) architectures with these blocks and how to quickly solve a new task using so-called pre-trained models.
- Module 4: Unsupervised representation learningThis week we're gonna dive into unsupervised parts of deep learning. You'll learn how to generate, morph and search images with deep learning.
- Module 5: Deep learning for sequencesIn this week you will learn how to use deep learning for sequences such as texts, video, audio, etc. You will learn about several Recurrent Neural Network (RNN) architectures and how to apply them for different tasks with sequential input/output.
Assessment Elements
- Онлайн курс
- ЭкзаменЭкзамен проводится в устной форме (опрос по материалам курса). Экзамен проводится на платформе Jitsi (http://meet.miem.hse.ru/). К экзамену необходимо подключиться согласно расписанию ответов, высланному преподавателем на корпоративные почты студентов накануне экзамена. Компьютер студента должен удовлетворять требованиям: наличие рабочей камеры и микрофона, поддержка Jitsi. Для участия в экзамене студент обязан: поставить на аватар свою фотографию, явиться на экзамен согласно точному расписанию, при ответе включить камеру и микрофон. Во время экзамена студентам запрещено: выключать камеру, пользоваться конспектами и подсказками. Кратковременным нарушением связи во время экзамена считается нарушение связи менее минуты. Долговременным нарушением связи во время экзамена считается нарушение минута и более. При долговременном нарушении связи студент не может продолжить участие в экзамене. Процедура пересдачи подразумевает использование усложненных заданий. В ходе освоения дисциплины формируются следующие компетенции: УК-1, УК-6, УК-7, УК-8, ОПК-3, ПК-21
Bibliography
Recommended Core Bibliography
- Deep learning, Goodfellow, I., 2016
- Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The Elements of Statistical Learning : Data Mining, Inference, and Prediction (Vol. Second edition, corrected 7th printing). New York: Springer. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=277008
- James, G. et al. An introduction to statistical learning. – Springer, 2013. – 426 pp.
Recommended Additional Bibliography
- Huang, K., Hussain, A., Wang, Q.-F., & Zhang, R. (2019). Deep Learning: Fundamentals, Theory and Applications. Cham, Switzerland: Springer. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=2029631
- Kim, P. (2017). MATLAB Deep Learning : With Machine Learning, Neural Networks and Artificial Intelligence. [New York, NY]: Apress. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1535764
- Глубокое обучение на R, Шолле, Ф., 2018
- Глубокое обучение, Гудфеллоу, Я., 2018