• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site
Master 2021/2022

Neural Networks and Deep Learning

Category 'Best Course for Career Development'
Category 'Best Course for Broadening Horizons and Diversity of Knowledge and Skills'
Category 'Best Course for New Knowledge and Skills'
Area of studies: Business Informatics
When: 2 year, 1 module
Mode of studies: distance learning
Open to: students of one campus
Instructors: Sergey Lisitsyn
Master’s programme: Big Data Systems
Language: English
ECTS credits: 6
Contact hours: 28

Course Syllabus

Abstract

Recently, deep learning has become a tool of choice for multiple problems in the fields of computer vision, robotics, biology, natural language processing, and many more. The technology is enabled by various milestones in computing and engineering, such as graphical processing units (GPU) and differentiable programming. Due to recent advances in open source software for deep learning, training a model has become a relatively simple exercise. However, the techniques used by researchers are fairly complicated. This course is focused on practical part of applying state-of-the-art techniques of deep learning on real-world problems. The course is based on popular and accessible MOOC called fast.ai and covers all the sections of the course with advice from machine learning practice.
Learning Objectives

Learning Objectives

  • Learn to train models that achieve state-of-the-art results in: computer vision, natural language processing, tabular data, and collaborative filtering
  • Learn approaches to turn models into web applications
  • Learn to implement deep learning training loop from the scratch
  • Acknowledge ethical implications of applying machine learning in practice
Expected Learning Outcomes

Expected Learning Outcomes

  • Able to use embedding for tabular data and recommenders
  • Able to use momentum and advanced optimizers for stochastic gradient descent
  • Able to use residual blocks with neural networks
  • Can construct a digit classifier using a deep learning model
  • Can construct a neural network from scratch
  • Can construct recurrent neural network from scratch
  • Can solve multi-class and multi-label problems with deep learning
  • Knows advanced neural networks such as U-Net and Siamese
  • Knows and uses state-of-the-art approaches to train neural networks
  • Knows the definitions of deep learning
  • Understands approaches to put machine learning system into production
  • Understands approaches to solve natural language processing problems
  • Understands data ethics and able to detect ethical problems
  • Understands the role of convolutions in image processing
Course Contents

Course Contents

  • Introduction
  • Machine learning in production
  • Data ethics
  • Training a digit classifier
  • Image classification
  • Other computer vision problems
  • Reaching state-of-the-art
  • Collaborative filtering
  • Tabular data
  • Natural language processing
  • Software libraries for data processing
  • A language model from scratch
  • Convolutional neural networks
  • Residual neural networks
  • Neural architectures
  • The training process of neural networks
  • A neural net from scratch
  • Class Activation Maps
  • Introduction to deep learning
  • Neural Networks Basics
  • Shallow neural networks
  • Deep Neural Networks
Assessment Elements

Assessment Elements

  • non-blocking Homework №1
    A student should provide a Jupyter notebook
  • non-blocking Homework №2
    A student should provide a Jupyter notebook
  • non-blocking Exam
    Exam format: the exam is taken in a written format, remotely (online) on MS Teams platform. A student is expected to complete the assignment and provide Jupyter (.ipynb) notebook to the instructor (the URL for Google Forms to submit is provided in MS Teams). To participate in the exam a student should have access to a compute that is either able to run Jupyter or work with Google Colab page. The file should be submitted one hour before the end of the exam. A student is allowed to use any kind of information to complete the assignment. A student should not consult or re-use solutions by other fellow students.
Interim Assessment

Interim Assessment

  • 2021/2022 1st module
    0.4 * Homework №1 + 0.4 * Homework №2 + 0.2 * Exam
Bibliography

Bibliography

Recommended Core Bibliography

  • Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The Elements of Statistical Learning : Data Mining, Inference, and Prediction (Vol. Second edition, corrected 7th printing). New York: Springer. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=277008
  • Ian Goodfellow, Yoshua Bengio, & Aaron Courville. (2016). Deep Learning. The MIT Press.
  • Jeremy Howard, & Sylvain Gugger. (2020). Deep Learning for Coders with Fastai and PyTorch. O’Reilly Media.

Recommended Additional Bibliography

  • M Narasimha Murty, & V Susheela Devi. (2015). Introduction To Pattern Recognition And Machine Learning. World Scientific.