Optimization Methods

2026/2027

Academic Year

ENG

Instruction in English

5

ECTS credits

Delivered at:

Big Data and Information Retrieval School

Course type:

Compulsory course

When:

3 year, 1, 2 module

Instructors

Баулин Евгений Алексеевич

Ignatov, Andrei

Касимов Илья Максимович

Matveeva, Anna

Course Syllabus

Abstract

Optimization holds an important place in both practical and theoretical worlds, as understanding the timing and magnitude of actions to be carried out helps achieve a goal in the best possible way. This course emphasizes data-driven modeling, theory and numerical algorithms for optimization with real variables. The course gives a comprehensive foundation for theory, methods and algorithms of mathematical optimization. The prerequisites are linear algebra and calculus.

Learning Objectives

• Students will study main concepts of optimization theory and develop a methodology for theoretical investigation of optimization problems.
• Students will obtain an understanding of creation, effectiveness and application optimization methods and algorithms on practice.
• The course will give students the possibility of solving standard and nonstandard mathematical problems connected to finding optimal solutions.

Expected Learning Outcomes

• Students should be able to acces the rate of convergence of the first and second order optimization methods.
• Students should be able to classify optimization problems according to their mathematical properties.
• Students should be able to describe the numerical complexity of the optimization algorithms studied during the course.
• Students should be able to implement different optimization codes in a computer environment.
• Students should be able to perform a theoretical investigation of a given optimization problem in order to access its complexity.
• Students should be able to solve simple optimization problems without computer.
• Students should be able to write down first and second-order optimality condition.
• Students should be able to write down first and second-order optimality conditions.

Course Contents

• One-dimensional optimization: unimodal functions, convex and quasiconvex functions, zero and first-order methods, local and global minima.
• Existence of solutions: continuous and lower semicontinuous functions, coercive functions, Weierstrass theorem, unique and nonunique solutions.
• Quadratic unconstrained optimization: algebraic solution, complete optimization analysis, steepest descent and conjugate gradient methods.
• Theory of optimality conditions: Fermat principle, the Hessian matrix, positive and negative semidefinite matrices, the Lagrange function and Lagrange multipliers, the Karush-Kuhn-Tucker conditions, regularity, complementarity constraints, stationary points.
• First-order optimization methods: the steepest descent method, conjugate directions, gradient-based methods.
• Second order optimization methods: Newton's method and modifications, trust-region methods.
• Convex optimization: optimality conditions, duality, subgradients and subdifferential, cutting planes and bundle methods, the complexity of convex optimization.
• Computational optimization methods: first-order, second-order, Quasi-Newton methods
• Linear Programming, Knapsack problem
• Combinatorial optimization, Transport problems
• Multi-criteria optimization

Assessment Elements

• exam
• colloquium
• homework

Interim Assessment

• 2026/2027 2nd module
  𝑂 total = 0.6 * 𝑂 sem + 0.4 * 𝑂 exam, where 𝑂 sem = 0.28 * 𝑂 colloquium + 0.18 * (𝑂 hw 1 + 𝑂 hw 2 + 𝑂 hw 3 + 𝑂 hw 4)

Bibliography

Recommended Core Bibliography

• A mathematical view of interior-point methods in convex optimization, Renegar, J., 2001
• Nonlinear programming, Mangasarian, O. L., 1994

Recommended Additional Bibliography

• Combinatorial Optimization, Theory and Algorithms, 5th edition, XX, 659 p., Korte, B., Vygen, J., 2012
• Lectures on modern convex optimization : analysis, algorithms, and engineering applications, Ben-Tal, A., 2001

Authors

• IGNATOV ANDREI DMITRIEVICH