Convex optimization algorithms and complexity bubeck

  • What is convex optimization in simple terms?

    Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets)..

  • Why convex optimization is important for machine learning?

    Convex optimization has become an essential tool in machine learning because many real-world problems can be modeled as convex optimization problems.
    For example, in classification problems, the goal is to find the best hyperplane that separates the data points into different classes..

  • f(x,y,z)=2x2−y+z2→min Convex optimization problem if: (1) f(x)→min My idea is to calculate the Hessian matrix of the objective function and constraints and check if the matrix is positive (semi) definite, which would imply (strictly) convex function.
  • Stochastic constrained convex optimization: This is a special case where each ft(x) is i.i.d. generated from an unknown distribution.
    This problem has many applications in operations research and machine learning such as Neyman-Pearson classification and risk-mean portfolio.

How to minimize a smooth convex function f over a compact convex set?

We describe now an alternative algorithm to minimize a smooth con- vex function f over a compact convex set X

The conditional gradient descent, introduced in Frank and Wolfe , performs the following update for t ≥ 1, where (γs)s≥1 is a fixed sequence, xt+1 = (1 − γt)xt + γtyt

What is convex optimization?

Convex Optimization: Algorithms and Complexity This monograph presents the main complexity theorems in convex op- timization and their corresponding algorithms

Starting from the fun- damental theory of black-box optimization, the material progresses to- wards recent advances in structural optimization and stochastic op- timization

Who wrote optimal convergence rates for convex distributed optimization?

K Scaman, F Bach, S Bubeck, Y T Lee, L

Massoulié, Optimal Convergence Rates for Convex Distributed Optimization in Networks

In Journal of Machine Learning Research, 2019

S Bubeck and R

Eldan, Exploratory distributions for convex functions

In Mathematical Statistics and Learning, 2018

[ arxiv]

Categories

Convex optimization algorithms by dimitri p. bertsekas
Convex optimization and euclidean distance geometry
Convex optimization absolute value
Convex optimization and engineering applications
Convex optimization additional exercises solutions
Convex optimization by stephen boyd
Convex optimization boyd solutions
Convex optimization bubeck
Convex optimization berkeley
Convex optimization bertsekas
Convex optimization boyd course
Convex optimization boyd cite
Convex optimization boyd slides
Convex optimization basics
Convex optimization boyd bibtex
Convex optimization boyd citation
Convex optimization boyd reddit
Convex optimization book recommendation
Convex optimization cambridge
Convex optimization coursera