Convex optimization nyu

Nonsmooth optimization refers to minimization of functions that are not necessarily convex, usually locally Lipschitz, and typically not differentiable at their

Are all cost functions convex?

Many of the cost functions that we consider in data analysis involve norms

Conveniently, all norms are convex

Lemma 1 5 (Norms are convex)

Any valid norm jj jj is a convex function

Proof

By the triangle inequality inequality and homogeneity of the norm, for any ~ x; ~ y 2 Rn and any

What is a global minimum for a convex function?

An immediate corollary is that for a convex function, any point at which the gradient is zero is a global minimum

If the function is strictly convex, the minimum is unique

This is very useful for minimizing such functions, once we nd a point where the gradient is zero we are done! Figure 5: An example of the rst-order condition for convexity

Categories