Ian Goodfellow曾经给在quora回答过，以下是原文：

There are various ways to test for convexity.

One is to just plot a cross-section of the function and look at it. If it has a non-convex shape, you don’t need to write a proof; you have disproven convexity by counter-example.

If you want to do this with algebra, one way is just to take the second derivatives of a function. If the second derivative of a function in 1-D space is ever negative, the function isn’t convex.

For neural nets, you have millions of parameters, so you need a test that works in high-dimensional space. In high-dimensional space, it turns out we can take the second derivative along one specific direction in space. For a unit vector d giving the direction and a Hessian matrix H of second derivatives, this is given by

For most neural nets and most loss functions, it’s very easy to find a point in parameter space and a direction where

is negative.