Show HN: The Hessian of tall-skinny networks is easy to invert
via news.ycombinator.com
Short excerpt below. Read at the original source.
It turns out the inverse of the Hessian of a deep net is easy to apply to a vector. Doing this naively takes cubically many operations in the number of layers (so impractical), but it’s possible to do this in time linear in the number of layers (so very practical)! This is possible because the […]