Elements Of Deep Learning Theory

While the field of Deep Learning has been advancing at frightening speeds during the last ten years, we do not have a plausible theoretical explanation of its success yet. One can safely say that Deep Learning works, but nobody really understands why. Nevertheless, starting around five years before now, a decent number of theoretical papers specifically concerning neural nets started to emerge at top machine learning venues. This makes us think that the core of Deep Learning theory has alre...