To get an easy-to-read, but still quality overview of the progress and ideas behind deep learning, here are good blogs and sites made by researchers.
A very good course on Machine Learning: These video courses by Nando De Freitas are quite generic about Machine Learning (beyond deep learning) and are of very high quality. This link is just to the first video of the serie, you have to look for all the others there.
Great lectures by Stanford Winter 2017
The above lectures are great as well, and they are more specialized into deep learning for Natural Language Processing.
Then you have these lectures above by A. Karpathy that complete the Standford lectures from the image processing point of view.
Deep learning book
Three excellent free e-books by Kevin Murphy: https://probml.github.io/pml-book/
There are many tutorials about deep learning on the internet, with enlightening practical programming exercices. Here are just a few of them: