最全面的深度學習自學資源匯總
深度學習作為機器學習的一個分支，是近年來最熱門同時也是發展最快的人工智能技術之一，相關學習資源包括免費公開教程和工具都極大豐富，同時這也為學習 深度學習技術 的IT人才帶來選擇上的困擾，Yerevann整理的這個深度學習完全指南 ，匯集了目前網絡上最優秀的深度學習自學資源，而且會不定期更新，非常值得收藏關注，以下是IT經理網編譯整理的指南內容：
自學基本要求（數學知識、編程知識）
數學知識：學員需要具備普通大學數學知識，例如 《Deep Learning》 一書中若干章節提到的數學概念：
編程知識：你需要懂得編程才能開發和測試深度學習模型，我們建議在機器學習領域首選Python。同時也要用到面向科學計算的NumPy/SciPy代碼庫。資源鏈接如下（本文出現的星標代表難度等級）：
Justin Johnson’s Python / NumPy / SciPy / Matplotlib tutorial for Stanford’s CS231n ★
Scipy lecture notes – 涵蓋了常用的各種庫，介紹也比較詳細，還涉及一些深入的技術話題 ★★
四大入門教程
如果你具備以上自學基本要求技能，我們建議從以下四大入門在線教程中任選一項或多項組合學習（星標為難度等級）：
Hugo Larochelle’s video course 這是油Tube上很火的一個深度學習視頻教程，錄制于2013年，但今天看內容并不過時，很詳細地闡釋了神經網絡背后的數學理論。 幻燈片和相關資料傳送門 . ★★
Stanford’s CS231n (應用于視覺識別的卷積神經網絡) 由已經投奔Google的李飛飛教授和 Andrej Karpathy、Justin Johnson共同執教的課程，重點介紹了圖像處理，同時也涵蓋了深度學習領域的大多數重要概念。 視頻 鏈接(2016) 、 講義傳送門 ★★
Michael Nielsen的在線著作： Neural networks and deep learning 是目前學習神經網絡 最容易的教材 ，雖然該書并未涵蓋所有重要議題，但是包含大量簡明易懂的闡釋，同時還為一些基礎概念提供了實現代碼。★
Ian Goodfellow、Yoshua Bengio and Aaron Courville共同編著的 Deep learning 是目前深度學習領域 最全面的教程 資源，比其他課程涵蓋的范圍都要廣。 ★★★
機器學習基礎
機器學習是通過數據教計算機做事的科學，同時也是一門藝術，機器學習是計算機科學和數學交匯的一個相對成熟的領域，深度學習只是其中新興的一小部分，因此，了解機器學習的概念和工具對我們學好深度學習非常重要。以下是機器學習的一些重要學習資源（以下課程介紹部分內容不再翻譯）：
Visual introduction to machine learning – decision trees ★
Andrew Ng’s course on machine learning , the most popular course on Coursera ★★
Larochelle’s course doesn’t have separate introductory lectures for general machine learning, but all required concepts are defined and explained whenever needed.
1. Training and testing the models (kNN) ★★
2. Linear classification (SVM) ★★
3. Optimization (stochastic gradient descent) ★★
5. Machine learning basics ★★★
Principal Component Analysis explained visually ★
How to Use t-SNE Effectively ★★
機器學習的編程學習資料：大多數流行機器學習算法都部署在Scikit-learn 這個Python庫中，從頭部署算法能夠幫我們更好地了解機器學習的工作原理，以下是相關學習資源：
Practical Machine Learning Tutorial with Python covers linear regression, k-nearest-neighbors and support vector machines. First it shows how to use them from scikit-learn, then implements the algorithms from scratch. ★
Andrew Ng’s course on Coursera has many assignments in Octave language. The same algorithms can be implemented in Python. ★★
神經網絡基礎
神經網絡是強大的機器學習算法，同時也是深度學習的基礎：
A Visual and Interactive Guide to the Basics of Neural Networks – shows how simple neural networks can do linear regression ★
1. Feedforward neural network ★★
2. Training neural networks (up to 2.7) ★★
5. Architecture of neural networks ★★
1. Using neural nets to recognize handwritten digits ★
2. How the backpropagation algorithm works ★
4. A visual proof that neural nets can compute any function ★
6. Deep feedforward networks ★★★
Yes you should understand backprop explains why it is important to implement backpropagation once from scratch ★★
Calculus on computational graphs: backpropagation ★★
神經網絡實操教程
Implementing softmax classifier and a simple neural network in pure Python/NumPy – Jupyter notebook available ★
Andrej Karpathy implements backpropagation in Javascript in his Hacker’s guide to Neural Networks . ★
Implementing a neural network from scratch in Python ★
改進神經網絡學習
神經網絡的訓練可不容易，很多時候機器壓根不會學習（underfitting），有時候又“死學”，照本宣科你輸入的知識，無法總結歸納出新的數據（overfitting）,解決上述問題的方法有很多，如下是
推薦教程：
2.8-2.11. Regularization, parameter initialization etc. ★★
7.5. Dropout ★★
6 (first half). Setting up the data and loss ★★
3. Improving the way neural networks learn ★
5. Why are deep neural networks hard to train? ★
7. Regularization for deep learning ★★★
8. Optimization for training deep models ★★★
ConvNetJS Trainer demo on MNIST – visualizes the performance of different optimization algorithms ★
An overview of gradient descent optimization algorithms ★★★
Neural Networks, Manifolds, and Topology ★★★
常用的主流框架
目前很多 深度學習算法 都對最新的計算機硬件進行了優化，大多數框架也提供Python接口（Torch除外，需要Lua）。當你了解基本的深度學習算法的部署后，是時候選擇一個框架開工了（這部分還可CTOCIO文章： 2016年人氣最高的六款開源深度學習工具 ）：
Theano provides low-level primitives for constructing all kinds of neural networks. It is maintained by a machine learning group at University of Montreal . See also: Speeding up your neural network with Theano and the GPU – Jupyter notebook available ★
TensorFlow is another low-level framework. Its architecture is similar to Theano. It is maintained by the Google Brain team.
Torch is a popular framework that uses Lua language. The main disadvantage is that Lua’s community is not as large as Python’s. Torch is mostly maintained by 非死book and 推ter.
There are also higher-level frameworks that run on top of these:
Lasagne is a higher level framework built on top of Theano. It provides simple functions to create large networks with few lines of code.
Keras is a higher level framework that works on top of either Theano or TensorFlow.
如果你有框架選擇困難癥，可以參考斯坦福課程 Lecture 12 of Stanford’s CS231n . ★★
卷積神經網絡
卷積神經網絡Convolutional networks (CNNs)，是一種特定的神經網絡，通過一些聰明的方法大大提高了學習速度和質量。卷積神經網絡掀起了計算機視覺的革命，并廣泛應用于語音識別和文本歸類等領域，以下是
推薦教程：
9. Computer vision (up to 9.9) ★★
6 (second half). Intro to ConvNets ★★
7. Convolutional neural networks ★★
8. Localization and detection ★★
9. Visualization, Deep dream, Neural style, Adversarial examples ★★
13. Image segmentation (up to 38:00) includes upconvolutions ★★
Image Kernels explained visually – shows how convolutional filters (also known as image kernels) transform the image ★
ConvNetJS MNIST demo – live visualization of a convolutional network right in the browser ★
Conv Nets: A Modular Perspective ★★
Understanding Convolutions ★★★
Understanding Convolutional neural networks for NLP ★★
卷積神經網絡框架部署和應用
所有重要的框架都支持卷積神經網絡的部署，通常使用高級函數庫編寫的代碼的可讀性要更好一些。
Theano: Convolutional Neural Networks (LeNet) ★★
Using Lasagne for training Deep Neural Networks ★
Detecting diabetic retinopathy in eye images – a blog post by one of the best performers of Diabetic retinopathy detection contest in Kaggle. Includes a good example of data augmentation. ★★
Face recognition for right whales using deep learning – the authors used different ConvNets for localization and classification. Code and models are available . ★★
Tensorflow: Convolutional neural networks for image classification on CIFAR-10 dataset ★★
Implementing a CNN for text classification in Tensorflow ★★
DeepDream implementation in TensorFlow ★★★
92.45% on CIFAR-10 in Torch – implements famous VGGNet network with batch normalization layers in Torch ★
Training and investigating Residual Nets – Residual networks perform very well on image classification tasks. Two researchers from 非死book and CornellTech implemented these networks in Torch ★★★
ConvNets in practice – lots of practical tips on using convolutional networks including data augmentation, transfer learning, fast implementations of convolution operation ★★
遞歸神經網絡
遞歸神經網絡Recurrent entworks（RNNs）被設計用來處理序列數據（例如文本、股票、基因組、傳感器等）相關問題，通常應用于語句分類（例如情感分析）和語音識別，也適用于文本生成甚至圖像生成。
教程如下：
The Unreasonable Effectiveness of Recurrent Neural Networks – describes how RNNs can generate text, math papers and C++ code ★
Hugo Larochelle’s course doesn’t cover recurrent neural networks (although it covers many topics that RNNs are used for). We suggest watching Recurrent Neural Nets and LSTMs by Nando de Freitas to fill the gap ★★
10. Recurrent Neural Networks, Image Captioning, LSTM ★★
13. Soft attention (starting at 38:00) ★★
Michael Nielsen’s book stops at convolutional networks. In the Other approaches to deep neural nets section there is just a brief review of simple recurrent networks and LSTMs. ★
10. Sequence Modeling: Recurrent and Recursive Nets ★★★
Recurrent neural networks from Stanford’s CS224d (2016) by Richard Socher ★★
Understanding LSTM Networks ★★
遞歸神經網絡的框架部署與應用
Theano: Recurrent Neural Networks with Word Embeddings ★★★
Theano: LSTM Networks for Sentiment Analysis ★★★
Implementing a RNN with Python, Numpy and Theano ★★
Lasagne implementation of Karpathy’s char-rnn ★
Combining CNN and RNN for spoken language identification in Lasagne ★
Automatic transliteration with LSTM using Lasagne ★
Tensorflow: Recurrent Neural Networks for language modeling ★★
Recurrent Neural Networks in Tensorflow ★★
Understanding and Implementing Deepmind’s DRAW Model ★★★
LSTM implementation explained ★★
Torch implementation of Karpathy’s char-rnn ★★★
Autoencoders
Autoencoder是為非監督式學習設計的神經網絡，例如當數據沒有標記的情況。Autoencoder可以用來進行數據維度消減，以及為其他神經網絡進行預訓練，以及數據生成等。以下課程資源中，我們還收錄了Autoencoder與概率圖模型整合的一個autoencoders的變種，其背后的數學機理在下一章“概率圖模型”中會介紹。
推薦教程：
14. Videos and unsupervised learning (from 32:29) – this video also touches an exciting topic of generative adversarial networks. ★★
14. Autoencoders ★★★
ConvNetJS Denoising Autoencoder demo ★
Karol Gregor on Variational Autoencoders and Image Generation ★★
Autoencoder的部署
大多數autoencoders都非常容易部署，但我們還是建議您從簡單的開始嘗試。課程資源如下：
Theano: Denoising autoencoders ★★
Diving Into TensorFlow With Stacked Autoencoders ★★
Variational Autoencoder in TensorFlow ★★
Training Autoencoders on ImageNet Using Torch 7 ★★
Building autoencoders in Keras ★
概率圖模型
概率圖模型（Probabilistic Graphical model，PGM）是統計學和機器學習交叉分支領域，關于概率圖模型的書籍和課程非常多，以下我們收錄的資源重點關注概率圖模型在深度學習場景中的應用。其中Hugo Larochelles的課程介紹了一些非常著名的模型，而Deep Learning一書有整整四個章節專門介紹，并在最后一章介紹了十幾個模型。本領域的學習需要讀者掌握大量數學知識：
3. Conditional Random Fields ★★★
4. Training CRFs ★★★
5. Restricted Boltzman machine ★★★
7.7-7.9. Deep Belief Networks ★★★
13. Linear Factor Models – first steps towards probabilistic models ★★★
16. Structured Probabilistic Models for Deep Learning ★★★
18. Confronting the Partition Function ★★★
20. Deep Generative Models – includes Boltzmann machines (RBM, DBN, …), variational autoencoders, generative adversarial networks, autoregressive models etc. ★★★
Generative models – a blog post on variational autoencoders, generative adversarial networks and their improvements by OpenAI. ★★★
The Neural Network Zoo attempts to organize lots of architectures using a single scheme. ★★
概率圖模型的部署
高級框架（Lasagne、Keras）不支持概率圖模型的部署，但是Theano、Tensorflow和Torch有很多可用的代碼。
Restricted Boltzmann Machines in Theano ★★★
Deep Belief Networks in Theano ★★★
Generating Large Images from Latent Vectors – uses a combination of variational autoencoders and generative adversarial networks. ★★★
Image Completion with Deep Learning in TensorFlow – another application of generative adversarial networks. ★★★
Generating Faces with Torch – Torch implementation of Generative Adversarial Networks ★★
精華論文、視頻與論壇匯總
Deep learning papers reading roadmap 深度學習重要論文的大清單。
Arxiv Sanity Preserver 為瀏覽 arXiv上的論文提供了一個漂亮的界面.
Videolectures.net 含有大量關于深度學習的高級議題視頻
/r/MachineLearning 一個非常活躍的Reddit分支. 幾乎所有重要的新論文這里都有討論。
來自：http://dataunion.org/29282.html