2012-2016 年被引用次數最多的深度學習論文

近些年來在深度學習熱潮的推動下，人工智能領域的研究猶如機器之心的吉祥物土撥鼠在春天里一樣不斷涌現，到今天，一個人要閱讀了解這一領域的所有研究已經不再具有任何實踐的可能性。擇其善者而讀之已經成為了人工智能研究者的一項重要技能，而其中非常值得關注的一項指標就是論文的引用次數，尤其是近期的引用次數。

滑鐵盧大學博士、GitHub 用戶 Terry Taewoong Um 就希望能在這方面做出貢獻，他在 GitHub 上創建了一個項目，羅列了自 2012 年以來被引用最多的深度學習論文。

項目地址：https://github.com/terryum/awesome-deep-learning-papers 

這是一個持續更新的項目。機器之心曾在 2016 年 6 月編譯發表過這個項目之前的一個版本《 學界 | 2010-2016 年被引用次數最多的深度學習論文（附論文下載） 》。近日，這個項目再次進行了更新，下面我們就來看看被引用最多的論文有哪些。

背景及相關資源

在這份榜單前后，也有一些其它很棒的深度學習榜單，比如：

• 深度視覺：https://github.com/kjw0612/awesome-deep-vision

• 循環神經網絡：https://github.com/kjw0612/awesome-rnn

• 深度學習閱讀路線圖：https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap

但要看完這些榜單中提及的內容就已經很困難了，更不要說還有更多不在這些列表中的內容。所以我在這里推出了深度學習論文百強列表，希望能對想要整體了解深度學習研究的人提供幫助。

評選說明

1. 這份深度學習論文百強列表的論文來自 2012-2016 年之間發表的論文。

2. 如果一篇論文被加入到了這個列表，那么就必然會有一篇論文被移出這個列表（因此，移出論文和加入論文一樣都是對這份列表的貢獻。

3. 重要但是卻無法被包含進這份列表的論文會收納到 More than Top 100 列表。

4.New Papers 和 Old Papers 分別包含了最近 6 個月和 2012 年之前發表的論文。

評選標準

• 小于 6 個月：New Papers，按討論加入

• 2016 年：至少 60 次引用

• 2015 年：至少 200 次引用

• 2014 年：至少 400 次引用

• 2013 年：至少 600 次引用

• 2012 年：至少 800 次引用

• 2012 年之前：Old Papers，按討論加入

請注意我們更偏愛開創性的可以應用于多種研究的深度學習論文，而非應用論文。基于這樣的原因，一些滿足評選標準的論文也被排除了。具體還要依賴該論文的影響、這一領域其它研究的稀缺性等等。

內容目錄

• 理解/泛化/遷移

• 優化/訓練技術

• 無監督/生成模型

• 卷積網絡模型

• 圖像分割/目標檢測

• 圖像/視頻等

• 循環神經網絡模型

• 自然語言處理

• 語音/其它領域

• 強化學習

• 2016 年其它論文

其它看點

• 新論文（New Papers）

• 舊論文（Old Papers）

• HW/SW/數據集：技術報告

• 書籍/調查/概述

理解/泛化/遷移

• Distilling the knowledge in a neural network (2015), G. Hinton et al. [pdf]

• Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2015), A. Nguyen et al. [pdf]

• How transferable are features in deep neural networks? (2014), J. Yosinski et al. [pdf]

• CNN features off-the-Shelf: An astounding baseline for recognition (2014), A. Razavian et al. [pdf]

• Learning and transferring mid-Level image representations using convolutional neural networks (2014), M. Oquab et al. [pdf]

• Visualizing and understanding convolutional networks (2014), M. Zeiler and R. Fergus [pdf]

• Decaf: A deep convolutional activation feature for generic visual recognition (2014), J. Donahue et al. [pdf]

重要研究者：Geoffrey Hinton, Yoshua Bengio, Jason Yosinski

優化/訓練技術

• Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), S. Loffe and C. Szegedy [pdf]

• Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (2015), K. He et al. [pdf]

• Dropout: A simple way to prevent neural networks from overfitting (2014), N. Srivastava et al. [pdf]

• Adam: A method for stochastic optimization (2014), D. Kingma and J. Ba [pdf]

• Improving neural networks by preventing co-adaptation of feature detectors (2012), G. Hinton et al. [pdf]

• Random search for hyper-parameter optimization (2012) J. Bergstra and Y. Bengio [pdf]

重要研究者：Geoffrey Hinton, Yoshua Bengio, Christian Szegedy, Sergey Ioffe, Kaming He, Diederik P. Kingma

無監督/生成模型

• Pixel recurrent neural networks (2016), A. Oord et al. [pdf]

• Improved techniques for training GANs (2016), T. Sallmans et al. [pdf]

• Unsupervised representation learning with deep convolutional generative adversarial networks (2015), A. Radford et al. [pdf]

• DRAW: A recurrent neural network for image generation (2015), K. Gregor et al. [pdf]

• Generative adversarial nets (2014), I. Goodfellow et al. [pdf]

• Auto-encoding variational Bayes (2013), D. Kingma and M. Welling [pdf]

• Building high-level features using large scale unsupervised learning (2013), Q. Le et al. [pdf]

重要研究者：Yoshua Bengio, Ian Goodfellow, Alex Graves

卷積網絡模型

• Rethinking the inception architecture for computer vision (2016), C. Szegedy et al.

• Inception-v4, inception-resnet and the impact of residual connections on learning (2016), C. Szegedy et al. 

• Identity Mappings in Deep Residual Networks (2016), K. He et al. 

• Deep residual learning for image recognition (2016), K. He et al. 

• Going deeper with convolutions (2015), C. Szegedy et al. 

• Very deep convolutional networks for large-scale image recognition (2014), K. Simonyan and A. Zisserman 

• Spatial pyramid pooling in deep convolutional networks for visual recognition (2014), K. He et al. 

• Return of the devil in the details: delving deep into convolutional nets (2014), K. Chatfield et al. 

• OverFeat: Integrated recognition, localization and detection using convolutional networks (2013), P. Sermanet et al.

• Maxout networks (2013), I. Goodfellow et al. 

• Network in network (2013), M. Lin et al. 

• ImageNet classification with deep convolutional neural networks (2012), A. Krizhevsky et al. 

重要研究者：Christian Szegedy, Kaming He, Shaoqing Ren, Jian Sun, Geoffrey Hinton, Yoshua Bengio, Yann LeCun

圖像分割/目標檢測

• You only look once: Unified, real-time object detection (2016), J. Redmon et al. 

• Region-based convolutional networks for accurate object detection and segmentation (2016), R. Girshick et al. 

• Fully convolutional networks for semantic segmentation (2015), J. Long et al. 

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015), S. Ren et al.

• Fast R-CNN (2015), R. Girshick

• Rich feature hierarchies for accurate object detection and semantic segmentation (2014), R. Girshick et al.

• Semantic image segmentation with deep convolutional nets and fully connected CRFs, L. Chen et al. 

• Learning hierarchical features for scene labeling (2013), C. Farabet et al. 

重要研究者：Ross Girshick, Jeff Donahue, Trevor Darrell

圖像/視頻/ETC

• Image Super-Resolution Using Deep Convolutional Networks (2016), C. Dong et al. 

• A neural algorithm of artistic style (2015), L. Gatys et al. 

• Deep visual-semantic alignments for generating image descriptions (2015), A. Karpathy and L. Fei-Fei

• Show, attend and tell: Neural image caption generation with visual attention (2015), K. Xu et al. 

• Show and tell: A neural image caption generator (2015), O. Vinyals et al. 

• Long-term recurrent convolutional networks for visual recognition and description (2015), J. Donahue et al. 

• VQA: Visual question answering (2015), S. Antol et al. 

• DeepFace: Closing the gap to human-level performance in face verification (2014), Y. Taigman et al. 

• Large-scale video classification with convolutional neural networks (2014), A. Karpathy et al. 

• DeepPose: Human pose estimation via deep neural networks (2014), A. Toshev and C. Szegedy 

• Two-stream convolutional networks for action recognition in videos (2014), K. Simonyan et al. 

• 3D convolutional neural networks for human action recognition (2013), S. Ji et al. 

重要研究者：Oriol Vinyals, Andrej Karpathy

循環神經網絡模型

• Conditional random fields as recurrent neural networks (2015), S. Zheng and S. Jayasumana. 

• Memory networks (2014), J. Weston et al. 

• Neural turing machines (2014), A. Graves et al. 

• Generating sequences with recurrent neural networks (2013), A. Graves.  [Key researchers] Alex Graves

自然語言處理

• A character-level decoder without explicit segmentation for neural machine translation (2016), J. Chung et al. 

• Exploring the limits of language modeling (2016), R. Jozefowicz et al. 

• Teaching machines to read and comprehend (2015), K. Hermann et al. 

• Effective approaches to attention-based neural machine translation (2015), M. Luong et al. 

• Neural machine translation by jointly learning to align and translate (2014), D. Bahdanau et al. 

• Sequence to sequence learning with neural networks (2014), I. Sutskever et al. 

• Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014), K. Cho et al. 

• A convolutional neural network for modelling sentences (2014), N. Kalchbrenner et al. 

• Convolutional neural networks for sentence classification (2014), Y. Kim 

• Glove: Global vectors for word representation (2014), J. Pennington et al. 

• Distributed representations of sentences and documents (2014), Q. Le and T. Mikolov 

• Distributed representations of words and phrases and their compositionality (2013), T. Mikolov et al. 

• Efficient estimation of word representations in vector space (2013), T. Mikolov et al. 

• Recursive deep models for semantic compositionality over a sentiment treebank (2013), R. Socher et al. 

重要研究者：Kyunghyun Cho, Oriol Vinyals, Richard Socher, Tomas Mikolov, Christopher D. Manning, Yoshua Bengio

語音/其它領域

• End-to-end attention-based large vocabulary speech recognition (2016), D. Bahdanau et al. 

• Deep speech 2: End-to-end speech recognition in English and Mandarin (2015), D. Amodei et al.

• Speech recognition with deep recurrent neural networks (2013), A. Graves 

• Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups (2012), G. Hinton et al. 

• Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition (2012) G. Dahl et al. 

• Acoustic modeling using deep belief networks (2012), A. Mohamed et al. 

重要研究者：Alex Graves, Geoffrey Hinton, Dong Yu

強化學習

• End-to-end training of deep visuomotor policies (2016), S. Levine et al.

• Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection (2016), S. Levine et al. 

• Asynchronous methods for deep reinforcement learning (2016), V. Mnih et al. 

• Deep Reinforcement Learning with Double Q-Learning (2016), H. Hasselt et al. 

• Mastering the game of Go with deep neural networks and tree search (2016), D. Silver et al. 

• Continuous control with deep reinforcement learning (2015), T. Lillicrap et al.

• Human-level control through deep reinforcement learning (2015), V. Mnih et al. 

• Deep learning for detecting robotic grasps (2015), I. Lenz et al. 

• Playing atari with deep reinforcement learning (2013), V. Mnih et al. 

重要研究者：Sergey Levine, Volodymyr Mnih, David Silver

2016 年其它論文

• Layer Normalization (2016), J. Ba et al. 

• Learning to learn by gradient descent by gradient descent (2016), M. Andrychowicz et al.

• Domain-adversarial training of neural networks (2016), Y. Ganin et al. 

• WaveNet: A Generative Model for Raw Audio (2016), A. Oord et al.

• Colorful image colorization (2016), R. Zhang et al. 

• Generative visual manipulation on the natural image manifold (2016), J. Zhu et al. 

• Texture networks: Feed-forward synthesis of textures and stylized images (2016), D Ulyanov et al. 

• SSD: Single shot multibox detector (2016), W. Liu et al.

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size (2016), F. Iandola et al. 

• Eie: Efficient inference engine on compressed deep neural network (2016), S. Han et al. 

• Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1 (2016), M. Courbariaux et al. 

• Dynamic memory networks for visual and textual question answering (2016), C. Xiong et al.

• Stacked attention networks for image question answering (2016), Z. Yang et al. 

• Hybrid computing using a neural network with dynamic external memory (2016), A. Graves et al.

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation (2016), Y. Wu et al.

新論文（New Papers）

最近六個月內值得一讀的論文：

• Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models, S. Ioffe.

• Understanding deep learning requires rethinking generalization, C. Zhang et al

舊論文（Old Papers）

2012 年之前發表的經典論文：

• An analysis of single-layer networks in unsupervised feature learning (2011), A. Coates et al. 

• Deep sparse rectifier neural networks (2011), X. Glorot et al

• Natural language processing (almost) from scratch (2011), R. Collobert et al

• Recurrent neural network based language model (2010), T. Mikolov et al

• Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion (2010), P. Vincent et al

• Learning mid-level features for recognition (2010), Y. Boureau 

• A practical guide to training restricted boltzmann machines (2010), G. Hinton   Understanding the difficulty of training deep feedforward neural networks (2010), X. Glorot and Y. Bengio 

• Why does unsupervised pre-training help deep learning (2010), D. Erhan et al

• Recurrent neural network based language model (2010), T. Mikolov et al. 

• Learning deep architectures for AI (2009), Y. Bengio. 

• Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations (2009), H. Lee et al. 

• Greedy layer-wise training of deep networks (2007), Y. Bengio et al

• Reducing the dimensionality of data with neural networks, G. Hinton and R. Salakhutdinov

• A fast learning algorithm for deep belief nets (2006), G. Hinton et al. 

• Gradient-based learning applied to document recognition (1998), Y. LeCun et al

• Long short-term memory (1997), S. Hochreiter and J. Schmidhuber. 

HW/SW/數據集

• OpenAI gym (2016), G. Brockman et al

• TensorFlow: Large-scale machine learning on heterogeneous distributed systems  (2016), M. Abadi et al. 

• Theano: A Python framework for fast computation of mathematical expressions,      R. Al-Rfou et al.

• MatConvNet: Convolutional neural networks for matlab (2015), A. Vedaldi and K.  Lenc 

• Imagenet large scale visual recognition challenge (2015), O. Russakovsky et al

• Caffe: Convolutional architecture for fast feature embedding (2014), Y. Jia et al

書籍/調查/概述

• Deep learning (Book, 2016), Goodfellow et al. 

• LSTM: A search space odyssey (2016), K. Greff et al. 

• Deep learning (2015), Y. LeCun, Y. Bengio and G. Hinton 

• Deep learning in neural networks: An overview (2015), J. Schmidhuber      Representation learning: A review and new perspectives (2013), Y. Bengio et al.

 

來自：http://www.jiqizhixin.com/article/2301