Deep Learning

Home * Learning * Neural Networks * Deep Learning

Deep Neural Network ¹

Deep Learning,
a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data - characterized as a buzzword, or a rebranding of neural networks. A deep neural network (DNN) is an ANN with multiple hidden layers of units between the input and output layers which can be discriminatively trained with the standard backpropagation algorithm. Two common issues if naively trained are overfitting and computation time. While deep learning techniques have yielded in another breakthrough in computer Go (after Monte-Carlo Tree Search), some trials in computer chess were promising as well, but until December 2017, less spectacular.

Go

Convolutional neural networks form a subclass of feedforward neural networks that have special weight constraints, individual neurons are tiled in such a way that they respond to overlapping regions. Convolutional NNs are suited for deep learning and are highly suitable for parallelization on GPUs ¹. In 2014, two teams independently investigated whether deep convolutional neural networks could be used to directly represent and learn a move evaluation function for the game of Go. Christopher Clark and Amos Storkey trained an 8-layer convolutional neural network by supervised learning from a database of human professional games, which without any search, defeated the traditional search program Gnu Go in 86% of the games ^{2 3 4 5}. In their paper Move Evaluation in Go Using Deep Convolutional Neural Networks ⁶, Chris J. Maddison, Aja Huang, Ilya Sutskever, and David Silver report they trained a large 12-layer convolutional neural network in a similar way, to beat Gnu Go in 97% of the games, and matched the performance of a state-of-the-art Monte-Carlo tree search that simulates a million positions per move ⁷.

In 2015, a team affiliated with Google DeepMind around David Silver and Aja Huang, supported by Google researchers John Nham and Ilya Sutskever, build a Go playing program dubbed AlphaGo ⁸, combining Monte-Carlo tree search with their 12-layer networks ⁹.

Chess

Giraffe & Zurichess

In 2015, Matthew Lai trained Giraffe's deep neural network by TD-Leaf ¹⁰. Zurichess by Alexandru Moșoi uses the TensorFlow library for automated tuning - in a two layers neural network, the second layer is responsible for a tapered eval to phase endgame and middlegame scores ¹¹.

DeepChess

In 2016, Omid E. David, Nathan S. Netanyahu, and Lior Wolf introduced DeepChess obtaining a grandmaster-level chess playing performance using a learning method incorporating two deep neural networks, which are trained using a combination of unsupervised pretraining and supervised training. The unsupervised training extracts high level features from a given chess position, and the supervised training learns to compare two chess positions to select the more favorable one. In order to use DeepChess inside a chess program, a novel version of alpha-beta is used that does not require bounds but positions αpos and βpos ¹².

AlphaZero

In December 2017, the Google DeepMind team with Matthew Lai involved published on their generalized AlphaZero algorithm, combining Deep learning with Monte-Carlo Tree Search. AlphaZero can achieve, tabula rasa, superhuman performance in many challenging domains with some training effort. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved a superhuman level of play in the games of chess and Shogi as well as Go, and convincingly defeated a world-champion program in each case ¹³.

Leela Chess Zero

Leela Chess Zero is an adaptation of Gian-Carlo Pascutto's Leela Zero Go project ¹⁴ to Chess.

See also

• AlphaGo
• AlphaZero
• Automated Tuning
• CrazyZero
• Convolutional Neural Networks in Go
• DeepChess
• Giraffe
• Keynote Lecture CG 2016 Conference by Aja Huang
• Leela Chess Zero

Selected Publications

1965 ...

• Alexey G. Ivakhnenko (1965). Cybernetic Predicting Devices. Naukova Dumka .
• Alexey G. Ivakhnenko (1971). Polynomial theory of complex systems. IEEE Transactions on Systems, Man, and Cybernetics, Vol. 1, No. 4

1980 ...

• Kunihiko Fukushima (1980). Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. Biological Cybernetics, Vol. 36, pdf ¹⁵
• David H. Ackley, Geoffrey E. Hinton, Terrence J. Sejnowski (1985). A Learning Algorithm for Boltzmann Machines. Cognitive Science, Vol. 9, No. 1, pdf
• Rina Dechter (1986). Learning While Searching in Constraint-Satisfaction-Problems. AAAI 86, pdf ¹⁶

1990 ...

• Sepp Hochreiter (1991). Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis, TU Munich, advisor Jürgen Schmidhuber, pdf (German) ¹⁷
• Simon Lucas (1991). Connectionist architectures for syntactic pattern recognition. Ph.D. thesis, University of Southampton
• Sepp Hochreiter, Jürgen Schmidhuber (1997). Long short-term memory. Neural Computation, Vol. 9, No. 8, pdf ¹⁸

2000 ...

• Igor Aizenberg, Naum N. Aizenberg, Joos Vandewalle (2000). Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications. Springer ¹⁹
• Faustino J. Gomez, Jürgen Schmidhuber (2005). Co-Evolving Recurrent Neurons Learn Deep Memory POMDPs. GECCO 2005, pdf
• Geoffrey E. Hinton, Simon Osindero, Yee Whye Teh (2006). A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, Vol. 18, No. 7, pdf
• Geoffrey E. Hinton, Ruslan R. Salakhutdinov (2006). Reducing the Dimensionality of Data with Neural Networks. Science, Vol. 313, pdf
• Ilya Sutskever, Vinod Nair (2008). Mimicking Go Experts with Convolutional Neural Networks. ICANN 2008, pdf

2010 ...

• Abdelrahman Mohamed, George E. Dahl, Geoffrey E. Hinton (2011). Acoustic Modeling using Deep Belief Networks. IEEE Transactions on Audio, Speech, and Language Processing, Vol. 20, No. 1, pdf
• Yoshua Bengio (2012). Deep Learning of Representations for Unsupervised and Transfer Learning. JMLR: Workshop on Unsupervised and Transfer Learning, 2011, pdf

2013

• Grégoire Montavon (2013). On Layer-Wise Representations in Deep Neural Networks. Ph.D. Thesis, TU Berlin, advisor Klaus-Robert Müller
• Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller (2013). Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602 ²⁰
• Jürgen Schmidhuber (2013). My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013. arXiv:1312.5548

2014

• Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio (2014). Generative Adversarial Networks. arXiv:1406.2661v1
• Yann Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, Yoshua Bengio (2014). Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. arXiv:1406.2572 ²¹
• Kyunghyun Cho (2014). Foundations and Advances in Deep Learning. Ph.D. thesis, Aalto University, supervisor Juha Karhunen
• Wojciech Jaśkowski, Marcin Szubert, Paweł Liskowski (2014). Multi-Criteria Comparison of Coevolution and Temporal Difference Learning on Othello. EvoApplications 2014, Springer, volume 8602
• Christopher Clark, Amos Storkey (2014). Teaching Deep Convolutional Neural Networks to Play Go. arXiv:1412.3409 ^{22 23}
• Chris J. Maddison, Aja Huang, Ilya Sutskever, David Silver (2014). Move Evaluation in Go Using Deep Convolutional Neural Networks. arXiv:1412.6564v1 » Go
• Jürgen Schmidhuber (2014). Deep Learning in Neural Networks: An Overview. arXiv:1404.7828

2015 ...

• Michael Nielsen (2015). Neural networks and deep learning. Determination Press
• Sergey Ioffe, Christian Szegedy (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv:1502.03167
• Matthew Lai (2015). Giraffe: Using Deep Reinforcement Learning to Play Chess. M.Sc. thesis, Imperial College London, arXiv:1509.01549v1 » Giraffe
• Nikolai Yakovenko, Liangliang Cao, Colin Raffel, James Fan (2015). Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games. arXiv:1509.06731
• Andrew Lavin, Scott Gray (2015). Fast Algorithms for Convolutional Neural Networks. arXiv:1509.09308 ²⁴
• Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, Demis Hassabis (2015). Human-level control through deep reinforcement learning. Nature, Vol. 518
• Emmanuel Bengio, Pierre-Luc Bacon, Joelle Pineau, Doina Precup (2015). Conditional Computation in Neural Networks for faster models. arXiv:1511.06297
• Ilya Loshchilov, Frank Hutter (2015). Online Batch Selection for Faster Training of Neural Networks. arXiv:1511.06343
• Yuandong Tian, Yan Zhu (2015). Better Computer Go Player with Neural Network and Long-term Prediction. arXiv:1511.06410, ICLR 2016 ^{25 26} » Go
• Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens (2015). Adding Gradient Noise Improves Learning for Very Deep Networks. arXiv:1511.06807
• Quoc V. Le (2015). A Tutorial on Deep Learning - Part 1: Nonlinear Classifiers and The Backpropagation Algorithm. Google Brain, pdf ²⁷
• Quoc V. Le (2015). A Tutorial on Deep Learning - Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks. Google Brain, pdf
• Jürgen Schmidhuber (2015). Deep Learning in Neural Networks: An Overview. Neural Networks, Vol. 61
• Zachary C. Lipton, John Berkowitz, Charles Elkan (2015). A Critical Review of Recurrent Neural Networks for Sequence Learning. arXiv:1506.00019v4
• Barak Oshri, Nishith Khandwala (2015). Predicting Moves in Chess using Convolutional Neural Networks. pdf ^{28 29}
• Yann LeCun, Yoshua Bengio, Geoffrey E. Hinton (2015). Deep Learning. Nature, Vol. 521 ³⁰
• Peter H. Jin, Kurt Keutzer (2015). Convolutional Monte Carlo Rollouts in Go. arXiv:1512.03375 » Go, MCTS
• Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (2015). Deep Residual Learning for Image Recognition. arXiv:1512.03385

2016

• Tim Dettmers (2016). 8-Bit Approximations for Parallelism in Deep Learning. arXiv:1511.04561v4, ICLR 2016
• Ziyu Wang, Nando de Freitas, Marc Lanctot (2016). Dueling Network Architectures for Deep Reinforcement Learning. arXiv:1511.06581
• David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, Demis Hassabis (2016). Mastering the game of Go with deep neural networks and tree search. Nature, Vol. 529 » AlphaGo
• Tobias Graf, Marco Platzner (2016). Using Deep Convolutional Neural Networks in Monte Carlo Tree Search. CG 2016
• Aja Huang (2016). AlphaGo: Combining Deep Neural Networks with Tree Search. CG 2016, Keynote Lecture
• Hung Guei, Tinghan Wei, Jin-Bo Huang, I-Chen Wu (2016). An Empirical Study on Applying Deep Reinforcement Learning to the Game 2048. CG 2016
• Omid E. David, Nathan S. Netanyahu, Lior Wolf (2016). DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess. ICAAN 2016, Lecture Notes in Computer Science, Vol. 9887, Springer, pdf preprint » DeepChess ^{31 32}
• Dror Sholomon, Omid E. David, Nathan S. Netanyahu (2016). DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem. ICAAN 2016, Lecture Notes in Computer Science, Vol. 9887, Springer ³³
• Ian Goodfellow, Yoshua Bengio, Aaron Courville (2016). Deep Learning. MIT Press
• Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu (2016). Asynchronous Methods for Deep Reinforcement Learning. arXiv:1602.01783v2
• Johannes Heinrich, David Silver (2016). Deep Reinforcement Learning from Self-Play in Imperfect-Information Games. arXiv:1603.01121 ³⁴
• Vincent Dumoulin, Francesco Visin (2016). A guide to convolution arithmetic for deep learning. arXiv:1603.07285
• Ilya Loshchilov, Frank Hutter (2016). CMA-ES for Hyperparameter Optimization of Deep Neural Networks. arXiv:1604.07269 ³⁵
• Dale Schuurmans, Martin Zinkevich (2016). Deep Learning Games. NIPS 2016
• Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, Raia Hadsell (2016). Progressive Neural Networks. arXiv:1606.04671
• Ilya Loshchilov, Frank Hutter (2016). SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv:1608.03983
• Shixiang Gu, Ethan Holly, Timothy Lillicrap, Sergey Levine (2016). Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates. arXiv:1610.00633
• Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Rémi Munos, Charles Blundell, Dharshan Kumaran, Matthew Botvinick (2016). Learning to reinforcement learn. arXiv:1611.05763
• Jonathan Rosenthal (2016). Deep Learning for Go. B.Sc. thesis, ETH Zurich
• James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, Raia Hadsell (2016). Overcoming catastrophic forgetting in neural networks. arXiv:1612.00796 ³⁶
• Zhenxing Niu, Mo Zhou, Le Wang, Xinbo Gao, Gang Hua (2016). Ordinal Regression with Multiple Output CNN for Age Estimation. CVPR 2016, pdf
• Li Jing, Yichen Shen, Tena Dubček, John Peurifoy, Scott Skirlo, Yann LeCun, Max Tegmark, Marin Soljačić (2016). Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs. arXiv:1612.05231 ³⁷

2017

• Tristan Cazenave (2017). Residual Networks for Computer Go. IEEE Transactions on Computational Intelligence and AI in Games, Vol. PP, No. 99, pdf
• Shi-Jim Yen, Ching-Nung Lin, Guan-Lun Cheng, Jr-Chang Chen (2017). Deep Learning and Block Go. IJCNN 2017
• Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, Michael Bowling (2017). DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. Science, Vol. 356, No. 6337
• Tristan Cazenave (2017). Improved Policy Networks for Computer Go. Advances in Computer Games 15, pdf
• Hirotaka Kameko, Jun Suzuki, Naoki Mizukami, Yoshimasa Tsuruoka (2017). Deep Reinforcement Learning with Hidden Layers on Future States. CGW@IJCAI 2017, pdf
• Keigo Kawamura, Naoki Mizukami, Yoshimasa Tsuruoka (2017). Neural Fictitious Self-Play in Imperfect Information Games with Many Players. CGW@IJCAI 2017, pdf
• Thomas Philip Runarsson (2017). Deep Preference Neural Network for Move Prediction in Board Games. CGW@IJCAI 2017
• David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, Demis Hassabis (2017). Mastering the game of Go without human knowledge. Nature, Vol. 550, pdf ³⁸
• Shantanu Thakoor, Surag Nair, Megha Jhunjhunwala (2017). Learning to Play Othello Without Human Knowledge. Stanford University, pdf » AlphaZero, MCTS, Othello ³⁹
• Masatoshi Hidaka, Yuichiro Kikura, Yoshitaka Ushiku, Tatsuya Harada (2017). WebDNN: Fastest DNN Execution Framework on Web Browser. ACM Multimedia 2017, pdf ⁴⁰
• Francisco A. Matos, Diogo R. Ferreira, Pedro J. Carvalho, JET Contributors (2017). Deep learning for plasma tomography using the bolometer system at JET. arXiv:1701.00322
• Masatoshi Hidaka, Ken Miura, Tatsuya Harada (2017). Development of JavaScript-based deep learning platform and application to distributed training. arXiv:1702.01846, ICLR 2017
• Sergey Ioffe (2017). Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models. arXiv:1702.03275
• Risto Miikkulainen, et al. (2017). Evolving Deep Neural Networks. arXiv:1703.00548
• Thomas Anthony, Zheng Tian, David Barber (2017). Thinking Fast and Slow with Deep Learning and Tree Search. arXiv:1705.08439
• Ti-Rong Wu, I-Chen Wu, Guan-Wun Chen, Ting-Han Wei, Tung-Yi Lai, Hung-Chun Wu, Li-Cheng Lan (2017). Multi-Labelled Value Networks for Computer Go. arXiv:1705.10701
• Olivier Bousquet, Sylvain Gelly, Karol Kurach, Marc Schoenauer, Michèle Sebag, Olivier Teytaud, Damien Vincent (2017). Toward Optimal Run Racing: Application to Deep Learning Calibration. arXiv:1706.03199
• Adams Wei Yu, Lei Huang, Qihang Lin, Ruslan Salakhutdinov, Jaime Carbonell (2017). Block-Normalized Gradient Method: An Empirical Study for Training Deep Neural Network. arXiv:1707.04822
• Alice Schoenauer-Sebag, Marc Schoenauer, Michèle Sebag (2017). Stochastic Gradient Descent: Going As Fast As Possible But Not Faster. arXiv:1709.01427
• Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, David Meger (2017). Deep Reinforcement Learning that Matters. arXiv:1709.06560
• Matthia Sabatelli (2017). Learning to Play Chess with Minimal Lookahead and Deep Value Neural Networks. Master's thesis, University of Groningen, pdf ⁴¹
• Marc Lanctot, Vinícius Flores Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Pérolat, David Silver, Thore Graepel (2017). A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning. arXiv:1711.00832
• Maithra Raghu, Alex Irpan, Jacob Andreas, Robert Kleinberg, Quoc V. Le, Jon Kleinberg (2017). Can Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games? arXiv:1711.02301
• Paweł Liskowski, Wojciech Jaśkowski, Krzysztof Krawiec (2017). Learning to Play Othello with Deep Neural Networks. arXiv:1711.06583 ⁴²
• David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis (2017). Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. arXiv:1712.01815 » AlphaZero
• George Philipp, Jaime Carbonell (2017). Nonparametric Neural Networks. arXiv:1712.05440
• George Philipp, Dawn Song, Jaime Carbonell (2017). The exploding gradient problem demystified - definition, prevalence, impact, origin, tradeoffs, and solutions. arXiv:1712.05577
• Qiming Chen, Ren Wu (2017). CNN Is All You Need. arXiv:1712.09662

2018

• Paweł Liskowski, Wojciech Jaśkowski, Krzysztof Krawiec (2018). Learning to Play Othello with Deep Neural Networks. IEEE Transactions on Games
• Matthia Sabatelli, Francesco Bidoia, Valeriu Codreanu, Marco Wiering (2018). Learning to Evaluate Chess Positions with Deep Neural Networks and Limited Lookahead. ICPRAM 2018, pdf
• Diogo R. Ferreira, Pedro J. Carvalho, Horácio Fernandes, JET Contributors (2018). Full-pulse Tomographic Reconstruction with Deep Neural Networks. arXiv:1802.02242
• Aditya Rawal, Risto Miikkulainen (2018). From Nodes to Networks: Evolving Recurrent Neural Networks. arXiv:1803.04439
• George Philipp, Jaime Carbonell (2018). The Nonlinearity Coefficient - Predicting Generalization in Deep Neural Networks. arXiv:1806.00179
• Sai Krishna G.V., Kyle Goyette, Ahmad Chamseddine, Breandan Considine (2018). Deep Pepper: Expert Iteration based Chess agent in the Reinforcement Learning Setting. arXiv:1806.00683 ⁴³
• Naman Agarwal, Brian Bullins, Xinyi Chen, Elad Hazan, Karan Singh, Cyril Zhang, Yi Zhang (2018). The Case for Full-Matrix Adaptive Regularization. arXiv:1806.02958
• Ashwin Srinivasan, Lovekesh Vig, Michael Bain (2018). Logical Explanations for Deep Relational Machines Using Relevance Information. arXiv:1807.00595
• David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, Vol. 362, No. 6419 ⁴⁴
• Hanhua Zhu, Tomoyuki Kaneko (2018). Comparison of Loss Functions for Training of Deep Neural Networks in Shogi. TAAI 2018
• Tianhe Wang, Tomoyuki Kaneko (2018). Application of Deep Reinforcement Learning in Werewolf Game Agents. TAAI 2018
• Hyunwoo Oh, Tomoyuki Kaneko (2018). Deep Recurrent Q-Network with Truncated History. TAAI 2018
• Ching-Nung Lin, Jr-Chang Chen, Shi-Jim Yen, Chan-San Chen (2018). Design of a Block Go program using deep learning and Monte Carlo tree search. ICGA Journal, Vol. 40, No. 3
• Chao Gao, Siqi Yan, Ryan Hayward, Martin Müller (2018). A transferable neural network for Hex. CG 2018, ICGA Journal, Vol. 40, No. 3
• Nai-Yuan Chang, Chih-Hung Chen, Shun-Shii Lin, Surag Nair (2018). The Big Win Strategy on Multi-Value Network: An Improvement over AlphaZero Approach for 6x6 Othello. MLMI2018

2019

• David J. Wu (2019). Accelerating Self-Play Learning in Go. arXiv:1902.10565
• Aston Zhang, Zack C. Lipton, Mu Li, Alex J. Smola (2019). Dive into Deep Learning. An interactive deep learning book with code, math, and discussions
• Johannes Czech (2019). Deep Reinforcement Learning for Crazyhouse. Master thesis, TU Darmstadt, pdf » CrazyAra
• Hsiao-Chung Hsieh, Ti-Rong Wu, Ting-Han Wei, I-Chen Wu (2019). Net2Net Extension for the AlphaGo Zero Algorithm. Advances in Computer Games 16
• Tomihiro Kimura, Kokolo Ikeda (2019). Designing Policy Network with Deep Learning in Turn-Based Strategy Games. Advances in Computer Games 16

2020 ...

• Garrett Bingham, William Macke, Risto Miikkulainen (2020). Evolutionary Optimization of Deep Learning Activation Functions. arXiv:2002.07224
• Jason Liang, Santiago Gonzalez, Risto Miikkulainen (2020). Population-Based Training for Loss Function Optimization. arXiv:2002.04225
• Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, David Silver (2020). Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, Vol. 588 ^{45 46}
• Reid McIlroy-Young, Siddhartha Sen, Jon Kleinberg, Ashton Anderson (2020). Aligning Superhuman AI with Human Behavior: Chess as a Model System. In Proceedings of the 26th ACM SIGKDD 2020, arXiv:2006.01855 » Maia Chess
• Reid McIlroy-Young, Russell Wang, Siddhartha Sen, Jon Kleinberg, Ashton Anderson (2020). Learning Personalized Models of Human Behavior in Chess. arXiv:2008.10086
• Tristan Cazenave, Yen-Chi Chen, Guan-Wei Chen, Shi-Yu Chen, Xian-Dong Chiu, Julien Dehos, Maria Elsa, Qucheng Gong, Hengyuan Hu, Vasil Khalidov, Cheng-Ling Li, Hsin-I Lin, Yu-Jin Lin, Xavier Martinet, Vegard Mella, Jeremy Rapin, Baptiste Roziere, Gabriel Synnaeve, Fabien Teytaud, Olivier Teytaud, Shi-Cheng Ye, Yi-Jun Ye, Shi-Jim Yen, Sergey Zagoruyko (2020). Polygames: Improved zero learning. ICGA Journal, Vol. 42, No. 4, arXiv:2001.09832
• Caspar van Leeuwen, Damian Podareanu, Valeriu Codreanu, Maxwell X. Cai, Axel Berg, Simon Portegies Zwart, Robin Stoffer, Menno Veerman, Chiel van Heerwaarden, Sydney Otten, Sascha Caron, Cunliang Geng, Francesco Ambrosetti, Alexandre M.J.J. Bonvin (2020). Deep-learning enhancement of large scale numerical simulations. arXiv:2004.03454
• Johannes Czech, Moritz Willig, Alena Beyer, Kristian Kersting, Johannes Fürnkranz (2020). Learning to Play the Chess Variant Crazyhouse Above World Champion Level With Deep Neural Networks and Human Data. Frontiers in Artificial Intelligence » CrazyAra
• Quentin Cohen-Solal (2020). Learning to Play Two-Player Perfect-Information Games without Knowledge. arXiv:2008.01188
• Quentin Cohen-Solal, Tristan Cazenave (2020). Minimax Strikes Back. arXiv:2012.10700

2021

• Johannes Czech, Patrick Korus, Kristian Kersting (2021). Improving AlphaZero Using Monte-Carlo Graph Search. Proceedings of the Thirty-First International Conference on Automated Planning and Scheduling, Vol. 31, pdf
• Maximilian Langer (2021). Evaluation of Monte-Carlo Tree Search for Xiangqi. B.Sc. thesis, advisors Kristian Kersting and Johannes Czech, TU Darmstadt, pdf » Xiangqi
• Maximilian Alexander Gehrke (2021). Assessing Popular Chess Variants Using Deep Reinforcement Learning. Master thesis, TU Darmstadt, pdf » CrazyAra
• Dominik Klein (2021). Neural Networks For Chess. Release Version 1.1 · GitHub ⁴⁷
• Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Demis Hassabis, Been Kim, Ulrich Paquet, Vladimir Kramnik (2021). Acquisition of Chess Knowledge in AlphaZero. arXiv:2111.09259 ⁴⁸
• Tristan Cazenave, Julien Sentuc, Mathurin Videau (2021). Cosine Annealing, Mixnet and Swish Activation for Computer Go. Advances in Computer Games 17
• Hung-Jui Chang, Cheng Yueh, Gang-Yu Fan, Ting-Yu Lin, Tsan-sheng Hsu (2021). Opponent Model Selection Using Deep Learning. Advances in Computer Games 17
• Rejwana Haque, Ting Han Wei, Martin Müller (2021). On the Road to Perfection? Evaluating Leela Chess Zero Against Endgame Tablebases. Advances in Computer Games 17
• Boris Doux, Benjamin Negrevergne, Tristan Cazenave (2021). Deep Reinforcement Learning for Morpion Solitaire. Advances in Computer Games 17
• Aðalsteinn Pálsson, Yngvi Björnsson (2021). Evaluating Interpretability Methods for DNNs in Game-Playing Agents. Advances in Computer Games 17
• Dennis Soemers, Vegard Mella, Cameron Browne, Olivier Teytaud (2021). Deep learning for general game playing with Ludii and Polygames. ICGA Journal, Vol. 43, No. 3

Forum Posts

2014

• Teaching Deep Convolutional Neural Networks to Play Go by Hiroshi Yamashita, The Computer-go Archives, December 14, 2014
• Teaching Deep Convolutional Neural Networks to Play Go by Michel Van den Bergh, CCC, December 16, 2014

2015 ...

• Who introduced the term “deep learning” to the field of Machine Learning by Jürgen Schmidhuber, Google+, March 18, 2015 ⁴⁹
• *First release* Giraffe, a new engine based on deep learning by Matthew Lai, CCC, July 08, 2015 » Giraffe

2016

• Chess position evaluation with convolutional neural network in Julia by Kamil Czarnogorski, Machine learning with Julia and python, April 02, 2016 ⁵⁰
• Deep Learning Chess Engine ? by Eren Yavuz, CCC, July 21, 2016

Re: Deep Learning Chess Engine ? by Alexandru Mosoi, CCC, July 21, 2016 » Zurichess

Re: Deep Learning Chess Engine ? by Matthew Lai, CCC, August 04, 2016 » Giraffe ⁵¹

• Neuronet plus conventional approach combined? by Rasmus Althoff, CCC, September 02, 2016
• DeepChess: Another deep-learning based chess program by Matthew Lai, CCC, October 17, 2016 » DeepChess
• The scaling of Deep Learning MCTS Go engines by Kai Laskos, CCC, October 23, 2016 » Deep Learning, Go, MCTS

2017

• No one really knows how the most advanced algorithms work by Daniel José Queraltó, CCC, April 11, 2017
• How far away are we from deep learning Stockfish, Komodo,... by John Margusen, CCC, May 19, 2017
• Is AlphaGo approach unsuitable to chess? by Mel Cooper, CCC, May 27, 2017 » AlphaGo, Giraffe

Re: Is AlphaGo approach unsuitable to chess? by Peter Österlund, CCC, May 31, 2017 » Texel

• Neural nets for Go - chain pooling? by David Wu, Computer Go Archive, August 18, 2017
• We are doomed - AlphaGo Zero, learning only from basic rules by Vincent Lejeune, CCC, October 18, 2017
• AlphaGo Zero by Alberto Sanjuan, CCC, October 19, 2017
• Zero performance by Gian-Carlo Pascutto, Computer Go Archive, October 20, 2017 » AlphaGo,
• Neural networks for chess position evaluation- request by Kamil Czarnogorski, CCC, November 13, 2017
• Google's AlphaGo team has been working on chess by Peter Kappler, CCC, December 06, 2017 » AlphaZero
• Historic Milestone: AlphaZero by Miguel Castanuela, CCC, December 06, 2017
• An AlphaZero inspired project by Truls Edvard Stokke, CCC, December 14, 2017 » AlphaZero
• To TPU or not to TPU... by Srdja Matovic, CCC, December 16, 2017

Re: To TPU or not to TPU... by Rémi Coulom, CCC, December 16, 2017

2018

• Announcing lczero by Gary, CCC, January 09, 2018 » Leela Chess Zero
• Connect 4 AlphaZero implemented using Python... by Steve Maughan, CCC, January 29, 2018 » AlphaZero, Connect Four, Python
• use multiple neural nets? by Warren D. Smith, LCZero Forum, December 25, 2018 » Leela Chess Zero

2019

• categorical cross entropy for value by Chris Whittington, CCC, February 18, 2019
• Analysis of policy nets by Chris Whittington, CCC, March 27, 2019
• Wouldn't it be nice if C++ GPU by Chris Whittington, CCC, April 25, 2019
• Training using 1 playout instead of 800 by Daniel Shawul, CCC, April 26, 2019
• Re: On-line engine blitz tourney April by Rémi Coulom, CCC, April 27, 2019
• Question to Remi about CrazyZero by Harm Geert Muller, CCC, April 28, 2019 » CrazyZero
• SL vs RL by Chris Whittington, CCC, April 28, 2019
• catastrophic forgetting by Daniel Shawul, CCC, May 09, 2019 » Nebiyu
• A question to MCTS + NN experts by Maksim Korzh, CCC, July 17, 2019 » Monte-Carlo Tree Search

Re: A question to MCTS + NN experts by Daniel Shawul, CCC, July 17, 2019

• My home-made CUDA kernel for convolutions by Rémi Coulom, Game-AI Forum, November 09, 2019
• high dimensional optimization by Warren D. Smith, FishCooking, December 27, 2019 ⁵²

2020 ...

• How to work with batch size in neural network by Gertjan Brouwer, CCC, June 02, 2020
• Maiachess by Marc-Philippe Huget, CCC, December 04, 2020 » Maia Chess

External Links

• Deep learning from Wikipedia
• Hierarchical Deep Learning from Wikipedia
• Deep Learning - Scholarpedia by Jürgen Schmidhuber
• Very Deep Learning Since 1991 by Jürgen Schmidhuber
• What is deep learning? - Quora
• Deep Learning

Networks

• Boltzmann machine from Wikipedia
• Convolutional neural network from Wikipedia

Convolutional Neural Networks for Image and Video Processing, TUM Wiki, Technical University of Munich

An Introduction to different Types of Convolutions in Deep Learning by Paul-Louis Pröve, July 22, 2017

Squeeze-and-Excitation Networks by Paul-Louis Pröve, October 17, 2017

• Deep belief network from Wikipedia
• Deep neural networks from Wikipedia
• Deep Residual Networks from TUM Wiki, Technical University of Munich
• Hopfield network from Wikipedia
• Long short-term memory from Wikipedia
• Neocognitron from Wikipedia
• Neocognitron - Scholarpedia by Kunihiko Fukushima
• Recurrent neural network from Wikipedia

Software

• Comparison of deep learning software from Wikipedia
• DeepDream from Wikipedia

Libraries

• Apache Singa from Wikipedia
• Caffe | Deep Learning Framework
• CNTK (The Microsoft Cognitive Toolkit) from Wikipedia
• Deeplearning4j from Wikipedia » Java
• keras/keras at master · fchollet/keras · GitHub » Python
• Keras Documentation
• PyTorch from Wikipedia » Python
• TensorFlow from Wikipedia
• GitHub - jtoy/awesome-tensorflow: TensorFlow
• scikit-learn from Wikipedia » Python
• Theano (software) from Wikipedia

Chess

• GitHub - BarakOshri/ConvChess: Predicting Moves in Chess Using Convolutional Neural Networks ⁵³
• GitHub - erikbern/deep-pink: Deep Pink is a chess AI that learns to play chess using deep learning » Deep Pink
• GitHub - benediamond/leela-chess: A chess adaption of GCP's Leela Zero
• GitHub - glinscott/leela-chess: A chess adaption of GCP's Leela Zero » Leela Chess Zero
• GitHub - CSSLab/maia-chess: Human like chess engines » Maia Chess
• GitHub - Zeta36/chess-alpha-zero: Chess reinforcement learning by AlphaGo Zero methods » Zeta36 ⁵⁴

Games

• GitHub - gcp/leela-zero: Go engine with no human-provided knowledge, modeled after the AlphaGo Zero paper by Gian-Carlo Pascutto et al. » Leela Zero
• How to build your own AlphaZero AI using Python and Keras by David Foster, January 26, 2018 » AlphaZero, Connect Four, Python ⁵⁵
• Open-sourcing Polygames, a new framework for training AI bots through self-play
• GitHub - facebookarchive/Polygames: The project is a platform of zero learning with a library of games

Music Generation

• Analyzing Six Deep Learning Tools for Music Generation by Frank Brinkkemper, The Asimov Institute, October 5, 2016
• GitHub - jisungk/deepjazz: Deep learning driven jazz generation using Keras & Theano!
• GitHub - tensorflow/magenta: Magenta: Music and Art Generation with Machine Intelligence
• GitHub - feynmanliang/bachbot: Use deep learning to generate and harmonize music in the style of Bach
• GitHub - MattVitelli/GRUV: GRUV is a Python project for algorithmic music generation

Nvidia

• Deep Learning | NVIDIA Developer » GPU
• Deep Learning in a Nutshell: Core Concepts by Tim Dettmers, Parallel Forall, November 3, 2015
• Deep Learning in a Nutshell: History and Training by Tim Dettmers, Parallel Forall, December 16, 2015
• Deep Learning in a Nutshell: Sequence Learning by Tim Dettmers, Parallel Forall, March 7, 2016
• Deep Learning in a Nutshell: Reinforcement Learning by Tim Dettmers, Parallel Forall, September 8, 2016

Reports & Blogs

• My First Deep Learning System 1991 / Deep Learning Timeline 1960-2013 by Jürgen Schmidhuber
• Deep Learning by Robert Hof, MIT Technology Review, early 2013
• Deep learning for… chess by Erik Bernhardsson, November 29, 2014
• Deep learning for… chess (addendum) by Erik Bernhardsson, December 8, 2014
• Deep learning for… Go by Erik Bernhardsson, December 11, 2014 » DCNN in GO
• Why GEMM is at the heart of deep learning « Pete Warden's blog, April 20, 2015 ^{56 57}
• The Brain vs. Deep Learning vs. Singularity by Tim Dettmers, July 27, 2015
• Faster deep learning with GPUs and Theano by Manojit Nandi, August 05, 2015 » GPU, Python
• Exploring Deep Learning with LI Zhe by Nivdul, November 17, 2015
• How Facebook’s AI Researchers Built a Game-Changing Go Engine | MIT Technology Review, December 04, 2015 ⁵⁸
• 7 Steps to Understanding Deep Learning by Matthew Mayo, KDNuggets, January 13, 2016
• Could DeepMind try to conquer poker next? by Alex Hern, The Guardian, March 30, 2016 ⁵⁹

Texas Hold'em: AI is almost as good as humans at playing poker by Matt Burgess, Wired UK, March 30, 2016

• Google Research Blog: Wide & Deep Learning: Better Together with TensorFlow by Heng-Tze Cheng, June 29, 2016
• Enabling Continual Learning in Neural Networks by James Kirkpatrick, Joel Veness et al., DeepMind, March 13, 2017
• Understand Deep Residual Networks — a simple, modular learning framework that has redefined state-of-the-art by Michael Dietz, Waya.ai, May 02, 2017
• AlphaGo Zero: Learning from scratch by Demis Hassabis and David Silver, DeepMind, October 18, 2017
• Kasparov on Deep Learning in chess by Frederic Friedel, ChessBase News, December 13, 2017 » AlphaZero
• A Simple Alpha(Go) Zero Tutorial by Surag Nair, Stanford University, December 29, 2017 » AlphaZero, MCTS ⁶⁰

GitHub - suragnair/alpha-zero-general: A clean and simple implementation of a self-play learning algorithm based on AlphaGo Zero (any game, any framework!)

• Lower Numerical Precision Deep Learning Inference and Training by Andres Rodriguez et al., Intel, January 19, 2018 » AVX-512

Videos

• Deep Learning SIMPLIFIED: The Series Intro YouTube Videos
• Deep Learning RNNaissance with Jürgen Schmidhuber, YouTube Video

Watch on YouTube

References