TensorFlow and Deep Learning Tutorials
Google's Deep Learning Tutorials
- TensorFlow Official Deep Learning Tutorial (http://wiki.jikexueyuan.com/project/tensorflow-zh/).
- MLP with Dropout TensorFlow (http://wiki.jikexueyuan.com/project/tensorflow-zh/tutorials/mnist_beginners.html) TensorLayer (http://tensorlayercn.readthedocs.io/zh/latest/user/tutorial.html#tensorlayer)
- Autoencoder TensorLayer (http://tensorlayercn.readthedocs.io/zh/latest/user/tutorial.html#denoising-autoencoder)
- Convolutional Neural Network TensorFlow (http://wiki.jikexueyuan.com/project/tensorflow-zh/tutorials/mnist_pros.html) TensorLayer (http://tensorlayercn.readthedocs.io/zh/latest/user/tutorial.html#convolutional-neural-network)
- Recurrent Neural Network TensorFlow (http://wiki.jikexueyuan.com/project/tensorflow-zh/tutorials/recurrent.html) TensorLayer (http://tensorlayercn.readthedocs.io/zh/latest/user/tutorial.html#lstm)
- Deep Reinforcement Learning TensorLayer (http://tensorlayercn.readthedocs.io/zh/latest/user/tutorial.html#id13)
- Sequence to Sequence TensorFlow TensorLayer(http://tensorlayercn.readthedocs.io/zh/latest/user/tutorial.html#id30)
- Word Embedding TensorFlow (http://wiki.jikexueyuan.com/project/tensorflow-zh/tutorials/word2vec.html) TensorLayer (http://tensorlayercn.readthedocs.io/zh/latest/user/tutorial.html#word-embedding)
Deep Learning Reading List
- MIT Deep Learning Book
- Karpathy Blog
- Stanford UFLDL Tutorials
- Colah's Blog - Word Embedding (http://dataunion.org/9331.html)
- Colah's Blog - Understand LSTN (http://mp.weixin.qq.com/s?__biz=MzI3NDExNDY3Nw==&mid=2649764821&idx=1&sn=dd325565b40fcbad6e90a9398414dede&scene=2&srcid=0505U2iFJ7tfXgB8yPfNkwrA&from=timeline&isappinstalled=0#wechat_redirect)
Tutorial index
0 - Prerequisite
1 - Introduction
2 - Basic Models
- Nearest Neighbor (notebook) (code)
- Linear Regression (notebook) (code)
- Logistic Regression (notebook) (code)
3 - Neural Networks
- Multilayer Perceptron (notebook) (code)
- Convolutional Neural Network (notebook) (code)
- Recurrent Neural Network (LSTM) (notebook) (code)
- Bidirectional Recurrent Neural Network (LSTM) (notebook) (code)
- Dynamic Recurrent Neural Network (LSTM) (code)
- AutoEncoder (notebook) (code)
4 - Utilities
- Save and Restore a model (notebook) (code)
- Tensorboard - Graph and loss visualization (notebook) (code)
- Tensorboard - Advanced visualization (code)
5 - Multi GPU
Dataset
Some examples require MNIST dataset for training and testing. Don't worry, this dataset will automatically be downloaded when running examples (with input_data.py).
MNIST is a database of handwritten digits, for a quick description of that dataset, you can check this notebook.
Official Website: http://yann.lecun.com/exdb/mnist/
Selected Repositories
Tricks
Examples
Basics
- Multi-layer perceptron (MNIST) - Classification task, see tutorial_mnist_simple.py.
- Multi-layer perceptron (MNIST) - Classification using Iterator, see method1 and method2.
Computer Vision
- Denoising Autoencoder (MNIST). Classification task, see tutorial_mnist.py.
- Stacked Denoising Autoencoder and Fine-Tuning (MNIST). Classification task, see tutorial_mnist.py.
- Convolutional Network (MNIST). Classification task, see tutorial_mnist.py.
- Convolutional Network (CIFAR-10). Classification task, see tutorial_cifar10.py and tutorial_cifar10_tfrecord.py.
- VGG 16 (ImageNet). Classification task, see tutorial_vgg16.py.
- VGG 19 (ImageNet). Classification task, see tutorial_vgg19.py.
- InceptionV3 (ImageNet). Classification task, see tutorial_inceptionV3_tfslim.py.
- Wide ResNet (CIFAR) by ritchieng.
- More CNN implementations of TF-Slim can be connected to TensorLayer via SlimNetsLayer.
- Spatial Transformer Networks by zsdonghao.
- U-Net for brain tumor segmentation by zsdonghao.
- Variational Autoencoder (VAE) for (CelebA) by yzwxx.
- Variational Autoencoder (VAE) for (MNIST) by BUPTLdy.
- Image Captioning - Reimplementation of Google's im2txt by zsdonghao.
Natural Language Processing
- Recurrent Neural Network (LSTM). Apply multiple LSTM to PTB dataset for language modeling, see tutorial_ptb_lstm.py and tutorial_ptb_lstm_state_is_tuple.py.
- Word Embedding (Word2vec). Train a word embedding matrix, see tutorial_word2vec_basic.py.
- Restore Embedding matrix. Restore a pre-train embedding matrix, see tutorial_generate_text.py.
- Text Generation. Generates new text scripts, using LSTM network, see tutorial_generate_text.py.
- Chinese Text Anti-Spam by pakrchen.
- Chatbot in 200 lines of code for Seq2Seq.
- FastText Sentence Classification (IMDB), see tutorial_imdb_fasttext.py by tomtung.
Adversarial Learning
- DCGAN (CelebA). Generating images by Deep Convolutional Generative Adversarial Networks by zsdonghao.
- Generative Adversarial Text to Image Synthesis by zsdonghao.
- Unsupervised Image to Image Translation with Generative Adversarial Networks by zsdonghao.
- Improved CycleGAN with resize-convolution by luoxier
- Super Resolution GAN by zsdonghao.
- DAGAN: Fast Compressed Sensing MRI Reconstruction by nebulaV.
Reinforcement Learning
- Policy Gradient / Network (Atari Ping Pong), see tutorial_atari_pong.py.
- Deep Q-Network (Frozen lake), see tutorial_frozenlake_dqn.py.
- Q-Table learning algorithm (Frozen lake), see tutorial_frozenlake_q_table.py.
- Asynchronous Policy Gradient using TensorDB (Atari Ping Pong) by nebulaV.
- AC for discrete action space (Cartpole), see tutorial_cartpole_ac.py.
- A3C for continuous action space (Bipedal Walker), see tutorial_bipedalwalker_a3c*.py.
- DAGGER for (Gym Torcs) by zsdonghao.
- TRPO for continuous and discrete action space by jjkke88.
Miscellaneous
- Distributed Training. mnist and imagenet by jorgemf.
- Merge TF-Slim into TensorLayer. tutorial_inceptionV3_tfslim.py.
- Merge Keras into TensorLayer. tutorial_keras.py.
- Data augmentation with TFRecord. Effective way to load and pre-process data, see tutorial_tfrecord*.py and tutorial_cifar10_tfrecord.py.
- Data augmentation with TensorLayer, see tutorial_image_preprocess.py.
- TensorDB by fangde see here.
- A simple web service - TensorFlask by JoelKronander.
- Float 16 half-precision model, see tutorial_mnist_float16.py