Benchmark of Popular Reinforcement Learning Algorithms

This post is a reflection of my study through OpenAI’s tutorial, Spinningup, of deep reinforcement learning. It mainly covers six popular algorithms including Vanilla Policy Gradient (VPG), Trust Region Policy Optimization (TRPO), Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3) and Soft Actor-Critic (SAC). I have implemented all these algorithms under the guidance of...

Install GPU-enabled Tensorflow with Ubuntu 18.04

Ubuntu 18.04 is released recently. Many developers will adapt the Tensorflow development environment with the new GNOME style’s Ubuntu OS. Here I show my successful installation of Tensorflow with GPU in Ubuntu 18.04. Hope it can help you out of the mess for multiple platforms’ compatability.

Platforms

• Ubuntu 18.04

• Tensorflow 1.7

• CUDA 9.1

  ...

Understansing the forward and backward pass of LSTM

Recurrent Neural Network(RNN) is a specific learning approach for the sequence generation. It is naturally applied in natural launguage processing(NLP), image captioning and time series prediction. One difficulty to train the RNN is that the gradient will be vanished in the Vanilla RNN, since the gradient of hidden state (\(h\)) involves many multiplication of the weight matrix(\(W\)). Long Short Term...

Understansing the forward and backward pass of Batch Normalization

Batch normalization, as it is proposed in [1], is a popular technique in deep learning to speed up the training progress and reduce the difficulty to train deep neural networks. As the authors in [1] hypothsize that the shifted distribution of the features may causes the training much harder, especially at deep layers. The deep learning methods usually works better...