Recurrent Neural Net: Memo

Posted on January 11, 2017


RNN (Recurrent Neural nets) are used to model sequences. Unlike the usual feedforward nets which are stateless in terms on inputs, RNNs have memory. In particular, its inputs are the output of previous step and also new observation in current step.

The basic RNN are notoriously hard to train. LSTM (Long short term memory) networks are an adaptation of RNNs which make training long sequence dependencies feasible.

In addition to the deep learning book’s chapters here are some of the nice blog post which explain RNN:

Posted in: Research Blog