Recurrent Neural Net: Memo

RNN (Recurrent Neural nets) are used to model sequences. Unlike the usual feedforward nets which are stateless in terms on inputs, RNNs have memory. In particular, its inputs are the output of previous step and also new observation in current step.

The basic RNN are notoriously hard to train. LSTM (Long short term memory) networks are an adaptation of RNNs which make training long sequence dependencies feasible.

[My Presentation]

In addition to the deep learning book’s chapters here are some of the nice blog post which explains RNN:


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s