RNN (Recurrent Neural nets) are used to model sequences. Unlike the usual feedforward nets which are stateless in terms on inputs, RNNs have memory. In particular, its inputs are the output of previous step and also new observation in current step.

The basic RNN are notoriously hard to train. LSTM (Long short term memory) networks are an adaptation of RNNs which make training long sequence dependencies feasible.

[My Presentation]

In addition to the deep learning book’s chapters here are some of the nice blog post which explains RNN:

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s