Understanding LSTM — Resources
Fun introduction
To get general idea, its fun to watch the deep learning celeb you-tuber, Siraj raval. It has nice explanation of the architecture, with full demonstration of the code on a simple stock market example.
Recurrent Neural networks
Recurrent neural network, RNN, are the basic concept of networks with memory, which are capable of learning sequential data. LSTM are specific (more complex) kind of RNN, and its the one most robust model.
This popular post: Understanding LSTM Networks — August 27, 2015, is a good starting point in understanding the building blocks of RNN architecture.
Very good explanation of the RNN concept ,architecture and limitations. A short summary is that RNN have a “fixed” memory structure, that enables the model to “remember” only what was designed in advance.
Clear explanation of LSTM, and how they overcome the RNN “fixed” memory limitation by learning long-term dependencies. LSTM have an architecture that enables learning of what to remember.
Stanford CS231 RNN lecture
There are few versions of this Deep learning iconic course, my favorite is: Stanford CS231n Winter 2016: Lecture 10: Recurrent Neural Networks, Image Captioning, LSTM. Lecture is given by Andrej Karpathy, and i recommend his blog as well. The older posts are on git.
Lecture slides: http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture10.pdf
Course syllabus: http://cs231n.stanford.edu/syllabus.html
LSTM examples in different fields
Character-level language models
Recommended article “The Unreasonable Effectiveness of Recurrent Neural Networks” from Andrej Karpathy blog. Those who like to understand concepts by writing codes (like me :), will enjoy this article that includes complete code to generate text by predicting the next character.
Time series classification
Clear well explained post, “A Hands-On Introduction to Time Series Classification (with Python Code)”.
