How To Create Magical Data Products Using Sequence to Sequence Models
Hamel Husain
About Me
Why Am I Giving This Talk?
Prerequisites For Seq2Seq
Seq2Seq often reserved for advanced topics
Andrew Ng’s deeplearning.ai
Jeremy Howard’s Fast.AI
But …. People give up before reaching this point. Only if they knew what they were missing!
Are you sure about math?
You don’t need lots of math right away. Think of different type of layers as APIs.
API: expects specific input, give you an output.
This way of thinking can help reduce fear of deep learning. May not work for everyone, but worked for me.
[‘this’, ‘is’, ‘a’, ‘sequence’]
[3, 2, 1, 5]
RNN
[0.1, 0.4]
Represent data numerically
*RNN expects a sequence as input
Returns latent features of sequence.
*There is an embedding step I skipped. Just giving you high-level intuition.
RNNs: APIs For Extracting Features From Sequences
Keras documentation
CNNs: APIs For Extracting Features From Spatial Data
CNN
Represent data numerically
Extracts latent features.
CNN expects data as spatial input. 1D, 2D, 3D, etc.
Keras documentation
What was the point of the last two slides?
Cannot teach your RNNs or CNNs in this talk.
Give you a way of thinking as you approach this subject so you can reduce fear.
Allow you to keep moving if you get stuck. Its ok to focus on high level intuition at first.
Seq2Seq Models : Machine Translation
This is the magical part! Extracts features from the input sequence.
Thinking of layers as APIs…. I really meant it.
Don’t try to read these. It’s just meant to show that I mentally think of layers as APIs, and it works for me. This is what I did before I got started on this project.
Github Issue Summarization
You can build/try it yourself! Including an end to end example.
More Magic
More Magic
All materials related to this presentation available here:
hamel.io