Senior Creative Engineer, Google
Machine Learning
101
Feel free to share this deck with others who are learning! Send me feedback here.
Dec 2017
2
This deck is old. I made it back in 2016 or so. I am amazed folk still reference it to this day - thank you. However for my new (and free) course on Web AI that covers all this and much more, please head to Google Developers YouTube channel:
New course available in 2023
3
The culmination of almost 2 years of head banging, so you don’t have to. The one stop shop to get answers to common questions you may have around Machine Learning (ML from this point forwards):
What is this deck?
4
What I need from you?
All I need is your undivided attention. Seriously. If you can read, you should be able to understand everything in this deck, but only without interruption.
5
Stop!
Continue only once you have locked yourself away in a room far far away from noise.
Really.
6
Green vs Blue slides!
BLUE slides are for those who are curious and want to go a bit deeper.
These slides may need you to search a few terms if not familiar and often contain reading material in the notes section below the slide.
GREEN slides are for EVERYONE. Do not skip these.
Seriously. Don’t skip them. I am watching you :-p
Any links are shown with an underline. You can click them to read more details.
7
INSPIRATION
Jason Mayes / 2017
What’s Artificial Intelligence, Machine Learning, and Deep Learning?
7
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
8
Artificial Intelligence
Artificial Intelligence (AI) is the science of making things smart. Can be defined as:
“Human intelligence exhibited by machines”
A broad term for getting computers to perform human tasks. The scope of AI is disputed and constantly changing over time. Let’s go deeper...
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
9
AI: Where we are at
The systems implemented today are a form of narrow AI - a system that can do just one (or a few) defined things as well or better than humans. Like recognizing objects / gestures we trained* it to learn.���* Needs code written by human to create system capable of learning that thing
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
10
AI: Common Use Cases
Image from NVIDIA research video here.
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
11
Machine Learning
Machine Learning (ML) can be defined generally as:
“An approach to achieve artificial intelligence through systems that can learn from experience to find patterns in a set of data”
ML involves teaching a computer to recognize patterns by example, rather than programming it with specific rules. These patterns can be found within data. In other words, ML is about creating algorithms (or a set of rules) that learn complex functions (or patterns) from data and make predictions on it - a form of “narrow AI”
These systems can be reused eg to recognize other objects with new data. Same code. Very powerful!
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
12
Learning by example
ML is all about predicting stuff essentially. It is intelligent because:�
The beauty of ML is that it learns by itself from the data passed to it.
So even though ML right now typically does one thing well, such as object recognition, that same ML system can then be re-used to learn any future objects too (given enough example data) without re-writing the code.
This is powerful.
Write a computer program �with explicit rules to follow
if email contains V!agrå
then mark is-spam;
if email contains …
if email contains …
Write a computer program �to learn from examples
try to classify some emails;
change self to reduce errors;
repeat;
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
13
A different way of doing things
Traditional Programming
Machine Learning Programs
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
14
Deep Learning
Deep Learning (DL from here on) can be defined generally as:
“A technique for implementing Machine Learning”
�One such DL technique is a concept known as deep neural networks (DNNs) which you may have heard of. If not, no worries, all will be explained.
Essentially DL in the context of DNNs is where the code structures you write are arranged in layers that loosely mimic the human brain, learning patterns of patterns.
These concepts go
back quite some time,
the idea of AI goes back
to the 1950s. In the 1980s �ML begins to grow in �Popularity. Around 2010 �DL drives large progress �towards achieving narrow �AI systems. You can see �how these 3 terms are linked - subsets of each other essentially. Deep learning drives machine learning, which then ultimately can enable artificial intelligence.
Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
15
Summary
Artificial Intelligence
Machine Learning
Deep Learning
16
INSPIRATION
Jason Mayes / 2017
How to choose data to train ML systems?
16
Collecting data for ML input
17
Features / Attributes
Features (aka attributes) are used to train an ML system. They are the properties of the things you are trying to learn about.
Colour: Orange
Weight: 340g
Collecting data for ML input
18
Features / Attributes
Taking fruit as an example. Features of a fruit might be weight and colour. 2 features, would mean there are 2 dimensions. A 2D system may be plotted on a graph if features are represented in a numerical way.
In the plot on the right, the ML system can learn to split the data up with a line to separate apples from oranges. This can now be used to make future classifications when we plot new points the system has not seen (anything above is orange, below is apple)
Weight
Colour
Plotting apples (red and green dots) and oranges (orange dots) weight and colour
Collecting data for ML input
19
Features / Attributes
Choosing useful input features can have big impact on the quality of the ML system. Some features may not be useful enough to separate the data points.
In this example we take bad features of fruits (ripeness and seed count) that do not allow us to learn any distinguishing factors for the fruit.
It takes practice and thought to figure out what are the best features to use as they are not always as clear as this trivial example.
Ripeness
Number of seeds
Plotting apple and oranges seed count and ripeness
Collecting data for ML input
20
Non trivial data
Our fruit example used 2 dimensions. If you
needed 3 dimensions to separate the data
meaningfully we could plot it in a 3D
chart, and separate the clusters of points with a
plane that divides the two sets of data as shown�
Note: Most ML problems have even higher �dimensionality! Like 20D or even millions in the �case of image recognition (each pixel is a feature). �Whilst we find it hard to visualise anything higher �than 3 dimensions, computers and ML can, the �principles are the same.
Image from Vision Dummy: https://goo.gl/u8w2Zi
Collecting data for ML input
21
A note on dimensionality
Adding more dimensions can often help
separate out the data points, allowing the
ML system to split the data up into groupings
for classification as shown on the right.
Too many dimensions however can lead to overfitting which means it knows the example data perfectly, but if you give it something new it has not seen before, it won’t be able to generalise well enough to get new things correct. That would be bad.
1D
Hard
To split
2D
Better separation
3D
Easy separation
Images from Vision Dummy: https://goo.gl/u8w2Zi
Collecting data for ML input
22
Data hunting
Once you know features to use, the biggest challenge is finding enough unbiased training data for all of those features in a format that can be fed into an ML system to learn from (depending on the type of ML algorithm being used)
Imagine you wanted to recognize cats. You may need 10,000 example images of cats if you want a chance of getting decent results.
Data is not always images, could be tables of data with multiple features, text, sensor recordings, sound samples, and more depending what it is you want to classify.
Collecting data for ML input
23
An ML system can not predict
stuff it does not know about
Let’s say you teach an ML system about animals like this:
Number of Legs, Colour, Weight, Animal:
4, Black, 10KG, Dog
2, Orange, 5KG, Chicken
If you now present it with a Cow: 4 legs, black, and 200KG it would predict “Dog”. This is because it only knows about dogs and chickens and this was the closest match.
Video break
24
Watch this
video
Now, take a break and watch this great overview to put some of what you learnt into a better context.
25
INSPIRATION
Jason Mayes / 2017
How ML systems are trained
(Learning Style)
25
Training ML
26
Supervised Learning
Where the ML program is provided with training data that is labeled. You tell the system how to categorize the example data. For example:
Colour, Weight, Label
Red, 200g, Apple
Orange, 300g, Orange
Green, 150g, Apple
...
Given the 2 inputs (colour and weight) I am then telling the system what the expected output label is in each case (orange or apple) - supervision. The ML system must then use this data to predict future unseen inputs. Currently this is the most studied area.
Training ML
27
Unsupervised Learning
The machine must learn from an
unlabelled data set.
Imagine we had a bunch of points on a
graph representing 3 different things.
Machine must realise by itself there are 3 �distinct clusters and categorize them as �such.
This is tricky, as the number of clusters may
not be known in advance, so it has to take a
best guess. Also, sometimes the clusters are not as clear as the ones shown here.
Y
X
Training ML
28
Reinforcement Learning
Learning by trial-and-error through reward or punishment.
The program learns by playing the game millions of times. We reward the program when it makes a good move. This strengthens the connections to make moves like it did. When it loses we give no reward (or negative reward).
Over time it learns to maximise reward without the human explicitly telling the rules. It can lead to better than human performance when it finds plays that no one ever thought of doing before...
29
INSPIRATION
Jason Mayes / 2017
But how does it all work?
29
How does ML work?
30
Many ways
There are many ways in which an ML system can try and learn patterns to classify data presented to it. We shall go through some very simple examples (real ML systems often use more than 2 dimensions) on the next few slides to give you some idea / insight of what is going on behind the scenes.
These examples are chosen to help aid understanding. In reality for cutting edge research there is much more going on (sometimes even combining multiple methods and more complex concepts beyond the scope of this deck), but the basic principles of finding patterns through some mathematical method still apply.
How does ML work?
31
Example 1
Remember in high school when you
had to plot points on a X,Y graph
and then draw a line of best fit? This
is like a very simple ML system. �
In this example, with that line you
drew, you can predict given X, what
Y value will probably be - even for
unseen examples by extending
the line further. This is a form of “regression”.
ML programs calculate these lines by themselves essentially!
How does ML work?
32
Example 2
In this example we use a line to divide the two
clusters. By doing this we can predict future
examples by saying anything above the line is
likely to be of the red class, and anything
below the line is of the blue class.
As you can see there may be a few outliers,
but most of the time it would get the prediction correct.
We have used a straight line here, but it could be
a more complex one such as a quadratic.
Y
X
outlier
How does ML work?
33
Example 3
Here, instead of lines, we see ML being used to learn how to classify two clusters of points by sampling x known points close by . The coloured background shows how it would predict future points it has not seen before in areas where there is no existing point. E.g. if we created an observation at bottom right, it would say it belongs to blue data as the 3 closest points are mostly blue ones. This type of classification is known as k-nearest neighbour.
How does ML work?
34
Neural Networks
Our brain consists of 86 billion or so interconnected “neurons”. Each neuron
responds to certain stimuli and passes output to another.
For example there may be a bunch of them dedicated to recognizing cats
(some for fur, eyes, whiskers etc), each having a different weighting (based on how important that feature is) to the overall contribution. If all of those fire, your brain tells you that you saw a cat.
In ML, artificial neural networks (loosely modeled on the brain) are used to calculate probabilities for features they are trained to look out for.
3�
(Bias)
A neuron simply has a bunch of weighted inputs (take the input number and multiply by the weight) that are summed together. A bias is then added to this total. The weights and bias are determined when we train the system. If final result is greater than a threshold, it activates, providing an output. Strength of output depends on the activation function chosen. The output is then fed into other neurons and the process repeats.
Artificial neuron (perceptron)
3
1
7
2
1
0
8
2
1
3
1
1
4
1
Weights
42
Activation function
35
Output to other neurons
Typically these input numbers are between 0 and 1 but for ease of visualisation we will use whole numbers
Inputs
(numbers from your sample data you want to learn from eg pixels in an image)
(3 * 2) + (1 * 1) + (7 * 3) +
(2 * 1) + (1 * 1) + (0 * 4) + (8 * 1)
+ 3 = 42
Input function
(multiplies inputs with weights and sums together)
Activation function (learn more)
.
.
.
36
This is a multi layered perceptron (or deep neural network) - one of the oldest forms of “neural nets” - conceptually goes back to the 60s! Each layer is fully connected to the next and data flows forwards only:
Multi layered perceptrons
28 px
28 px
Input Layer
(image pixel values)
2 Hidden Layers
Output Layer
(10 possible classifications 0 - 9 digits)
784 Inputs
7 Neurons
5 Neurons
10 Neurons
50 Connections
35 Connections
5488 Connections
0
1
2
3
4
5
6
7
8
9
Output classification is the most strongly activated output neuron
Fully connected layers
(each neuron in next layer is connected to every neuron in the previous - 5573 connections)
How does ML work?
37
Deep Neural Networks
A Deep Neural Network �(DNN) simply consists of �many “hidden layers” �between the input and �the output. Each layer
can learn from the one
before it from which �higher level learning can
take place. These hidden layers typically are of lower dimensionality so they can generalise better and not overfit to the input data. These middle layers in the system can learn features of features. For example bunches of “edges” can lead to “face parts” which lead to “faces” that the system can then recognize.
How does ML work?
38
Deep Neural
Networks
Watch this video for
a deeper dive.
Also check out this awesome deck that explains a bit more of the math behind them too (but simply!)
How does ML work?
39
Convolutional
Neural
Networks
Watch this video by Brandon Rohrer for
a deeper dive on CNNs for image recognition.
40
INSPIRATION
Jason Mayes / 2017
Types of ML output
40
Types of ML
41
Common ML Output types
Classification
One of n labels...
(cat, dog, human)
Regression
Predict numerical values�(e.g. price of house)
Clustering
Most similar other examples
(e.g. related products on Amazon)
Sequence Prediction
What comes next?
“If you want something done ____, do it yourself”
Types of ML output
42
Continuous Output
This means the output of the �ML system is a decimal number e.g. �8.3984 (I made that number up)
So given some input, you get some �numerical output. For example, if an �orange weighs 200g, its radius is 4.2 �inches. The output here is 4.2 and this �is what the ML system is trying to �predict. Linear regression for example tries to fit a straight line to the example data so you can then predict what some value will be (y) for a given input (x)
Types of ML output
43
Classification
Is all about determining the class (or label) for a set of inputs. I.e. the output is discrete. For example:
Inputs: hair, paws, whiskers
Output: cat
The output is a label, not a number like regression or probability estimation
Types of ML output
44
Probability Estimation
This means the output of the ML system is some decimal number between 0 and 1 indicating the probability we think that the given input is some desired output (0.751 == 75.1%)
�Example: An ML system to predict how many white bricks are in the jar on the right. If it output “9” with a probability of 54.3% we can then choose to do something useful based on this knowledge and confidence.
Types of ML output
45
Using output
Once you have an output in the form of regression, probability, or classification it is then down to the programmer to do something useful with that gained knowledge.
If we are 80% sure what we see is a cat, maybe we want to feed it. If we are 75% sure, maybe we will wait until we are more sure.
46
INSPIRATION
Jason Mayes / 2017
Types of ML algorithms�(how they really work, for the curious)
Optional: Skim read these blue slides if you are not interested in the details
46
How ML works: Types of algorithms
47
Regression
Regression is concerned with iteratively modeling the �relationship between variables using a measure of error
in the predictions made by the model.
Regression methods are a workhorse of statistics and
have been co-opted into statistical machine learning.
Regression is a process.
Straight lines of best fit are a form of linear regression.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
48
Instance-Based
Instance-based learning model is a decision problem
with instances or examples of training data that are
deemed important or required to the model.
Such methods typically build up a database of example
data and compare new data to the database using a
similarity measure in order to find the best match and
make a prediction. For this reason Instance-based methods�do not need any training, just example data. Focus is put on �the representation of the stored instances and similarity
measures used between instances.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
49
Decision Trees
Decision tree methods construct a model of
decisions made based on actual values of attributes
in the data. Decisions fork in tree structures until a
prediction decision is made for a given record.
Decision trees are trained on data for classification
and regression problems. Decision trees are often
fast and accurate and a big favorite in machine learning.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
50
Bayesian
Bayesian methods are those that explicitly apply
Bayes’ Theorem for problems such as classification
and regression. A good example is spam email detection which figures out if an email is spam based on various likelihoods of features being present (eg words used).
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
51
Clustering
Clustering, like regression, describes the class of
problem and the class of methods. Clustering methods
are typically organized by the modeling approaches
such as centroid-based and hierarchical.
All methods are concerned with using the inherent
structures in the data to best organize the data into
groups of maximum commonality to then categorize it.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
52
Association Rules
Association rule learning methods extract rules that
best explain observed relationships between
variables in data.
These rules can discover important and commercially
useful associations in large multidimensional datasets
that can be exploited by an organization.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
53
Artificial Neural Networks
Artificial Neural Networks are models that are
inspired by the structure and/or function of biological
neural networks in the brain.
They are a class of pattern matching that are
commonly used for regression and classification
problems but are really an enormous subfield
comprised of hundreds of algorithms and variations
for all manner of problem types.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
54
Deep Learning
Deep Learning methods are a modern update to
Artificial Neural Networks that exploit abundant
cheap computation.
They are concerned with building much larger
and more complex neural networks and, many methods �are concerned with semi-supervised learning problems �where large data sets contain very little labeled data.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
55
Dimensionality Reduction
Like clustering methods, dimensionality reduction
seeks and exploits the inherent structure in the data. �In this case it’s in an unsupervised manner or order �to summarize or describe data using less information. ��This can be useful to visualize dimensional data or to �simplify data which can then be used in a supervised �learning method. Many of these methods can be �adapted for use in classification and regression.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
How ML works: Types of algorithms
56
Ensemble
Ensemble methods are models composed of
multiple weaker models that are independently
trained and whose predictions are combined in
some way to make the overall prediction.
Effort goes into determining what types of weak �Learners to combine and the ways in which to �combine them. This is a very powerful class of
techniques and as such is very popular.
Example:�
Description adapted from http://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
57
INSPIRATION
Jason Mayes / 2017
How is Google using ML?
57
How is Google using ML?
58
Google Photos
Smart search.
Gmail & Inbox
Smart replies.
Teaching robots hand-eye coordination
Where Google uses it
How is Google using ML?
59
ML improves consumer experiences
Search
Search ranking
Speech recognition
Android
Keyboard & speech input
Gmail
Smart reply
Spam classification
Drive
Intelligence in Apps
Chrome
Search by image
Assistant
Smart connections
across products
Maps
Parsing local search
Translate
Text, graphic and speech translation
Cardboard
Smart stitching
Photos
Photos search
60
How is Google using ML?
Autodraw
Get millions of people to draw doodles of objects and teach an ML system to learn how to draw for itself! Really awesome use of ML that makes your bad sketches look awesome :-)
61
Google Products / APIs / Creative Thought Starters
Soli
Soli is a small chip you can embed into devices which uses radar like features to detect gestures so you can control the device. No need to look at screen anymore.
61
62
Google Products / APIs / Creative Thought Starters
Pixel Buds
Pixel Buds have gesture control, interface with the Google Assistant, and most impressively, enable real-time language translation. They will literally translate spoken language as someone is talking to you.
62
Google Lens
INSPIRATION: What are people making?
Announced at Google IO 2017, it makes your smartphone camera contextually smart. Hold it up to a restaurant, recognize it, bring in menu, reviews, photos, before going in!
63
64
INSPIRATION
Jason Mayes / 2017
How are others using ML?
64
65
INSPIRATION: What are people making?
Cutting Edge Research
These examples may not be useable at scale just yet but a sign of things to come...
66
Robotics
INSPIRATION: What are people making?
A whole tonne of stuff going on in robotics right now. Just take a look at Boston Dynamics YT channel for some mind blowing research, aided by various ML techniques.
66
67
INSPIRATION: What are people making?
Style Transfer
Take 2 images, use a neural network to sample the content from one, and the style from the other. Ask it to output the result. This is not Photoshop, this is ML learning how to draw in the style of your favourite artist for any photo you give it!
68
INSPIRATION: What are people making?
Video Generation
What you see here are videos “dreamt up” by a neural network. It studied various videos of beaches, train stations, etc and then given an image it has never seen before, it will imagine what might be happening in that scene and turn it into a video. Amazing research in its very early stages, but one to watch for the future. Nothing you see here is “real” or actually exists.
69
Skip Thought Vectors
INSPIRATION: What are people making?
70
INSPIRATION: What are people making?
Understand Images
“Cameras are your entry point to understand the world”
Thing Translator
INSPIRATION: What are people making?
Recognize a “thing”, get the textual word for it, translate that word into another language, and then display it. You can make this in just a few hours using Google Cloud JS APIs!
71
Treasure Hunt
A treasure hunt based on photos you take.
You’re given a list of objects to ‘capture’, and every time you snap a photo of one, you get points.
INSPIRATION: What are people making?
72
Dragon Spotting
INSPIRATION: What are people making?
Use pre-trained machine learning APIs (Google Cloud Vision) that recognize object, places and people to trigger creative AR experience.
73
74
INSPIRATION: What are people making?
Predict complexity
“Identify patterns and behaviors at scale”
Alpha Go
INSPIRATION: What are people making?
AlphaGo by Google: example of using AI to solve incredibly complex challenges such as playing Go. Go has millions of possible combinations and was long thought to be too complicated for machines to beat a human at.
75
76
INSPIRATION: What are people making?
Optimize user experiences
“Design for context and personalization”
Spotify
INSPIRATION: What are people making?
Spotify uses ML to customize content for its users. Drives stronger customer loyalty as suggestions are better tailored leading to discovery and more playback.
77
Project Muze
INSPIRATION: What are people making?
Google ZOO and Zalando launch Project Muze, a machine-learning experiment for 3D fashion design using neural networks to generate new ones based on preferences.
78
79
INSPIRATION: What are people making?
Converse with users
“Natural interactions in our everyday lives”
Disney: Book ears
INSPIRATION: What are people making?
Augment the Disney book experience by playing sounds for specific voice cues using the Google Speech API.
80
Westworld
INSPIRATION: What are people making?
Created a digital version of Aeden, a character in Westworld to whom you could talk with in a natural way. Thousands of unique conversations, and even won an Emmy award.
81
82
INSPIRATION
Jason Mayes / 2017
Google products and APIs that enable ML
82
83
Machine Learning overview
Google Products / APIs / Creative Thought Starters
Custom models
Pre-trained models
Retrainable models
83
Update: New products added!
84
Cloud Vision API
Google Products / APIs / Creative Thought Starters
Gives machines a human understanding of the world. Understands objects, landmarks, logos, and faces (even emotion!) Based on Google’s knowledge of the world.
84
Logo Detection
Google Products / APIs / Creative Thought Starters
86
Cloud Vision API
Always improving recognition database, but
TEST FIRST.
Geolocation
New York City
/m/02nd_
Knowledge
Graph ID!
Photo moderation
95% sure there’s a dog
Mood detector
97% angry
99% happy
99% sad :(
Google Products / APIs / Creative Thought Starters
86
Identify entities and label by types such as person, organization, location, events, products and media.
Enables you to easily analyze text in multiple languages including English, Spanish and Japanese.
Extract tokens and sentences, identify parts of speech (PoS) and create dependency parse trees for each sentence.
Syntax analysis
Entity Recognition
Multi-Language Support
Understand the overall sentiment expressed in a block of text.
Sentiment Analysis
Derive insights from unstructured text using Google machine learning.
87
Natural Language API
Google Products / APIs / Creative Thought Starters
TRY THE API
87
Powered by deep learning neural networking to power your applications..
No need for signal processing or noise cancellation before calling API. Can handle noisy audio from a variety of environments.
Noise Robustness
Can provide context hints for improved accuracy. Especially useful for device and app use cases.
Word Hints
Speech Recognition
Recognizes over 80 languages & variants. Can also filter inappropriate content in text results
Over 80 languages
Can stream text results, returning partial recognition results as they become available. Can also be run on buffered or archived audio files.
Real-time results
88
Speech API
Google Products / APIs / Creative Thought Starters
TRY THE API
88
Supports more than 100 languages and thousands of language pairs.
Behind the scenes, Translation API is learning from logs analysis and human translation examples. Existing language pairs improve and new language pairs come online at no additional cost.
Sometimes you don’t know your source text language in advance. Can automatically identify languages with high accuracy.
Automatic language detection
Text Translation
Continuous Updates
89
Translation API
Google Products / APIs / Creative Thought Starters
TRY THE API
Detect entities within the video, such as "dog", "flower" or "car".
You can now search your video catalog the same way you search text documents..
Extract actionable insights from video files without requiring any machine learning or computer vision knowledge.
Enable Video Search
Label Detection
Insights From Videos
90
Video Intelligence API
Google Products / APIs / Creative Thought Starters
90
91
Let’s try them out!
Google Products / APIs / Creative Thought Starters
91
Machine Learning Platform
TensorFlow
Not limited to ‘Deepdream’ art.
Runs locally on CPUs and GPUs.
TensorFlow in the cloud (faster, faster!).
Custom models
Google Products / APIs / Creative Thought Starters
92
TensorFlow
Most well known for this:
Or this:
Can be used to train just about anything
Google Products / APIs / Creative Thought Starters
93
TensorFlow
94
WHAT IS IT?�Open source machine learning library that can be trained to recognize images, sounds, text, anything. Must be trained from scratch, like an extremely fast learning child. Fast, flexible, and production-ready on all major platforms. Try Cloud TPU to train faster.
WHAT DOES IT MEAN CREATIVELY?�• Analyse user sentiment across all your social platforms
• Recognize an object and trigger a creative experience
AVAILABILITY
Now, for any website or app. Can pair with Cloud ML Engine to deploy ML models with serverless, fully managed hosting that responds in real time with high availability.
Google Products / APIs / Creative Thought Starters
94
TensorFlow Research
95
Google Products / APIs / Creative Thought Starters
Neural Audio Synthesis, Music Generation�Magenta.tensorflow.org��Sequence to Sequence�research.googleblog.com/2017/04/introducing-tf-seq2seq-open-source.html
Parsey McParseface�research.googleblog.com/2016/05/announcing-syntaxnet-worlds-most.html
Show and Tell�research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html
�
Improving Inception�research.googleblog.com/2016/08/improving-inception-and-image.html
Parsey Saurus�research.googleblog.com/2017/03/an-upgrade-to-syntaxnet-new-models-and.html
Parsey’s Cousins�research.googleblog.com/2016/08/meet-parseys-cousins-syntax-for-40.html
Supercharging Style Transfer�research.googleblog.com/2016/10/supercharging-style-transfer.html
Summarization �research.googleblog.com/2016/08/text-summarization-with-tensorflow.html
95
96
Google Assistant
Google Products / APIs / Creative Thought Starters
96
Dialog Flow
BUILD YOUR OWN BOT
�WHAT IS IT ?�A one-stop-shop tool allowing you to easily create a smart bot that work across all platforms. AKA API.AI in the past.
WHAT DOES IT MEAN CREATIVELY ?�• Integrate your services for straight-forward requests
• Connect your own bots and assistant to the Google Assistant.
AVAILABILITY
Now on web or mobile.
97
Google Products / APIs / Creative Thought Starters
97
98
SUMMARY: General geeky thoughts
ML / Data Science is a mixture of many disciplines
��This image by Nisarg Dave shows the complexity involved when building these systems. To train and make the underlying code you need to pull from many verticals of knowledge to create the amazing examples you have seen in this deck.
It takes time, research, and money to build great ML systems that work well and scale. It is not a trivial matter, but as we get more competent at it, we shall see an exponential rise of ML in our everyday lives, products, and services.
Closing thoughts
98
99
SUMMARY: General geeky thoughts
Want to learn more? I recommend the following to get started:�
What next?
99
Learn more
Get started fast!
Recommended Reading
Learning TensorFlow.js (O’Reilly)
oreilly.com/library/view/learning-tensorflowjs/9781492090786
Deep Learning with JavaScript (Manning)
manning.com/books/deep-learning-with-javascript
Website / API: tensorflow.org/js
Models: tensorflow.org/js/models
Github Code: github.com/tensorflow/tfjs
Forum: discuss.tensorflow.org
Codepen: codepen.io/topic/tensorflow
Glitch: glitch.com/@TensorFlowJS
Inspiration: goo.gle/made-with-tfjs
People to follow
SUMMARY: People to follow
101
THANK YOU!
This deck started as a mind dump for my own personal sanity but has evolved over time to turn it into a one stop fountain of knowledge for ML for all.
A quick shout out to others who have contributed in some shape or form from tech reviews to suggestions for new slides:�
SUMMARY: General geeky thoughts
102
103
Enjoyed this deck / questions?
Connect online if you want access to my stream of daily discoveries of awesome tech, tutorials, and inspiration.
If you enjoyed the deck, or have feedback, also let me know!
Jason Mayes / www.jasonmayes.com / 2017