Voice
Augmented
eXperiences
for Apps
How does voice work?
Trees for the forest.
NLP
ASR
Speech to text
Text to Intention
TTS
The chinese room problem
How does my computer understand me?
史蒂夫
史蒂夫
何人
你是我
帅哥
有是不
你我们
你我们
*The Chinese is gibberish
The chinese ML room problem
How does my computer understand me?
史蒂夫
史蒂夫
何人
你是我
帅哥
有是不
你我们
你我们
0.25
0.75
0.25
*The Chinese is gibberish
How does my computer understand me?
The English ML room problem
Book Flight
Here some flights
0.25
0.75
Flight List
Only Morning
Filter
Book Flight
All that’s cool, do I need to worry about this?
Slightly, but more on that later...
Why do you want to voicify your app?
Accessibility for the disabled
Enable users get things done faster
Lack of screen real estate
Accessibility for the elderly
Agentive
Design
Increase throughput through familiarity
Understand true intent.
Accessibility across languages
The Bottom Line
Voice follows the Visual?
The Visual follows the Voice?
The Visual works in Tandem with Voice?
Accessibility
Avataars
GUI elements control
The Bottom Line
Design to get the best of both Voice + Touch
Voice:
Can I order a Margherita Pizza.
Touch:
Scroll through the various option find Margarita and click it
I’d like to just repeat my order.
Search for repeat order button and click
Can you tell me all the pizza’s available
Look through the list and browse
The Basics
Intent
Entities
Prompts
Utterances
Intent
The action the user wants to perform.
Maps to an executable function
Book a ticket
List the flights
Filter for the morning ones.
Cancel my order
Go Back
How many bags are allowed?
Repeat that
Entities
Parameters to be collected if the function is to be fulfilled.
Book a ticket from Bangalore to Delhi for the 21st
Entity: Source
Entity Type: City
Entity: Destination
Entity Type: City
Entity: Dep Date
Entity Type: Date
Utterances
Examples that help distinguish between Intents given the spoken sentence
Book a ticket
Can I book a ticket
Are there any tickets available from Bangalore to Delhi
I need a ticket from Bangalore to Delhi
I want to book a ticket from Bangalore to Delhi
Any reservations from Bangalore to Delhi
Tickets for two from Bangalore to Delhi
Prompts
Help collect required entities, by asking the user for them.
Date: For which day should I book the onward journey?
Return: Is it one way or return?
Dest: Where do you want to fly to?
Source: Where are you flying from?
Number: How many tickets do you want to book?
Confirmations
Feedback to the user.
Let them know all is well, or not.
Can be a prompt:
Sure, would you like to continue?
Can be a positive statement:
Your ticket for the 21st from Mumbai to Bangalore for one person has been booked successfully.
Can be a negative statement:
I am sorry there are no flights available that day from Mumbai to Bangalore.
The Basics
not so
^
Conversations
String Dialogues Together
Make them human like
I’d like to book a flight
Alright, for when?
Umm, the 21st.
Sure, from where?
From Mumbai to Bangalore
Alright, here are some flights...
(Scrolling a bit)
Umm, show me only the non stop flights.
Alright, filtered out the non stop flights.
(Scrolling a bit more)
Or you know what scratch that just show me all of them.
Sure, removing the number of stops filter.
Conversations
Okay, show me flights to Chennai
You want to see flights from Mumbai to Chennai for the 21st?
No No No, for the 22nd.
Alright, showing flights from Mumbai to Chennai for the 22nd.
(Scrolls through and touches a flight listing)
Okay, I selected my flight, I want to checkout.
Sure, selecting the Voice Airways flight V9 2910, from Mumbai to Chennai for one person for the 21st.
Would you like to pre book some of our award winning in flight refreshments?
String Dialogues Together
Make them human like
Context
The glue between conversation dialogues
Okay, show me flights to Chennai
You want to see flights from Mumbai to Chennai for the 21st?
Intent: Book Flight
Entities: {
Source_city: Mumbai
Dest_city: Chennai
Date: 21-12-2018
Return: False
}
Multi- modality
Alright, showing flights from Mumbai to Chennai for the 22nd.
(Scrolls through and touches a flight listing)
Okay, I selected my flight, I want to checkout.
Switch between Touch and Speech Seamlessly
Select the 2nd one and go to checkout.
Error Handling
Personality
*Stolen Used without permission from the New Yorker
Privacy
Thank You
🍺 🍻 🍸 🎉 🎊
42@slanglabs.in