IA, AI & Voice:
How can we make voice interactions easier?
a talk by
Lade Tawak
The cognitive considerations for voice interfaces
miller’s law = the span of immediate memory is limited to around 7 pieces of information
Avoid lengthy responses, chunk responses,
confirm attention and understanding
LADE TAWAK
Low digital literacy +
no/low visual context
= fear and anxiety
CONSIDERATION 1
I press something and something happens
✅
❓
I’m talking to someone?
No direct manipulation
LADE TAWAK
“The main bottleneck of using a voice assistant system is that the user cannot know all the commands.” — Park, Park, Song (2020)
What does this do?
What do I need to make a marinade?
You need oil, salt…
(still listing)
Okay
(affirming)
…
…
Conversational cues
How do we come in?
Set context, explain to the user what the agent can do
Low digital literacy
Proper data structures which allow AI properly parse user requests to map to attributes
Low digital literacy
How do we come in?
Create Dialog Flows and maps that include, clarification paths, error messages, and error recovery
Low digital literacy
How do we come in?
That’s card activity, is that correct?
How can I help you? You can say report a stolen card, card activity, insurance, or I need help with something else
Card activity
Yes
What card activity can I help with? Say: balance, recent transactions, dispute a charge or I need help with something else.
…
LADE TAWAK
Communication styles and language
CONSIDERATION 2
Identifying user intent
Book me an uber to the airport
I’d like a pizza
Order a medium triple decker bbq chicken pizza
I need to go pick up Tunta and then go to church
Context and content
Cultural cognition
When I say stove, what do you picture?
Cultural cognition
When I say cooker, what do you picture?
Two chatbots to teach about consent in the context of sexual activities. One LC, one HC. (Fromet, 2021)
Low context, high context
Can the agent match my interaction style
| HC Culture Participants | LC Culture Participants |
Low context interaction | Neutral | Slight preference |
High context interaction | Strong preference | Neutral |
Call Onyinye
I didn’t get that
Call Onyinye
…
Try again
Accent recognition
How to make stock
I’m sorry I can only teach how to make a marinade or curry or stock
I want to make stock
Okay, we can resume whenever you’re ready
What would you like to learn
Accent recognition
LADE TAWAK
“Having to adapt our way of speaking to interact with speech recognition technologies is a familiar experience for people whose first language is not English or who do not have conventionally American-sounding names. I have even stopped using Siri because of it” — Claudia Lopez Lloreda
Multilingual behaviour
Can the agent understand me if I speak multiple languages at the same time
Ensure data sets represent a wide range of accents, dialects, and cultural references.
Communication styles
How do we come in?
Ensure taxonomies, metadata, and data labelling are free from biases, but also consider cultural contexts
Communication styles
How do we come in?
Customise agent personality based on location and other signals
Communication styles
How do we come in?
Allow personalisation and customization
Communication styles
How do we come in?
In creating DialogFlows, include multiple sample utterances that cover diverse cultural approaches
Communication styles
How do we come in?
Communication styles
This is not new
Communication styles
Different food menu items based on location
This is not new
LADE TAWAK
Environmental context
CONSIDERATION 3
What do I need to make a marinade?
You need oil, soy sauce, garlic, pepper, and lemon juice
I don’t have lemon juice
…..
Should I place an order for you?
Unavailable in my country
Create flexible interaction structures such as branching choices and decision points that can respond based on predetermined attributes
Communication styles
How do we come in?
Customize what agent can do and not do based on what is available to the user in their context (e.g. based on location)
Communication styles
How do we come in?
Map user journeys and understand the surrounding context outside of your product
Communication styles
How do we come in?
Wrapping up
…understand user (intent)
…retrieve relevant information
…generate the right output
HELP THE AGENT
…customizing agent based on sociocultural context
…using inclusive taxonomies and data labelling
…designing multiple branching choices in the dialogflow
BY…
make voice interactions more inclusive, usable, and accessible
TO…
Lade Tawak
THANK YOU