the digital / physical divide

Our intimate web : sensors, speech, and a physical digital future

This presentation can be seen at:

April 2013.

Tom Uglow, Creative Lab, Google


Isn't this a beautiful place to start? Artist is Alejandro Guijarro

It is a blackboard from an advanced quantum mechanics class.

If you're into quantum mechanics you'll know that certainty and prediction are tenuous ideas.

So, obviously, the parallel with what is about to follow is that I have no idea about the future.

At least no more than a proton does.

3 :: WORK

I work at Google for a group called the Creative Lab.

I work with culture, and what we can do with artists and cultural organizations around the world to find new forms in their practice using the internet and digital tools.

Not digitizing old culture - but creating new culture that can only exist using the internet.

what does that mean?

I got to work with famous orchestras and musicians for our YouTube Symphony project,

with curators at the Guggenheim showcasing video art,

I went to Sundance to launch a film we made with Ridley Scott called Life in a Day.

I was lucky enough to work with the amazing team behind the Google Art Project,

and we've worked with the Opera House here in Sydney on festivals like VIVID and Graphic.

At the moment I am working with the Royal Shakespeare Company on a play about some fairies; a new way of teaching history, a book about maps…you know, stuff.

We get to play about at the intersection between the arts and digital .

And, frankly, sometimes that looks a little clumsy -

This is a screenshot from a video by BERG about the future and how information enhances life.


Life filled with information that flows around you like magic, that is personal & intimate.

I'm going to talk about what information looks like:: Streams and Tools and Signals.

AND how we find it :: sensors, speech, and the quantified self

AND how we consume it :: wearable tech, intimate technology,  and enchanted objects.

And then I want to talk about what that might mean for you

[Image from CyborgCamp]


How many of you have your own web page?

How about a phone App? Me neither.

Which seems a shame because that was one of the things I liked the most about the old internet.

The barriers to entry were incredibly low

Who thinks the hyperlink. the click is a great invention?

It is the simplest, most elegant, universally comprehensible  way to initiate an action.

The internet, my internet, was a series of pages on computers connected by links that you clicked. Sounds old fashioned - but incredibly successful in organizing the worlds information. OLD SCHOOL.

[image is a detail of evan roth's angry birds all levels]

5 :: SWIPE

What is the new click?

"the image is a detail of evan roth's angry birds all levels"

THE CLICK has been overtaken by a huge variety of interactions.

These split into ACTIONS & CONTEXTS

ACTIONS: The swipe, the scan, the press, speech, shake, bump, gesture, or better still - 'it just knows'? etc

CONTEXTS:  Now try to think of all the other things that could trigger an action.

If the device knows a few details that you choose to share, like who you are, where you are going, where you came from - if it has some context.

Then it could use your location, or the temperature, PROXIMITY, sounds, velocity, weather, signal strength, density, face recognition, emotion recognition, key words, the time, or any combination - all these become the equivalent of a trigger, a "click".

And they can work in combination with data from your calendar, social networks, your browser history, your preferences. There are gazillions of ways to click.

Nice quote :: David Holtz the CEO of Leap Motion who said at SXSW, 'This world [of natural gestural interfaces] perceives you in new ways. It is very much a new reality, when you reach out for an object using Leap Motion, it comes to you – it's like being a Jedi."

And he is talking about one of the 8 ACTIONS listed.

WHat it says on the slide.

Next up - WTF does that look like...? Streams, Tools & Signals.

With cloud based solutions, we don't just expect machines to be personal when we log in,

we expect the different screens or objects to work together in smart ways.

We are always logged in, devices are always synched, and increasingly, they play well together.

7:: Streams

At the moment data seems to exist in three forms: Streams and Tools and Signals

The first are streams:

We all understand streams - they existed before internet, before TV or radio. Time is a stream.

Fb Timelines are a stream. Feeds are a stream, content streams, youtube streams, playlists, news and entertainment streams

Streams  move past you like they are fixed in time. Like standing in a river. You can go back and find it if need be but generally it moves past you, endlessly -  it is only of that moment.

Hmm, let's not get into finite streams (books) and infinite streams (facebook) and finite streams within infinite streams (looping news footage on 24hour tv stations).

While we’re here - let’s also acknowledge STATIC content which doesn’t really fit into this model - but then this whole talk is not an academic paper; nor comprehensive; it’s just a useful set of metaphors to explain an idea. deal with it.

[Image via banoootah_qtr]

8:: Tools

Tools are like, well, tools. Just like we have maps or a camera or a notebook - we have maps, and camera's and notebooks on our digital devices.

Anything that let's you do something functional. Games are tools. They distract you. You pick them up, they distract you, you put them down. They're just like knitting, except you can't knit on an iPhone.

9:: Signals

The most interesting bit is SIGNALS. Signals are knowledge; information presented to you at the right time or place to either help you make a decision, or to make a decision for you.

Signals exist because of who you are, where you are, what time it is and what data you have shared.

NUDGEs... & we need to be careful. Get them wrong and you're a pushy friend or worse.

Information that is specific to you at that moment. Like knowing you're about to miss the bus you take. That information is only useful to you - and that's what contextual data means.

Thinking that way changes the world you live in, so we need to change the way we think.


SPEECH was one of the 8 ACTIONS

With speech and Glass the internet becomes an Invisible internet. sans interface.

"OK Glass" is a command that will change how you think about computing.

Speech is also incredibly dynamic. I am fascinated to see how it changes us.

For example the brain, the mouth and the hand all move at different speeds.

In fact if you think about body language and language - the two oldest form of communications that exist then you see that actually speech & gesture aren’t progress at all - they’re returning to nature. To a natural state. Just with computers and, temporarily, screens in-between.



Ultimately the future is a vision of synthesizing known information with captured data from your environment;

Reliant on SENSORS.

The Internet of things - objects that can be uniquely identified and that supply information to power your signals


Image is from a company selling a simple sensor that outputs to an app.

Did you know a webcam can track your pulse by identifying your face, locating the forehead and then analysing the optical intensity of the green channel. Cool eh? No?!

[Image via]

12:: The Quantified SELF - is self tracking (via digital tools) and self-knowledge through numbers - a hot topic at the moment.

Not just physical health and activity, but technological, personal, intimate - from the internet of things, to mental health tracking, location data and identity //

It is truly life-tracking, like writing a diary at a microscopic level.

Tictract aggregates:

"Email, blood pressure, supermarket foods... pretty much anything you can think of is possible to track using Tictrac. "

At the moment at a wearable tech auto-capture level this is: Jawbone UP, a Fitbit, or a Nike+ Fuel Band and most everything else you need to measure yourself. It's clunky.

ULTIMATELY all this tracking creates data sets  - those set that still require diagnosis. Synthesizing the data to tell a story.

At a personal level this is probably still going to be a job for doctors and therapists - humans are great at pattern recognition.

But at a big data level computers will give us insights into our health and behaviour that will change the way we live.


Screens aren't going away but they're going to stop dominating.

PROGRESS doesn't stop here.

One of the greatest mistakes is to think about technology as a static form. because the world doesn't stop here.  This, is not it.

If you gave me a phone from 5 years ago to use I would throw it out the window in frustration.

Really, you really need to be thinking about tomorrow. About stuff you don't even want to happen. yet.

Where does your ad go when people consume all the content that they want through a contact lens display, personalised media, and wearable tech. Which formats will survive.

A: branded content and branded utility, (& pre-roll, & billboards)

[image via getpebble]

14:: Wearable Technology WATCHES

Interfaces:  WHAT MIGHT THESE LOOK LIKE  and how do we get these "SIGNALS"?

At the moment it's your phone. But we are around the corner from some pretty interesting wearable technology,

Glasses, watches, bendable screens, transparent touch-screens, digital ink, nano-sensors and metabolic sensors.

those signals, that Information, might be delivered visually, haptically, or sonically?

[Image via Shutterstock]

15::  On bicycles

1. The Dutch are testing "vibrobelts" that deliver navigation to cyclists through navigational nudges -  buzzing gently in the direction you need to move.

The solution appears to make cyclists more aware of the world around them - they see where they're going.

2. Another set of students spent 7 years developing invisible cycle helmets

"Think of it as satnav for your waist. The "vibrobelt", a vibrating belt to help guide cyclists, has proven successful in early tests. It uses vibrating actuators that indicate left, right, backward and forward turn directions, and even tickles the user with coded buzzes that tell them how far they have to go to their destination.

Developed in a masters project by Haska Steltenpohl of the Intelligent Systems Lab at theUniversity of Amsterdam, the Netherlands, alongside supervisor Anders Bouwer, the system aims to give cyclists a "heads-up" navigator, allowing them to keep their eyes on the road after they have chosen their destination on a GPS smartphone. They simply set off and get directional nudges from the vibrators just before each turn.

To see if the vibrotactile navigation compared well with using a standard GPS map on a handlebar-mounted smartphone, 20 volunteers tried both methods on a variety of unfamiliar routes. While all the cyclists reached their destinations successfully, the researchers noted an important difference: when questioned about landmarks they had passed, the vibrobelt users proved much more aware of their surroundings en route than those who were constantly glancing at a GPS screen.

[image via Electricfoxy]

Electricfoxy make jackets that integrate your audio equipment, that allow you to adjust volume with the zip and plug in via the pocket - this is very passive - but at least it looks like something you might wear... with some jeans.

More "wearable" clothing technology. 

I rather love


17:: SONY pillow

This is just a patent application,

Sony describes two different ways technology embedded in the pillow could help sleepers. The first is by monitoring their brain activity. In so doing, it's possible to identify when a person is falling asleep, when they are fully asleep, when they have entered REM sleep and when they have existed REM sleep. And of course when they have woken up. The second is by emitting special radio waves tuned to coincide with certain brain waves during different phases of sleep. [which doesn’t sound like Brave New World at all]

Sony engineers believe it's possible to help the process of falling asleep by emitting radio waves that match brain wave patterns that are consistent with falling asleep. This, they say, helps prevent people who are falling asleep from waking again.

18: Electronic Tattoos

John Rogers at University of Illinois.

"Scientists have developed a flexible circuit board that can be printed directly on to skin to transmit data back to a doctor

The project, pioneered by materials scientist John Rogers, is the latest development in ongoing work into flexible "epidermal electronics". His team have previously engineered stick-on stretchable electronic circuitswith an elastic polymer backing, which could simply be applied to the skin like a temporary tattoo – but with the downside that they could easily wash off.

Nanshu Lu is developing long-lasting “electronic tattoos” that can bond to skin and track and report on the wearer’s vital signs or translate small muscle movements into commands for controlling devices.

Will work inside you too, warn of heart attacks

Great TEDx talk on this tech:

[image via berg]

19:: Enchanted Objects

In 2011 I worked with BERG (who are amazing) on some ideas about what Google might look like without screens. Of an invisible internet. [I think my entire input was basically that question]. They produced amazing work that is now in Lamps and Connbox. for example:

initiate effects by using light to read gestures. For example this is a radio.

If you’re even a tiny bit interested you should read their blog post on it here:

20 :: Simplicity beats complexity

Cognitive overloading.

Looking simple doesn't mean simple  - 3 really simple things beats one simple thing.

we use simple tools to create complex solutions, rather than complex tools to achieve simple solutions

Build knives not sporks.


There is another interesting bit - if you take a big view of all this.

All the QS and multi-screen and user-first signal-based speech-operated physical-web future:

To get the best system for managing all this information that interacts with you all the time - then you would expect to use one giant integrated service that holds all of your information for you and can join it up.


But, then you wouldn't.

Because small focussed services will outperform bigger integrated services. And then they'll get bought. It's the circle of life.

- plus, as a species, we are naturally competitive -

Integrated Aggregator systems will ultimately rely on Disintegrated Boutique services.

20 :: Make it do something, not everything.

My partner finds it infuriating that  she needs 4 remote controls to operate our TV.

And she's right - that is broken.  Somehow I don't  think the answer is another remote control.

We did a complex data analysis of our Remote usage.

We discovered we used it for 4 things.

1. Turn it on. 2. Choose a stream. 3. Change volume. 4. Change Channel.

So I changed the remote to do what SHE wants and we put the other features either on a separate device or buried them.

You have the data to design around this.

21 :: Make it about me. Not about you*

This is the home screen of Trip View - my favourite travel app.

It's home screen ONLY shows the routes you ask it to. I only have one bus route.

It's putting me first.

You have the data to know what your user wants. Use it. Ultimately this is the only thing that will matter.

Whether they are physical or online, whether they are streams, tools or signals - everything needs to be:


This is super relevant. Once I press it the next screen shows me all the buses on that route and whether they are late.

22: Make it good for society*

We design for the user. The individual.

The importance of societal feedback loops over user feedback.

Build for legacy; for trends; for youth.

Design for what we want society to be - not simply what we can do.

You have to go out and create the world that you want.

Pitch a good idea.

If you sell toilet roll pitch an idea for an app that uses location data of every public toilet to show the nearest loo.


and when you think about this you realize we're actually at an exciting but clumsy launch phase of the next internet(s).

Certainly you should be thinking about it, about building enchanting experiences that inspire.

Not websites.

Personally I look forward to the day when we arrive in a physical reality that we augment with data, I think it will be soon and don't sit back and think it will never catch on, that wearable tech is clumsy, or stupid, because, in your life-time, all this will become ubiquitous.

Thank you!