1 of 52

CSE 122/222C ; WES 269�Non-RF stuff

Pat Pannuto, UC San Diego

ppannuto@ucsd.edu

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

2 of 52

Today’s Goals

  • Talk about other fun ways embedded stuff communicates

2

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

3 of 52

Outline

  • Infrared

​

  • Visible Light Communications

​

  • Ultrasonic

​

  • Vibratory

3

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

4 of 52

The Infrared Spectrum

  • 300 GHz (1 mm) to 430 THz (700 nm)
    • Lower bound moving? Microwave up to 3 THz (0.1 mm)?
  • Several meaningful subdivisions
    • Near-infrared (NIR) - Night vision, imager support
    • Short-Wavelength (SWIR) - Fiber optics
    • Mid-Wavelength (MWIR) - Thermal (~most heat xfer)
    • Long-Wavelength (LWIR) - Thermal imaging
    • Far Infrared (FIR) - PIR, towards Microwave/RF

​

4

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

5 of 52

The point: not all “IR” created equal

  • Pay attention to the actual wavelengths when sourcing LEDs, photodiodes
  • Also to the selectivity & response
  • For communication, usually want narrow to minimize interference

5

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

6 of 52

The IR communications everyone has probably used:�TV Remote Control

  • Simple on-off keying
  • Two “big” protocols: Phillips RC-5 and NEC
    • Though manufacturers of course have variations
    • RC-5 is 36 kHz carrier, Manchester encoded with 1.8 ms bit period, 1 start bit, 2 header bits (toggle, field), 5-bit address, 6-7 bit commands [why carrier?]

6

Graphics from & more details on Phillips RC-5: http://www.pcbheaven.com/userpages/The_Philips_RC5_Protocol/

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

7 of 52

TV remotes usually use a NIR wavelength (850~950 nm)

  • Many imagers ‘see’ into NIR spectrum, e.g. a smartphone video of a remote:

​

​

​

​

​

​

  • Tip if trying yourself: Newer phones add IR filters (better for taking photos of naturally brightly lit [sunny] scenes), might block remote signals
    • However, at least on my iPhone8, only added to the rear camera, front facing camera still works great ☺

7

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

8 of 52

Newer remotes are switching to wireless technology

  • Why: IR requires line-of-sight
    • ish… reflects well, including off semi-gloss paint (ever point remote at ceiling?)
  • Cool new emerging trend: ‘battery-free’ remotes!
    • Envelope Math:
      • RC-5: 0.9 ms * 14 bits * 100 mW ~= 1.2 mJ
      • BLE Advertisement: ~100 uJ [optimized DA14581]
      • Average button presses per day: 100?
      • State-of-art-WiFi harvesting: 50 uW (@ 2.5cm)
        • At 2.5 cm distance…, so let’s say more like 0.22 nW @ 5m
      • 100 uJ / 0.22 nW ~= 5.26 days

8

https://www.theverge.com/2022/1/2/22860390/samsung-eco-remote-2022-solar-rf-harvesting-charging

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

9 of 52

This will be cool, eventually, but today is probably still just a marketing gimmick [indoor photovoltaic ~30 µW/cm2]

  • Why: IR requires line-of-sight
    • ish… reflects well, including off semi-gloss paint (ever point remote at ceiling?)
  • Cool new emerging trend: ‘battery-free’ remotes!
    • Envelope Math:
      • RC-5: 0.9 ms * 14 bits * 100 mW ~= 1.2 mJ
      • BLE Advertisement: ~100 uJ [optimized DA14581]
      • Average button presses per day: 100?
      • State-of-art-WiFi harvesting: 50 uW (@ 2.5cm)
        • At 2.5 cm distance…, so let’s say more like 0.22 nW @ 5m
      • 100 uJ / 0.22 nW ~= 5.26 days

9

https://www.theverge.com/2022/1/2/22860390/samsung-eco-remote-2022-solar-rf-harvesting-charging

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

10 of 52

Other popular uses of IR: Localization�Why use IR instead of RF?

  • “I’m here to pick up fish”

​

​

​

​

​

​

​

  • Walls are very contextually important, but difficult for localization systems to detect

10

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

11 of 52

IR localization systems semi-ubiquitous in healthcare settings

  • General model is mixed-mode:
    • Anchor infrastructure in rooms beacon unique code for each room [LoS?]
    • Receivers (ID badges, patient trackers, asset tags…) use RF to beacon receiver ID + the IR code; central infrastructure then can track ~everything
    • Beacons mix wide angle emitters and tight lenses
      • E.g. for handwash detection
  • Some tags also include passive RFID as extra redundancy
    • Takeaway: Different�technologies suit sub-�needs for applications

11

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

12 of 52

Outline

  • Infrared

​

  • Visible Light Communications

​

  • Ultrasonic

​

  • Vibratory

12

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

13 of 52

VLC vs IR

  • “next up” in the spectrum: ~380nm – 750nm
    • Very narrow compared to IR
  • Interesting as its own topic because we build many visible light systems
  • Generally single channel or tri-/quad-channel
    • White (brightness), RGB, RGBW
    • Of course, could be finer-grained…
  • Very different interference model
    • Lots of visible light around…

13

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

14 of 52

You’ve been using VLC a lot more lately�[at least, under the broad definition that some VLC researchers like to claim for the area]

  • VLC endpoints are usually highly capable or very simple
    • Transmitters: Screen (large, 2D array of pixels); point source LED (1–small n)
    • Receivers: Imager(s) (large, 2D array of pixels); photodiodes (1–small n)
  • Arguably, imager communicating from a still is VLC too
    • Usually, people are talking about digital / flexible at both endpoint; but blurry

14

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

15 of 52

VLC (and IR really) offer many dimensions for high-bandwidth data transmission

  • Spatial diversity of transmitter, receiver elements
  • Frequency diversity (spectrum / color)
  • Intensity (brightness)
  • Time (high frequency control, decode much easier than RF)
  • Mechanical support
    • …

15

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

16 of 52

Most common VLC modulation schemes

16

  • OOK (On-Off Keying)

​

​

​

  • Variable PPM (VPPM)

​

​

​

  • Color Shift Keying (CSK)

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

17 of 52

VLC (and IR really) offer many dimensions for high-bandwidth data transmission

  • Spatial diversity of transmitter, receiver elements
  • Frequency diversity (spectrum / color)
  • Intensity (brightness)
  • Time (high frequency control, decode much easier than RF)
  • Mechanical support
    • Lenses / optics for beamforming — note: same fundamental concept as waveguides / directional antennas in RF, just much easier to manipulate shorter wavelengths

17

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

18 of 52

FSPL vs. FSO�Free Space Path Loss [RF] vs. Free Space Optical Communication [VLC / laser]

  • Recall that RF falls off aggressively with distance
    • This is partially because we assumed isotropic radiators; non-iso RF is hard

18

https://www.urbe.edu/info-consultas/web-profesor/12697883/articulos/Free%20Space%20Optics%20FSO/Physics-of-FSO.pdf

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

19 of 52

Now let’s toss backscatter principles in the mix, and build something really wild, in the late 1990s[!]

  • For 50 µW, VLC can communicate data for miles
    • Literally, they sent data across the SF Bay

19

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=895117

Challenge:�How to line up sender and receiver?

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

20 of 52

Modern Smart Dust does mixed mode: VLC for inbound data and RF for outbound data

20

A Millimeter-Scale Wireless Imaging System with Continuous Motion Detection and Energy Harvesting �Gyouho Kim, ZhiYoong Foo, Pat Pannuto, Ye-Sheng Kuo, Benjamin Kempke, Mohammad Hassan Ghaed, Suyoung Bang, Inhee Lee, Yejoong Kim, Seokhyeon Jeong, Prabal Dutta, Dennis Sylvester, and David Blaauw�2014 Symposium on VLSI Circuits (VLSIC)

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

21 of 52

More conventional systems also use VLC communication

  • The Electric Imp uses VLC to load WiFi credentials
    • As do many other IoT devices — Cheap answer to “credentials no keyboard?”

21

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

22 of 52

The solid state lighting revolution should not be a simple substitute good

22

00010100 🡪

~10-4 Hz

~104 Hz

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

23 of 52

Digital lighting creates application potential for smart spaces beyond baseline illumination

23

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

24 of 52

VLCP: visible light communications and positioning

  • LED luminaires
    • Slightly-modified
    • Transmit beacons
    • Identities or coordinates

​

  • Smart phones
    • Run background mobile app
    • Take images periodically
    • Perform local processing
    • Offload to cloud/cloudlet

​

  • Cloud/cloudlet server
    • Do photogrammetry
    • Do AoA Localization
    • Estimate location
    • Estimate orientation
    • Provide location-based services

Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta, "Luxapose: Indoor Positioning with Mobile Phones and Visible Light,” In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (MobiCom'14), Maui, HI, Sep. 7-11, 2014.

24

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

25 of 52

Indoor localization using VLCP

25

S1

S2

S3

S4

S1

f1

S2

f2

S3

f3

S4

f4

01100101000

Image processing extracts beacon locations and frequencies

Smart Phone

Compute

Minimize

LED Luminaire

Illuminate

Idle

TX <66>

TX packet

Captured using a rolling shutter

Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta, "Luxapose: Indoor Positioning with Mobile Phones and Visible Light,” In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (MobiCom'14), Maui, HI, Sep. 7-11, 2014.

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

26 of 52

One vision: Software Defined Lighting�Multiplex all these services on same infrastructure

26

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

27 of 52

Real-world study: Illumination + Localization + Sensor Communication

27

Synchronization Event

Localization Service

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

28 of 52

With sufficient synchronization, transmitted signals can ‘constructively interfere’ (coherently combine)

28

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

29 of 52

VLC as a sensing channel

  • Transmitter does not have to intentionally send a data signal…
    • Infer grid and bulb information!

29

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

30 of 52

VLC as an attack channel

  • From simple, ‘static’ modifications…

​

​

​

  • …rolling shutter to create ’invisible’ attacks!

30

Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, Dawn Song. Robust physical-world attacks on deep learning visual classification. CVPR’18.

Athena Sayles, Ashish Hooda, Mohit Gupta, Rahul Chatterjee, Earlence Fernandes. Invisible perturbations: Physical adversarial examples exploiting the rolling shutter effect. CVPR’21.

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

31 of 52

Outline

  • Infrared

​

  • Visible Light Communications

​

  • Ultrasonic

​

  • Vibratory

31

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

32 of 52

Moving from the EM spectrum to the acoustic spectrum

  • Many of the same principles apply
    • Recall the discussion on symmetry in waves earlier

​

  • Note / Resources:

32

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

33 of 52

The wavelength represents variation in air pressure and is the distance over which the wave’s shape repeats.

​

An oscillation of a wave is defined how long it takes for a wave to move from starting position, one position to the next and back to the start.

​

Amplitude refers to the intensity or power of the sound wave, and relates to volume.

​

The frequency of a wave is defined as number of oscillations the wave completes in a certain amount of time (usually seconds).

​

The unit of measurement for sound is a hertz (Hz), and it is based on how many oscillations occur per second.

Vocabulary

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

34 of 52

Infrasound

Ultrasound

Audible frequencies

200MHz

2MHz

20Hz

20KHz

Sound comes in many frequencies

Can humans hear all types of sounds?

No, audible frequencies are what we can detect.

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

35 of 52

Acoustic spectrum does not have as many divisions as EM

  • Possibly a statement of opportunity / underutilization for communication

35

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

36 of 52

Infrasound

Ultrasound

Audible frequencies

200MHz

2MHz

20Hz

20KHz

Ultrasonic applications

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

37 of 52

Measuring distance with sound

​

In air, sound travels at a constant speed, which means we can measure distance by seeing how long it takes for a sound to hit and object and bounce back to the sensor.

​

Ultrasonic waves can be used to accurately measure distances with special sensors.

Sender

Receiver

14cm

Reflected wave

Original wave

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

38 of 52

Practical communications with Ultrasound?

  • Not much that does data on the UL channel
  • Acoustic often useful for data on otherwise difficult spaces
    • Recent paper on UL transmit through solid media gets 12.5 kbps
    • Another new one does UL backscatter (!) at 1~3 kbps

​

  • In most sensor network work, UL used with other technologies
    • Primarily for ranging, easy to measure acoustic time-of-flight (b/c slow)
    • Not never data: ALPS labels chirps to distinguish transmitters
    • But very little, second generation ALPS is hybrid BLE + UL

38

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

39 of 52

Neat tricks with UL: Near-zero energy wakeup

  • Opo is UL + RF ranging platform with 60 µW “UL wakeup radio”

39

G

G

G

C

1000x Gain Stage

Comparator:

Digitizes signal

Integrator/Detector:

Stops spurious noise

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

40 of 52

Opo’s (and others’) actual ranging operation is simple —�Time Difference of Arrival (TDoA) ... 3*108 m/s vs 343 m/s

40

TX

RX

UL

UL

RF

UL

RF/UL

Time

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

41 of 52

Pros and Cons of UL for embedded applications

  • Directional
    • Can be problematic or useful
    • Middle point between optical and RF in shape options
  • Physically larger transducers
    • Need to move a membrane that moves air; not that small
  • Propagation
    • Generally does not cross physical media well
      • Stays in a room; but can also travel along a wall if transducers on wall…
    • “Echo-y”: Can take 50-200 ms for UL energy to die off in a room

41

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

42 of 52

Outline

  • Infrared

​

  • Visible Light Communications

​

  • Ultrasonic

​

  • Vibratory

42

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

43 of 52

Premise: Exploit physical linkage for communication channel that has physical world implications

  • (Slides stolen from conference presentations for Ripple and Ving)

43

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

44 of 52

44

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

45 of 52

45

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

46 of 52

46

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

47 of 52

47

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

48 of 52

Basic insight: Can do reasonably complex modulation schemes on this channel to get decent bandwidth using microphone to RX

48

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

49 of 52

Decent data rates nowadays

49

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

50 of 52

Ving argument: Mixed-mode again, use RF for high-bandwidth, but vibratory channel for wakeup [lower power] and security

50

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

51 of 52

Outline

  • Infrared
  • Visible Light Communications
  • Ultrasonic
  • Vibratory

​

  • Is there more?
    • So much more

51

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell

52 of 52

Next time: You tell me, just one more! :/

52

CSE 122/222C ; WES 269 [WI25]

CC BY-NC-ND Pat Pannuto – Content developed in coordination with Branden Ghena and Brad Campbell