Hands

Jitendra Malik

The sensorimotor continuum of human hand function

• Passive Tactile Sensing
• Active Haptic Sensing
• Prehensile movements e.g. pick and place
• Non-prehensile movements e.g. gestures, typing, piano playing

Jones & Lederman, Human Hand Function, Fig. 1.1

Proprioception

• Aristotle in “De Anima” listed 5 senses: sight, hearing, touch, taste and smell.
• Bell (1826) proposed a sixth sense – the “muscular sense”, which was renamed as “proprioception” by Sherrington.
• Proprioception refers to the awareness of the position or movement of the body or parts of the body relative to each other
• Musculoskeletal system : sensors in muscles, tendons and joints
• Vestibular system in the inner ear: semicircular canals & otoliths

The term “kinesthesia” can be used if we ignore the vestibular system

In robots..

• Joints are driven by electric motors where a shaft rotates and applies a torque in response to a control input
• A rotary encoder senses the angle
• From these we can compute the various joint angles and velocities

The sense of touch

• When the skin is contacted by an external stimulus, its surface is indented or stretched as the skin is flexible.
• The mechanical deformation is detected by mechanoreceptors that signal location of contact, force exerted, and motion speed.

The sense of touch

• Our subjective perception is that of light touch or pressure, and for a moving stimulus that of stroking or vibration.
• Other receptors underneath the skin sense warmth, cold, pain, itching.

The quarter in the parking meter� Touch: The science of Hand, Heart, and Mind: David Linden

• Find the quarter in your pant pocket (or purse)
• Grip it in your fingers and carry it to the coin slot
• Insert the coin
• Grasp and twist the meter’s handle

Block Diagram of the Primate Visual System

Common aspects: What vs Where/How

• Ventral stream connects to multimodal conscious perception. Object names, attributes etc. Today we might think of VLMs
• Dorsal stream connects to action. Perception guides action. For example detecting incipient slip may lead us to increase the grip force without conscious awareness

The sensorimotor continuum of human hand function

• Passive Tactile Sensing
• Active Haptic Sensing
• Prehensile movements e.g. pick and place
• Non-prehensile movements e.g. gestures, typing, piano playing

Jones & Lederman, Human Hand Function, Fig. 1.1

How amazing is this..

• Average Braille readers are at 120 words per minute. Experts at 200 wpm
• This corresponds to recognizing a character in 50 ms
• Average visual reading speed is 240 wpm

Exploratory Procedures (Lederman & Klatzky 1987)

The sensorimotor continuum of human hand function

• Passive Tactile Sensing
• Active Haptic Sensing
• Prehensile movements e.g. pick and place
• Non-prehensile movements e.g. gestures, typing, piano playing

Jones & Lederman, Human Hand Function, Fig. 1.1

Napier (1956) proposed two main categories

Sign language

Italianate