Prof. Christopher Rasmussen

​

May 9, 2017

CISC 181

Android:

Sensors, camera, faces

The Project

• Project milestone #1 due tomorrow
• Thursday we are working on projects in class

​

SensorManager

• Class for reading sensors:
• Light level
• Temperature (ambient)
• Accelerometer (gravity)
• Magnetic (i.e., compass) direction
• Atmospheric pressure
• Heart rate
• Also has constants like GRAVITY_EARTH, GRAVITY_JUPITER, LIGHT_SUNLIGHT, LIGHT_SHADE

​

SensorManager

• Class for reading sensors:
• Light level
• Temperature (ambient)
• Accelerometer (gravity)
• Magnetic (i.e., compass) direction
• Atmospheric pressure
• Heart rate
• Also has constants like GRAVITY_EARTH, GRAVITY_JUPITER, LIGHT_SUNLIGHT, LIGHT_SHADE
• Not all may be supported on device, so

List<Sensor> getSensorList(Sensor.TYPE_ALL)

​

SensorManager

• Class for reading sensors:
• Light level
• Temperature (ambient)
• Accelerometer (gravity)
• Magnetic (i.e., compass) direction
• Atmospheric pressure
• Heart rate
• Also has constants like GRAVITY_EARTH, GRAVITY_JUPITER, LIGHT_SUNLIGHT, LIGHT_SHADE
• Not all may be supported on device, so

List<Sensor> getSensorList(Sensor.TYPE_ALL)

• Need permission -- add to manifest

​

Sensors: Accelerometer

SensorManager SM = (SensorManager)

getApplicationContext().getSystemService(Context.SENSOR_SERVICE);�SM.registerListener(

new SensorListener(), // this is our custom class

SM.getDefaultSensor(Sensor.TYPE_ACCELEROMETER),

SM.SENSOR_DELAY_NORMAL);

​

​

​

Sensors: Accelerometer

SensorManager SM = (SensorManager)

getApplicationContext().getSystemService(Context.SENSOR_SERVICE);�SM.registerListener(

new SensorListener(), // this is our custom class

SM.getDefaultSensor(Sensor.TYPE_ACCELEROMETER),

SM.SENSOR_DELAY_NORMAL);

​

class SensorListener implements SensorEventListener {� public void onSensorChanged(SensorEvent event) {� if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {� aX = event.values[0]; aY = event.values[1];� aZ = event.values[2]; haveAccel = true;� }� }

� public void onAccuracyChanged(Sensor sensor, int accuracy) { }

}

​

​

And if you're really bored...

And if you're really bored...

​

VIBRATION!!!

The Camera

• Two major ways to interact with the camera functionality:
1. "Ask" another app to handle picture-taking/ video-recording task
2. Directly access camera from within app

​

Getting Images/Video via Intents

• We have already seen Intents used for starting another Activity and exchanging information with it
• But that Activity was part of our app -- now we want another app to take care of the camera interaction
• Don't need to change manifest permissions to take picture, but do need it if we want to store full-size photo

​

Review: Starting an activity

• Intent intent = new Intent(this, SecondActivity.class);
• This is called an explicit Intent
• Add arguments to Intent with putExtra()
• startActivity(intent) to actually transfer control to the new activity

​

​

Review: Returning information

• Use startActivityForResult()
• Besides Intent, extra int parameter is "request code" -- will help identify which activity is returning results
• When new activity ends, onActivityResult() automatically called in old activity
• int request Identifies which Intent started new (just finished) activity
• int result Indicates how new activity ended
• Intent data Holds extra information sent back from new activity

​

Starting an "external" photo activity

• Now the activity was not created by us...and we may not even know its name

​

Starting an "external" photo activity

• Now the activity was not created by us...and we may not even know its name
• But we do know what action we want it to perform, so we create an Intent with an argument that specifies this:

Intent takePictureIntent =

new Intent(MediaStore.ACTION_IMAGE_CAPTURE);

• This is called an implicit Intent

​

Common Intents

• Alarm Clock: Set alarm, create timer, show current alarms
• Calendar: Add event
• Camera: Capture photo/video
• Contacts/People App: Select/view/edit/add contact
• Email: Send, with and without attachment(s)
• File Storage: Get/open a specific type of file
• Maps: Show location on map
• Music or Video: Play a media file based on search terms
• New Note: Create a note
• Phone: Initiate a phone call
• Search: Perform a web search
• Settings: Open a particular settings page
• Text Messaging: Compose SMS/MMS message
• Web Browser: Load a web page

​

Starting an "external" photo activity

• Now the activity was not created by us...and we may not even know its name
• But we do know what action we want it to perform, so we create an Intent with an argument that specifies this:

Intent takePictureIntent =

new Intent(MediaStore.ACTION_IMAGE_CAPTURE);

• This is called an implicit Intent
• Before starting the activity, make sure one exists that can perform the action (otherwise app dies):

​

if (takePictureIntent.resolveActivity(getPackageManager()) != null)

​

Starting an "external" photo activity

• Now call it:

​

startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE);

​

• And handle the result when it finishes:

​

protected void onActivityResult(int requestCode, int resultCode, Intent data) {

if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {

Bundle extras = data.getExtras();

Bitmap imageBitmap = (Bitmap) extras.get("data");

mImageView.setImageBitmap(imageBitmap);

}

}

​

• But this is just a thumbnail, not the full-resolution image -- more details here

​

​

Starting an "external" video activity

• Specify action:

Intent takeVideoIntent =

new Intent(MediaStore.ACTION_VIDEO_CAPTURE);

• Start the activity:

​

if (takeVideoIntent.resolveActivity(getPackageManager()) != null) {

startActivityForResult(takeVideoIntent, REQUEST_VIDEO_CAPTURE);

}

​

• Handle result (comes back as a Uniform Resource Indicator (URI) pointing to file):

​

protected void onActivityResult(int requestCode, int resultCode, Intent intent) {

if (requestCode == REQUEST_VIDEO_CAPTURE && resultCode == RESULT_OK) {

Uri videoUri = intent.getData();

mVideoView.setVideoURI(videoUri);

}

}

​

​

​

Getting Images Directly

• Directly access camera from within app
• Need to change manifest permissions for this

​

Getting Images Directly

• Directly access camera from within app
• Need to change manifest permissions for this
• Instantiate Camera object (doing it in a separate thread recommended)
• Create a SurfaceView and pass its holder to the Camera's setPreviewDisplay(), then call startPreview()

​

Getting Images Directly

• Directly access camera from within app
• Need to change manifest permissions for this
• Instantiate Camera object (doing it in a separate thread recommended)
• Create a SurfaceView and pass its holder to the Camera's setPreviewDisplay(), then call startPreview()
• Further topics
• Change camera settings (preview size, focus, white balance, etc.)
• Save picture (either one-off, or continuously)
• Process picture

​

Ever Wonder How This Works?

Google Mobile Vision API

• Detect faces, barcodes, text (OCR!) in raw images
• Face detection pipeline

FaceDetector detector = new FaceDetector.Builder(context)

.setLandmarkType(FaceDetector.ALL_LANDMARKS)

.setClassificationType(FaceDetector.ALL_CLASSIFICATIONS)

.setTrackingEnabled(true)

.setMode(FaceDetector.FAST_MODE)

.setProminentFaceOnly(mIsFrontFacing)

.setMinFaceSize(mIsFrontFacing ? 0.35f : 0.15f)

.build();

​

Google Mobile Vision API

• Detect faces, barcodes, text (OCR!) in raw images
• Face detection pipeline

FaceDetector detector = new FaceDetector.Builder(context)

.setLandmarkType(FaceDetector.ALL_LANDMARKS)

.setClassificationType(FaceDetector.ALL_CLASSIFICATIONS)

.setTrackingEnabled(true)

.setMode(FaceDetector.FAST_MODE)

.setProminentFaceOnly(mIsFrontFacing)

.setMinFaceSize(mIsFrontFacing ? 0.35f : 0.15f)

.build();

• Face examples
• FaceTracker
• googly-eyes