Proposed Implementation of a Face Recognition-based Mobile Device Lock

Brian Attwell (20277461)  and Jeffrey Morgan (20294442)

University of Waterloo

January 29, 2012

Abstract. Lock mechanisms used to authenticate smartphone owners are impractical or suffer important usability issues.  Users are often left to decide between enduring a tardy unlocking process or choosing a very insecure password to save time.  We propose a password-less, easy-to-use, biometric authentication system that would enable secure access to data on any mobile device with a front-facing camera.

1 Problem Statement

While small and very portable, mobile devices are often susceptible to theft or loss.  To avoid compromising valuable data, many smartphone owners resort to a local authentication system named a device lock.  Popular solutions include 4-digit PIN or full plain language (ASCII) password-based locks.  However, because these systems are inconvenient, 38% to 70% of smartphone users do not lock their mobile devices [1].

Users who do decide to employ a device lock often employ a four digit PIN system, such as the one that ships by default with Apple iOS products.  A system that relies on a four digit PIN without any type of punishment (such as a retry counter and delay) for incorrect attempts can be cracked in milliseconds [2].  Further, because some vendors implement password retry counters in software, there is a high risk that they can be bypassed when subject to a brute-force attack [3].

Smartphone owners who opt-in to use a safe, though lengthy password may guarantee the security of their data, but are greatly inconvenienced as unlocking their device requires a large amount of time and may be done several times a day.

The minimal input mechanisms provided by mobile devices for the input of digits or plain language characters are not well-suited for password entry.  Further, smartphones offer a standard set of additional sensors such as cameras.  These additional components are powerful and underused in the context of security [1].

2 Detailed Proposal

2.1 Overview

We propose the design of a mobile authentication system that relies on front-facing cameras that are becoming increasingly prevalent on smartphones.  The solution makes use of face recognition technique based on Local Binary Patterns (LBP) as presented by Ahonen et al. in [4].  To avoid storing user-identifiable information produced by the invertible LBP process, the system uses an approach such as those presented in [5] or [6] to ensure resulting feature vectors cannot be used to reproduce images of the user’s face, i.e. the stored biometric facial features are cancelable.  Liveness detection, such as the approach presented in [7] is employed to avoid false positives due to forgery (such as using existing images of the user’s face).

2.2 Motivation & Reason for Interest

While face-recognition based authentication has been the subject of work for quite some time, the following elements of this undertaking are of striking interest to our team:

• Authentication based solely on a user’s unique features and the potential elimination of the need for a password to identify ourselves.

• Privacy issues regarding the use and storage of facial features. These features must be protected at all costs but simultaneously used in the context of an imperfect classification problem.
• The idea of needing minimal physical interaction to unlock phones or other devices.

2.3 Implementation

The access-control system will be used to authorize access to secrets inside an Android application. The authorization process may or may not reside on the same host as the user features.

Registration: The user sets their face password

1. The phone takes a picture of the user’s face
2. The location of their face is identified by using either the android.media.FaceDetector [8] or the open source library OpenCV [9].
3. The LBP operation converts the identified face bitmap into a feature vector unique to the user.  This feature vector is not stored permanently.
4. A random projection of the user’s feature data is generated using the method proposed by Zhou Lingli and Lai Jianghuang in [5], which applies a projection onto the user’s feature vector. The projection matrix is created with a user-specific random key.  The projection and random key are stored as user metadata. According to [5], this method is non-invertible without both the original, unprojected user biometrics and the user key.

We will implement the random projection technique in [5] ourselves, while relying on an existing open-source implementation of LBP offered by [10].

Authentication: When an unauthorized user wishes to gain authorization

1. The phone begins processing a video of the unauthorized-user’s face, converting each frame of video into a separate image.
2. The location of the unauthorized user’s face is identified, as well as the location of the background.
3. The android.hardware.Camera.Parameters API can be used to determine the distance between the unauthorized-user’s face and the background [11]. If the background is at the same distance as the unauthorized-user’s face, then the unauthorized-user is rejected.
4. The android.media.FaceDetector.Face API is used to detect eye locations and head orientation. If the unauthorized-user doesn’t wink and move their head orientation relative to the background, then the unauthorized-user is rejected. We may decide to implement Junjie Al et al’s Face-Background Consistency Analysis [7] if analysing facial position proves to be difficult.
5. If the banding effect or Moiré patterns are detected, the user is rejected.
6. Lingli et al’s algorithm presented in [5] is again used to generate random projections from several video frames of the unauthorized-user. Chi square statistics is used to compare the similarity between the unauthorized user’s projected features and stored user’s projected features. If the similarity is low, the unauthorized-user is rejected. If the similarity is high, the unauthorized-user is given authorization.

The entire process for both use cases, similar to [5], is outlined in Figure 1 below.

Figure 1: Registration and Authentication Process

An alternative to Lingli’s algorithm, presented in [7] by Toeh et al., uses a method called BioHash to produce cancelable/non-invertable biometric features.  Principally, it also uses a set of randomized numbers from a token to iteratively compute the dot product of the parts of the feature vector with a set of randomly generated basis.  Because our proposal assumes the token or random key may be stolen (we assume the user is not required to enter his or her key upon every authentication, and must therefore store it in our system), the method with the lowest probability of success for an attacker given a stolen key or token will be preferred.

Division of Labor

Each step in the authorization process is independent. We will assign half the authorization steps to each of us and work on them independently.

Improvements on Previous Work & Wow Factor

• Unlike PINs, this access control method doesn’t require error-prone touch screen typing on mobile devices
• Previous implementations of face-recognition based access-control methods suffer the Liveness Problem [6]. In step 3 and 4 of authorization process, we make an effort to reduce the impact of this problem. This is discussed more in the Security Analysis.
• Lingli’s random projection can’t be inverted [5], and neither can the BioHash method described by Toeh et al. in [7]. This reduces potential privacy concerns and exploit vectors if used in a distributed environment, both of which have been present in previous biometric access-control systems [12].

2.4 End-to-end security analysis

We have countermeasures for several attack scenarios spanning artificial biometric attacks and database exploits.

Artificial Biometric Attacks:

Attacker forges photo of authorized user’s face (Liveness Problem)

Images of people are easy to retrieve from social media sites. If an attacker uses an image of the authorized user in order to fool the proposed system, they will be rejected in step 4 of the authorization procedure. We require the user to wink at the camera, and plan to utilize other methods of verifying the liveness of input images.

Attacker finds or creates a video of user’s face (Liveness Problem)

Step 3 and 5 of the authorization procedure will reject the attacker

• Step 3: will detect the edges of the video monitor
• Step 5: because the pixel density of the camera doesn’t perfectly align with the video monitor, Moiré patterns will be noticeable

Attacker animates printed photo of authorized user (Liveness Problem)

An attacker may attempt to gain authorization using a printed photo of the authorized user. The attacker might add a wink animation to the photo using string or elastics. Our proposed system will attempt to reject these types of attacks at the liveness verification stage.

Attacker Attempts Brute-Force Attack

The computational complexity to guess the projected or biohashed feature vector is , where m is the bit length of the projected or biohashed feature vector [5,6].  Attempting to guess the cancelable feature vector is therefore infeasible.

Database Attacks:

Attacker attempts to modify projected features in database to match their own face

Projected feature data will be stored using Android android.database.sqlite library. DB files are only accessible to the app that creates them [13]. If the attacker manages to bypass Android’s security model, they have mostly already won. Nonetheless, we are considering storing the projected data on a separate server in order to avoid this problem.

Attacker get read access to projected feature data (possible if we are transmitting over network)

The random projection is non-invertible as stated in [5]. Therefore, they cannot use the projected features to construct facial model suitable to gain authorization.

Attacker obtains user-specific key or token

Because we propose a system that does not require the user to remember their specific key or token, we must store the user-specific key in plain text.  While this severely increases the chance of success for an attacker, it will not, under any circumstances, allow the attacker access to the raw user features.

Note that using the BioHash approach in [6] decreases the chance of false positives to 1.77%, when the user specific key is known. This attack may be further prevented by including a retry counter or punishments for too many incorrect attempts (such as requiring a secondary password).  Our team will research ways to mitigate this issue.

2.5 Security Pitfalls

There are a number of problems with our solution. In most cases, these are addressable but are difficult issues to fully resolve.

Existing code may contain exploits

We intend to use OpenCV and Android for components of face-detection, face-recognition and image processing. The code is open source. Nonetheless, there may an exploitable aspects hidden in the code. Theoretically, we could avoid using these libraries.

Twins

If your identical twin wishes to gain authorization they will succeed. People with identical twins shouldn’t use our access-control system.

Man in the middle (Possible if we decide to make a separate process or server for authorization)

A man in the middle could intercept messages sent to the authorization server. The man in the middle could modify the messages so that the user is rejected.

Similarly, a man in the middle could intercept messages sent from the authorization server and replace them with approval messages. This would allow the attacker to gain authorization.

Both of these situations could be solved via public key cryptography. However, we may not bother focusing on this aspect of the project.

Access to keys used to generate Random Projections

As mentioned in the possible attacks section, our proposed solution requires less work from the user at the cost of an increased chance of success for attackers attempting to find a colliding hash between the user’s biometrics .

2.6 Test Plan

Face Classifier

To train our LBP-based face recognizer, we will first utilize an existing set of faces such as those presented in [10].  A subset of these examples will be used exclusively for testing.

Security Testing

Most of the system will be tested in an automated fashion. We will create a test set of attempted attacks. We will run the authorization process over these these videos. In addition, we will also run our authorization process over the video’s used by Junjie Al et al.

Step 3 of the authorization process cannot be tested against videos. Therefore, we will perform automated testing for step 3.

2.7 Usability Issues

False negatives due to lighting or other conditions that introduce variance

There will be occasional rejections of authorized users. We could provide a backup authorization (ie, PIN).

No OS integration

Our access-control system can’t be used to unlock your phone. We are only writing an app. Theoretically, we could add OS integration since most Android’s unlock mechanisms are open source [14].

3 References

[1] N. Asokan and C. Kuo, “Usable Mobile Security,” ICDCIT’12, vol. 7154, pp. 1-6, Feb. 2012.

[2] M. Landman, “Managing Smart Phone Security Risks,” Proc. InfoSecCD’10, pp. 145 - 155, 2010.

[3] J. Heider, R. E. Khayari, (2012, Dec. 5). iOS Keychain Weakness FAQ [Online]. Available: http://sit.sit.fraunhofer.de/studies/en/sc-iphone-passwords-faq.pdf

[4] T. Ahonen, A. Hadid, M. Pietikainen, “Face Description with Local Binary Patterns: Application to Face Recognition,” IEEE TPAMI, vol. 28, no. 12, pp. 2037-2041, Dec. 2006

[5] Z. Lingli, L. Jianghuang, “Security Algorithm of Face Recognition Based On Local Binary Pattern and Random Projection,” IEEE ICCI’10, pp. 733 - 738, Jul. 2010.

[6] A. Teoh, Y. W. Kuan, S. Lee, “Cancellable biometrics and annotations on BioHash” Pattern Recognition, vol. 41, no. 6, pp. 2034-2044, Jun. 2008

[7] J. Yan, Z. Zhang, Z. Lei, D. Yi, S. Li, “Face Face Liveness Detection by Exploring Multiple

Scenic Clues,” Recent Advances in Biometric Technology, Dec. 2012

[8] Android Team (2013, Jan 15). FaceDetector Class [Online]. Available: http://developer.android.com/reference/android/media/FaceDetector.html (URL)

[9] S. Audet (2012). JavaCV [Online]. Available: http://code.google.com/p/javacv/ (URL)

[10] OpenCV Dev Team (2012, Dec 26). Face Recognition with OpenCV [Online]. Available: http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html (URL)

[11] Android Team (2013, Jan 15). Camera.Parameters Class [Online]. Available: http://developer.android.com/reference/android/hardware/Camera.Parameters.html (URL)

[12] A. Nagar, “On The Security of Non-Invertible Fingerprint Template Transforms,” Information Forensics and Security’09, pp. 81-85, Dec. 2009

[13] Android Team (2013, Jan 15). android.database.sqlite [Online]. Available: http://developer.android.com/reference/android/database/sqlite/package-summary.html

[14] Google (Apr 08, 2010). ChooseLockPattern.java [Online], Source code: android/settings. Available: https://android.googlesource.com/platform/packages/apps/Settings/+ /bbb4afa19f75694c585506b0c091372d60e07ca7/src/com/android/settings/ChooseLockPattern.

java