Face ID on the iPhone X: Apple releases Face ID white paper and support document

Glenn Fleishman
28 September, 2017
View more articles fromthe author
AAA
News

The upcoming iPhone X will use Face ID, technology that unlocks your iPhone X by using infrared and visible light scans to uniquely identify your face. The company says it will work in a variety of conditions and is extremely secure. Though no one outside Apple has yet been able to confirm how well it works.

[Editor's note: This article was updated on 27 September 2017, with links to Apple's Face ID white paper and Face ID support document.]

This brings up a lot of questions about Face ID. We’ll work to get answers to you as you ponder whether you want to buy an iPhone X when it starts shipping November 3.

Face ID white paper and support document from Apple

Apple has released a Face ID white paper (PDF) that provides a bit of detail on how it works. There’s also a Face ID support document that explains how the technology protects your information.

What is Face ID?

Face ID a form of biometric authentication. Rather than a password (something you know) or a security dongle or authentication app (something you have), biometrics are something you are. Fingerprint recognition is also a biometric.

Instead of one or more fingerprints, as with Touch ID, Face ID relies on the unique characteristics of your face. Apple is betting that its technology can meet six separate hurdles:

  • Initially scan your face accurately enough to recognise it later.
  • Compare a new scan with the stored one with enough flexibility to recognise you nearly all the time.
  • Scan your face in a wide variety of lighting conditions.
  • Update your facial details as you age, change hairstyles, grow a moustache, change your eyebrows, get plastic surgery and so forth to still recognise you.
  • Let you wear hats, scarves, gloves, contact lenses and sunglasses, and still be recognised.
  • Not allow a similar-looking person, a photograph, a mask or other techniques to unlock your phone.

If you’ve had trouble with Touch ID and your fingerprints over time, you might have concerns about whether scanning and recognising a face is easier. It might be!

What devices use Face ID?

At this announcement, only the iPhone X is slated to include Face ID.

Does Face ID replace Touch ID?

For now, the hardware requirements for Face ID are only found in the upcoming iPhone X. All other new iPhone models and all existing ones will retain Touch ID. Apple might choose to retain Face ID as a premium differentiating feature, but based on the company’s previous patterns of high-end feature rollouts, it will likely be in both new iPhone models in late 2018, and then in the iPhone SE at some point thereafter. It would seemingly also be likely for future iPad Pro updates, but perhaps not until the second quarter of 2018 or even mid-year.

What will I use Face ID for?

The same things you currently do: Apple Pay, App Store and iTunes purchases and third-party apps that currently rely on Touch ID. Apple says that third-party apps – as with Touch ID – will be able to allow a Face ID authentication, and iOS only informs the app whether or not the match was accurate.

Interestingly, Apple says developers can use Face ID without a fallback to a passcode, if a developer wants to use the biometric identification as a kind of second factor that can’t be bypassed.

Third parties will also have access to live depth maps, just as the rear two-camera systems provide in iOS 11, but not the raw data of sensors sampling your face.

How do I set up Face ID?

Face ID uses an ‘enrolment’ process just like with Touch ID. You’ll go to Settings > Face ID & Passcode and tap Enrol Face, and then the iPhone will use the front-facing camera to display your face within a circle with green tick marks surrounding it. The enrolment software will overlay quasi-3D markings onscreen to show your eye line and facial centre. You’ll be prompted to move your head in a circle, while your facial characteristics are captured.

face id setup

Apple says the odds that someone else’s fingerprint will unlock Touch ID is one in 50,000, a pretty low number given there’s no way to test for that without trying. Apple says Face ID’s chance of another face matching is one in 1,000,000.

Apple’s senior vice president of worldwide marketing, Phil Schiller, did say during the iPhone X introduction that, “The statistics are lower if the person shares a close genetic relationship with you.” Apple clarified this in its white paper, noting that the accuracy is ‘different’ for twins and siblings. If you have an evil twin, you should probably avoid Face ID.

It also said that children under 13 had a higher rate of false matches, though it didn’t provide a number, because distinct facial features “may not have fully developed.”

How does Face ID work?

Apple uses a combination of infrared emitter and sensor (which it calls TrueDepth) to paint 30,000 points of infrared light on and around your face and also capture flat or 2D infrared snapshots. For the points, the reflection is measured, which allows it to calculate depth and angle from the camera for each dot and construct a depth map.

animoji iphone x

Live depth mapping is also used for live tracking for Animoji, the talking animals heads – and piles of poo – that match your facial expressions and lip movement, and other selfie special effects, and is provided to third-party developers. But live depth mapping doesn’t offer up raw sensor data that would let a developer recreate a Face ID map.

How do I unlock with Face ID?

The phone isn’t scanning all the time, thank goodness! Instead, you’ll need to wake the phone with one of several methods, which includes rise to wake or tapping the Sleep/Wake button or an app or Apple service requesting authentication. Then you’ll look at the phone. Apple says the infrared sensors should penetrate sunglasses, but your eyes need to be open – so kids can’t unlock the phone of a sleeping parent by sneaking into their room, unless said parent sleeps with their eyes open.

To avoid unintentional unlocking, Apple says Face ID is ‘attention aware’. If you’re not looking at the phone attentively – that is, you’re looking away or engaged in an activity on the phone’s lock screen – it won’t perform an unlock operation. This awareness can be turned off by a user as an accessibility option. (Thus a sleeping parent with eyes open would also have to be looking intently!)

Apple says that Face ID can be used in a wide variety of lighting conditions, including dark rooms. On one diagram of the iPhone X, Apple shows a ‘flood illuminator’, which conceivably provides infrared illumination in the dark to aid the TrueDepth system. No visible light is used, so the scan will be invisible in all lighting conditions.

Over time, Apple will create temporary updates to your Face ID profile for good matches that aren’t perfect to keep it up to date. But it warns these updates are only good for a ‘finite’ number of unlocks, meaning it expects changes to your face to either revert to the mean – you changed eye shadow and eyebrow shape and change back, or grow a moustache and shave it? – or you will have to re-enrol your face. The details aren’t clear.

Interestingly, if you fail to unlock your phone with Face ID but the failure was iffy – close, but not close enough of a match – and then you immediately enter your passcode successfully, Apple says it will take a new capture and adapt the stored Face ID profile with it. It will retain this new information for a period of time as with other facial changes above, but it will also discard it if you stop matching against the changes and look like your old self again.

face id light

How secure is Face ID?

Apple’s description of enrolment and comparison is very similar to Touch ID. The enrolment sends data through a one-way channel to the Secure Enclave, a special tamper-resistant chip bound deeply inside the iPhone and iPad architecture that can only respond with limited information, such as confirming a match was made when unlocking for Apple Pay and the like. Secure Enclave also stores some other private information.

As a result, Apple doesn’t collect this information and process it centrally, nor does it store it on the device in a manner that can be retrieved by cracking a phone, a phone backup or intercepting information to and from it.

However, the concern remains that, with proprietary technology under the control of Apple, a government could force changes that would pass or extract facial identification information or perform comparisons with faces that a government is looking for.

In the current hardware architecture, however, that seems unlikely. Apple has engineered its systems so that there’s no reasonable way to rework it to change the flow of facial (or, with Touch ID, fingerprint) information to a different source. It would have to create a whole new kind of phone and new firmware.

Apple notes that developers can create encryption keys protected by Face ID that are stored in an iPhone X’s Secure Enclave. These keys can be used by the developer’s app entirely within the Secure Enclave, performing operations that are as protected as Apple Pay and biometric identification. It will be interesting to see how developers make us of this additional security level. (It doesn’t seem as if this is an option for Touch ID.)

Can someone fool Face ID?

The answer should be no, based on the approach Apple has taken: number of data points, use of infrared scanning and attention awareness. With other systems, people have used photographs, plaster models and other approaches that Face ID would seemingly resist. A plaster model doesn’t offer the same reflection as a 100 percent identical human face, because infrared reflects off living skin differently than off an inanimate material.

Apple says in its white paper that it introduces randomness to make it even more difficult. The sequence of 2D infrared scans and depth-map dot captures are sent in a random order, and the project dot-pattern is both random and unique for each device. This makes it harder for an attacker to use predictable elements to dupe a scanner – or they might succeed in fooling an iPhone X in their possession, but fail with any other iPhone X out there.

The Face ID security white paper notes in passing that while there’s a primary neural network that performs identification, a second neural network checks against spoofing, looking out for telltales of photos and masks.

You can imagine that the second the phones hit the market, security researchers (and government agencies) will start testing ways to fool Face ID and that some will have limited success – which, for cases disclosed to Apple, will improve deterrence from those workarounds.

Can Face ID be set up so that multiple people’s faces unlock the same iPhone?

For now, an iPhone X will recognise only a single face. That could change in the future. But you can no longer give a spouse, partner or other person access to your phone through a biometric means, like you could by enroling one of their fingers with Touch ID. You’ll need to share a password with them.

Will Face ID recognise faces of people of colour as well as it does white people’s faces?

We hope Apple has learned from machine-learning and body-recognition debacles at other companies that have led to people of colour not having the same accuracy of automatic photo tagging, facial recognition and other problems. (See the video of a man unable to get a hand drier seemingly to recognise his hand colour.)

The kind of machine learning used widely now for voice, image and other recognition relies on training databases. Companies or academic projects have to find often hundreds of millions of examples that they can mark correctly to feed into a deep-learning system to have it develop the pathways that let it recognise features more generically instead of as exact matches.

In the past, these training databases have apparently been heavily biased towards white faces and often towards men, leading to racially insensitive and upsetting results. Apple vice president Phil Schiller says in the keynote that Apple used a billion images to train Face ID, but not which faces.

In its announcement and on its website, Apple features a number of people of colour more in proportion with the global population than American or European ones in the Face ID and other TrueDepth sections, as well as showing heavily freckled faces and women with elaborate and enormous hair.

Apple’s senior vice president of software engineer, Craig Federighi, later expanded on this to TechCrunch, explaining, “We’d done data gathering around the globe to make sure that we had broad geographic and ethnic data sets.” In Apple’s white paper, the company notes it created “a representative group of people accounting for gender, age, ethnicity and other factors.”

Until all-comers can start using Face ID, we can’t know whether or not it will perform up to the standards that Apple has set for wide-scale introduction of features other companies offer failed or weak versions of. Touch ID’s early days were generally good, but plenty of people – this writer included – had to regularly retrain fingers. Over time, that problem seems to have disappeared, as complaints are rare these days in forums in which they were once common.

The real test is one of time: as you change make-up, glasses and hair styles, and as our features tick away the passage of time, will Face ID keep up with us?

Face ID appeared to fail for Apple exec Craig Federighi during the demo. Apple later said that because the demo unit was handled by a number of people between when Federighi trained it and the demo, it exceeded the bad-match limit, a security feature and locked out Face ID as it was supposed to.

Do I have to use Face ID?

No, you can use a passcode.

How do I disable Face ID?

You have several options:

  • You can use Settings in iOS 11 to disable it.
  • You can power down your phone and restart it.
  • You can make five failed attempts to unlock it.
  • You can press the Wake/Sleep button five times in sequence. (This disables Touch ID in iOS 11 on all models.)
  • You can press and hold the Wake/Sleep button and either volume button on the opposite side. This new option triggers a power-down screen, but also disables Face ID. (This also works to disable Touch ID on an iPhone 8 and 8 Plus, but isn’t available on older phones.)

When can’t I use Face ID?

In several cases, you have to use a passcode (these parameters also apply with Touch ID):

  • After restarting.
  • After any 48-hour period in which you haven’t used Face ID to unlock the phone.
  • Every six days, an eight-hour timer starts, and if you don’t use Face ID within those eight hours, you’re prompted for a passcode the next time you unlock.
  • After five failed attempts to unlock with Face ID.
  • After disabling using any technique above not already mentioned in this list.

Leave a Comment

Please keep your comments friendly on the topic.

Contact us