Apple’s new Photos app will recognise seven different facial expressions

Oscar Raymundo
22 June, 2016
View more articles fromthe author

Apple’s new update to Photos is one of the reasons we’re most excited about iOS 10, and now we’re learning more about how the app will integrate ‘Advanced Computer Vision’.

After Apple released iOS 10 and macOS to developers at WWDC (Worldwide Developers Conference), a developer discovered the list of hardcoded objects, scenes and facial expressions that the Photos app will be able to recognise. According to developer Kay Yin, Photos will be able to recognise seven types of facial expressions: greedy, disgusted, neutral, screaming, smiling, surprised and suspicious.

More impressively, the Photos app will also be able to detect 4432 objects and scenes, from ‘alleyways’ to ‘zucchinis’. This object and scene recognition is used by Photos to yield accurate search results in all languages.

“We do 11 billion computations per photo to be able to detect things like, there’s a horse, there’s water, there’s a mountain, and with this you can do really powerful searching on your device,” Craig Federighi, Apple’s senior vp of software engineering, said during the WWDC keynote.

Additionally, object and scene recognition is used to generate ‘Moments’, the new video montage feature in Photos that automatically stitches together media based on theme, date and location metadata. According to Yin, Moments will be generated according to 33 different categories, like ‘day in history’ and ‘last weekend’.

Why this matters: Apple’s improved Photos app in iOS 10 is now a lot like Google Photos: using face, object and scene recognition so you can search your camera roll by person, location or topic. Unlike Google Photos, however, Apple has made sure that Photos uses only machine learning to analyse the photos that are stored locally on your iPhone. Apple believes there doesn’t necessarily have to be a trade-off between machine learning and privacy. At WWDC, the iPhone-maker introduced the concept of ‘differential privacy‘, which employs crowdsourcing to make a more useful product, without exposing any one individual’s data. And Apple seems to be relying on this concept in the new Photos app, the place where we store our most personal information.

Leave a Comment

Please keep your comments friendly on the topic.

Contact us