Computers that know your mood

Machines are getting increasingly able to tell our faces apart, which has big implications in security. But can this technology be a double edged sword? What's more, modern computers are getting much better at recognising emotions. Peter Cowley explained to Kat Arney how this technology works, and where we might see it being used in the future...

Peter - Well it's quite technical, but if I run through very briefly.  There are really four methods that do that.  One is geometric i.e. the position, size of the eyes, nose, check bones, etc. The next version is to add 3D to that, so that's the depth of the eye sockets, the nose and the chin.  There's a photometric way of doing it which is effectively taking a statistical comparison between your face and one stored on the database and finally, we add to that skin texture analysis - lines, patterns, spots, colour, etc.

Kat - How could this work in things like passports...  those kind of security systems.  Because my face doesn't look a lot like my passport photo a lot of the time?

Peter - The big issues are really in detecting  whether you are looking directly on or not, or turning your head.  The lighting, hair, whether you've got your glasses on or not which is why, of course, you have to take your glasses off at the passport thing at the airports.  And also if you're still alive or not, of course.  You could have just had somebody print out a photograph of you.  But yes, the ones that are being used, to answer your question are, as you say passports.  In fact, Australia and New Zealand are already using that.  Things like criminals in crowds. I'm sure the security forces use that a tremendous amount.  Silly little things like greeting hotel guests, so as you walk in it says "hello Mr Smith" etc.  ATMs - to replace pins.  That might be better than a pin might it etc. etc. And we already use them, in fact because of these smile detectors in our cameras on our phones that you say "smile" and then it detects the smile has occurred and will then take the photograph.

Kat - I find that kind of stuff really fascinating - that can see faces and all these sort of things.  And you said the smile detector.  So we're starting to see stories now coming out about facial recognition software that can detect emotions.  Tell me a bit about this.  What is this kind of technology?

Peter - Again a bit of tech to start with.  First of all we have apparently 33 muscles in our face of which 9 are around the eyes, 4 is on the nose and 12 around the mouth.  They've produced something that is standard throughout the world, a facial action coding system which splits into action units, some of which are really unusual like nostril dilator, and lip corner depressor.  So based on those 30 action units, and the movement between that, then systems are in place to detect one of 7 base emotions which are joy, sadness, surprise, anger, fear disgust and contempt.  So, if you are looking at somebody and in the conversation you will have, you know, a pretty constant emotion.  But there are also what's called 'micro-expressions' which just last about 50 milliseconds. And I actually tested this on somebody the other day - I mentioned somebody's name - you see this sort of look of slight unease and then a smile appeared after that.

Kat - How well does this kind of stuff work.  Because maybe some people hide their emotions.  Is this technology actually good?

Peter - Well.  Apparently there are something like 1 in 400 of us who are actually pretty good lie detectors.  Who can actually recognise the lie because of this micro-expression occurring.  This is being used primarily for research at the moment and it's moving into other applications like training autistic children - about emotions, because they struggle with recognising what the emotion is of the person they're facing.  Robert carers - these are starting to happen in Japan.

Kat - On a more trivial thing.  Could it perhaps say, if you're chatting with someone on social media or something you could go "hey, they look like they fancy you a bit" or a smart mirror that says "oh you look at bit sad today" and tells you "you're looking fabulous.

Peter - Some listeners might have heard of 'Tinder' which is an app that's used by younger people than me for finding mates you could be doing by just looking at the photograph and swiping either way.

Kat - It's quite a terrifying thought. Could this have  security implications?  What's going on here?

Peter - The two big issues around at the moment are the usual, the sort of false positive and false negative.  If you start making decisions based on something that's statistically correct or not, and you get it wrong, then you are potentially going to make the wrong decision.  Lie detector, for instance, accusing somebody of lying and their not.

Kat - And potentially things like banking and security.  Stuff like that..

Peter - Exactly, yes.  And the big one that's floating around which is in all technology spheres is privacy.  And, in fact, I have noticed that people put post-it notes over the camera when they're not using Skype so it can't be looked at, you can't be hacked into.