Microexpressions: More Than Meets The Eye

Transcript

IRA FLATOW, HOST:

This is SCIENCE FRIDAY, I'm Ira Flatow. We're broadcasting today from Stanford University. This hour we're going to talk about a phenomenon that you've probably seen many times during your life but never quite noticed, and these are microexpressions, flashes of emotions across a person's face that can last just a tenth of a second, long enough for the trained eye to spot but all but invisible to the rest of us. So what are we all missing?

My next guest has trained officials at the FBI, the State Department, the TSA to read these microexpressions. But how much do these expressions actually communicate? Can you tell if someone's lying just by looking at them, as the show "Lie to Me" might have you believe? Or could these expressions sometimes just lead you astray?

David Matsumoto is the director of Humintell, professor of psychology at San Francisco State University. And he's also the owner and head instructor of the East Bay Judo Institute in El Cerrito, and Dr. Matsumoto served as the head coach of the 1996 Atlanta Olympic judo team, was the team leader for the 2006 Sydney Olympic judo team, and he holds a seventh-degree black belt. So I'll be asking very careful questions of Dr. Matsumoto.

(LAUGHTER)

FLATOW: Welcome to SCIENCE FRIDAY.

DAVID MATSUMOTO: Thank you very much for having me.

FLATOW: What is a microexpression? Explain that to us.

MATSUMOTO: Well, I think the best way to explain what a microexpression is is to first describe what a macroexpression is, a normal facial expression of emotion. So when we are with our friends or - close friends or family or just even alone, and we have an emotion, and it comes onto our face, it usually stays on our face from a half to four or five seconds for a single episode of an emotion. And that's a normal emotional expression, the normal duration of an expression for one single episode. And we call those macroexpressions.

Microexpressions, therefore, are expressions that occur much more quickly than that, less than half a second. We've clocked them as fast as a tenth or a fifteenth of a second, as you've mentioned. But these are very quick, fleeting signs of an emotion that show up on a person's face and go off very quickly, obviously, so that most people either don't see them, or even if they see something, they think something happened, but they don't know exactly what it was.

But that's what a microexpression is, it's a very quick, fleeting, less-than-half-a-second expression that comes onto - comes and goes onto your face really quickly.

FLATOW: Are they significant, though, these tiny little expressions?

MATSUMOTO: Well, I guess that depends upon what you mean by significant. We believe, and there is some data to suggest, that they are signs of the fact that you are having an emotion, but you're trying to control it. So it may be significant, or - I think you mean meaningful.

FLATOW: Meaningful, yeah, meaningful.

MATSUMOTO: So it would be meaningful if you're in a situation where you are talking with a person and it's very emotional, but they're trying to control their feelings and control what they're showing to others, and you might want to pick those things up. And so in that case it might be very meaningful.

FLATOW: In law enforcement interrogations, things like that.

MATSUMOTO: That's true. Now I mean in everyday life, they're probably not meaningful in that sense, because I think humans have been around for a long time, depending upon which theory you want to believe, whether it's 100,000, a couple hundred thousand, or a couple million. It's very clear that we can get by without reading microexpressions.

We need to read the other expressions, however, macroexpressions. And I think societies would not function if we did not read each other's macroexpressions correctly. But with the microexpressions, you know, it's very clear that humans have done well without reading the microexpressions. So in terms of everyday social life, probably not that meaningful.

FLATOW: If you have a question, you're welcome in the audience to step up to the microphones here. Would a macroexpression be something we used to call body language?

MATSUMOTO: Well, body language is a term that refers to the entire body. So it would include your gestures, your hand movements, your postures, the movements of your head as well as your facial expressions, as well as your tone of voice, paralinguistic cues. So all of that together it what we would call body language. So facial expressions would be one specific channel of the many channels of body language.

FLATOW: So you can train people, then to recognize these microexpressions?

MATSUMOTO: Yes, we have developed ways in which we can train people to see these things. Most individuals can take as little as 30 minutes or an hour...

FLATOW: No kidding?

MATSUMOTO: ...to get to a very basic level of being able to see them. Of course like any other skill that we have, you'd have to keep practicing to keep it up. But yeah, within an hour or an hour and a half, maybe, most people can be able to see them.

FLATOW: Are they cross-cultural?

MATSUMOTO: We believe - well, macro facial expressions of emotion we know are cross-cultural, that they're universal, that people all around the world, regardless of race, culture, ethnicity, nationality, gender, religion, sexuality, et cetera, et cetera, will produce those same facial expressions on their faces when they're emotional. So we know that for macros, that is the case, that they're universal.

There's really no data - or I take that back. There's really little data to indicate that the microexpressions are cross-cultural. But because the macros are cross-cultural, we believe that they are, that micros are cross-cultural, as well.

FLATOW: What would be some examples of a microexpression?

MATSUMOTO: Well, it's hard to describe them on the radio.

(LAUGHTER)

MATSUMOTO: But if you take - even if you take a smile that everybody knows, if you take a smile that we all know, if you can just imagine that going on and off your face in a tenth of a second, that's what a - it just, it's a flash of the expression. And if you don't know what you're looking at, it'll just look like something changed. It looks like, if you're watching video, that there's a video glitch in the video.

FLATOW: A jump in the video.

MATSUMOTO: Yeah, a jump in the video. And so it could be a smile, it could be a grimace, it could be an expression of anger or an expression of disgust. Whatever you see in the full face in normal, everyday life, they'll occur in microexpressions.

FLATOW: Is there - and you train people to see these in interrogations. Is there a danger that people can overly use them and put too much weight on what they see in a microexpression and think they can conclude something from that?

MATSUMOTO: Well, you're touching on a very important part about what one does with that information. Reading microexpressions and reading any aspect of nonverbal behavior or anything in the words, for example, they're just cues to things that are possibly going on in a person's mind. And everybody we train and everybody we talk to, we always say that if you are going to be using these kinds of skills in an interview or in an interrogation...

FLATOW: Right, I wish I had those skills.

MATSUMOTO: Well, I'm sure you're doing just fine.

FLATOW: (Chuckling)

MATSUMOTO: If you're going to use those skills, that you just use them as investigative or interview aids to help you follow up in a way in which you can then ferret that information out. What you don't want to do is jump to conclusions based upon these things.

FLATOW: Being an expert in this, do you just look at these now when you look at people, do you just observe their microexpressions? You're looking at me as we're speaking, you're saying oh, he's got this and this or this and this going on, you know? Or can you - does it automatically just come to you like that?

MATSUMOTO: Yes.

(LAUGHTER)

FLATOW: OK, let's go to the audience. Yes, sir.

UNIDENTIFIED MAN #1: Yes, hello. I believe there's a company in San Diego that's commercializing this with computer processing so that you don't have to go through so much training or whatever. Is it - are there commercial possibilities for this? And I assume if it's easier to use because somebody has processing to help them, there's the chance for misuse, as well. Could you comment on that?

MATSUMOTO: Of course. There are a number of entities that we know of that are trying to develop ways in which to automate the ability to recognize facial expressions in general and then microexpressions in particular. They are at various different places and stages of their development. If such a tool existed, I'm sure there are some applications - there are many applications of that.

You mentioned commercial in your questions, and I would imagine there are commercial applications, as well. I know that the - where the field is right now, that field is not at the place where things are yet commercializable or yet reliably producing those kinds of findings, but I'm sure it's just a matter of time.

FLATOW: Thank you. With so many cameras photographing us for Homeland Security and in other work, are they being trained, the cameras themselves or the programs that run them, to recognize facial expressions that might stand out and what...

MATSUMOTO: Well, I'm not exactly sure what's going on in the cameras that are out in the field that's being used. I'm sure that if a reliable application existed, that might be one of the kinds of potential applications in field areas in which it could be used.

FLATOW: But you're not aware of a reliable one that does exist.

MATSUMOTO: That's correct.

FLATOW: That's right, well...

MATSUMOTO: For microexpressions.

FLATOW: For microexpressions. But macroexpressions?

MATSUMOTO: Well, there's several, you know, technologies that do relatively well for macroexpressions. But, you know, they are at the stage of development where you've got to have a person that's not moving around, kind of sitting in front of a camera, not moving their head too much, with certain type of lighting.

And so when you talk about the difference between a laboratory situation and a field setting, there's a big difference there in the technology's ability to detect the face and to detect the expressions.

FLATOW: Yes, sir?

CARL HEWITT: Yeah, Carl Hewitt, iRobust. It seems to me that there's a non-deceptive use of microexpressions, namely an infant at learning and their significant others where the significant other needs to give feedback to the infant extremely rapidly, rapidly shifting between positive and negative and using microexpressions to convey the information that the infant needs to learn the language or to learn the task. They're paying intense, intent attention to their significant other when this going on. They're not doing it by themselves.

FLATOW: Do you know about that?

MATSUMOTO: Well, I would like to comment a little bit. I believe that you're entirely right about the function that communicating expressions well to significant others would play. I don't think that that's - that would be the microexpression that we're talking about, however. It could be subtle expressions or other kinds of expressions...

HEWITT: But you're on this tight time constraint to give very rapid feedback.

MATSUMOTO: But microexpressions are going to be things that we don't have voluntary control over. These are things that are leaking out despite our attempts to control our expressions.

HEWITT: That's one interpretation.

MATSUMOTO: That's one interpretation. So in the situation that you're describing, it may be difficult, under my interpretation, for a person to create the micros in order to achieve that function.

FLATOW: OK, we're going to take a break. When we come back, we'll talk more about these expressions with David Matsumoto, director of the Humintell company and professor of psychology at San Francisco State University in California. We're looking for questions from the audience. If you have any, please don't be afraid to step up to the mic. We'll be right back after this break.

(SOUNDBITE OF MUSIC)

FLATOW: This is SCIENCE FRIDAY. I'm Ira Flatow coming to you from Stanford, California. We're talking this hour about macro- and microexpressions with my guest David Matsumoto, director of Humintell and professor of psychology at San Francisco State University in California. Why do we have these? Are other primates, do they have these expressions also?

MATSUMOTO: That's a really good question. There is no data that I know of of other primates showing microexpressions. Other primates certainly have macroexpressions. Non-human primates actually have the same facial muscles that humans do for emotion signaling. Humans have a few additional muscles, especially in the lower face, mostly for speech articulation, which non-human primates don't have.

So we know non-human primates do signal emotions in their faces well. But whether they have micros is not known. I think that - you asked where they come from. You know, I think that really they're the product of this interaction between the subcortical areas of our brain, which controls our emotions, which is ancestrally very long with us and shared with our phylogenetic ancestors, the product of the interaction of that plus the cortex, which humans have, for controlling our expressions. And so since human cortexes evolved to have that, I think the microexpressions then became a byproduct of the fact of that evolution.

FLATOW: What is the pathway that the expressions get from our brain into our face? How do they get there?

MATSUMOTO: OK, well so there's two major pathways that - there's one major nerve in the brain stem that enervates all of our - most of our facial muscles.

FLATOW: Just one nerve for all the muscles?

MATSUMOTO: Just one nerve for most of the muscles.

FLATOW: Most of the muscles.

MATSUMOTO: Most of the muscles. And so there's a pathway, a neural pathway, from the subcortical areas that goes to that nerve that says fire the face when we're emotional - as well the rest of our physiology. And then there's another pathway from the cortex in the motor strip that goes to that nerve that says control our expressions in a situation we want to control it. So there's two pathways going to that thing, and it's times of conflict of those two impulses that the microexpressions are leaking out.

FLATOW: Oh, I get it. Yes, yes, ma'am.

BREN: Hi there, my name is Bren(ph). So I was wondering, if you can train people to read microexpressions, could you train people to better control their microexpressions?

MATSUMOTO: That's a great question. You know, we don't know. I don't know the answer to that. We've never been able to do the study to figure that out. So some of us in the field have considered whether that's possible or not. I think probably not because as I mentioned, under my interpretation, the micros are involuntary reflexes that are occurring in the situation where the parts of your brain are at conflict with each other in the first place.

And so to know when that's occurring and then know, to be able to voluntarily override that probably is a really difficult thing to do. But we don't know.

FLATOW: She's sort of asking the lie detector question, you know, could you control a lie detector.

MATSUMOTO: Or could you train somebody to beat a lie detector is really what I'm hearing the question to be.

FLATOW: Right. Is it the sort of same kind of conflict that goes on here, the same kind of little electrical signals like that might happen on a lie detector?

MATSUMOTO: Oh, of course. If you're in that situation where you're emotional, and it's hot and heavy, and the questions are tough, but you've got to control what you're doing, it's the same like when you're on the box or when you're at a hiring interview or anytime you've got that kind of context.

FLATOW: So people who are in HR, human resources, and hiring people, they get trained to watch the people that they're interviewing?

MATSUMOTO: I think they could be trained to do that, and I'm sure - you know, there are people who are interviewers who do that naturally to begin with. I'm sure that if they learn to get additional data, it would help their interviews, certainly.

FLATOW: Now I mentioned all your judo experience on the judo teams. Do you use microexpression training on your judo teammates?

MATSUMOTO: Well, I've got to tell you during training and competition, there's not too many things that are micro.

(LAUGHTER)

MATSUMOTO: There's just a lot of pain and grimacing and anguish.

FLATOW: But can you use those facial expressions to know the state of your opponent?

MATSUMOTO: Of course, and when you're - because when you're in competition, athletes are trying to control their feelings as they're in the middle of combat. And as a coach, being able to read what's going on in your opponent and then communicate to your athlete what's going on, it helps a little. It also helps when you're communicating with your athlete afterwards, you know, and you see brief glimpses of things that are occurring on their faces. It gives you that additional insight as to what other things are going on in their mind that they may not be in touch with so that you can help them achieve peak performance.

FLATOW: Could you also tell them that - you're telegraphing your emotions to your opponent, you know?

MATSUMOTO: Sure, and that's one thing you could do, and the reason why that becomes a little difficult is when you have these - this kind of skill, what you don't want to do is what the character in that show did, which is go and call people out and say oh, you look like this, or you're doing this.

FLATOW: So you don't believe in that show, that that really works?

MATSUMOTO: Well, I don't think that people like being told what they're doing on their faces. And the character in the show did that. And it's part of the drama, and I understand that. But I think in everyday life, people won't want to go and be told that oh, you've got - you just did a micro of disgust. I want to know what you're - why you're feeling that.

(LAUGHTER)

FLATOW: You know, they talk about - you know, you play poker and that there are tells that people have. Is that a microexpression or a macroexpression?

MATSUMOTO: Well, it depends - I mean, the microexpressions that we're talking about are little things that occur in the face. But poker tells, as far as I understand, I'm not an expert on poker, you know, are nonverbal behaviors that can occur anywhere, whether it's the way that a person shuffles their cards or handles the chips or raises their brow or all those other things.

FLATOW: Yeah, I was looking for a little help here, but I'm not going to - for my weekly poker game.

MATSUMOTO: We can talk later on.

FLATOW: Let's go to the audience here, yes.

DANTE SIMONE: Hi, I'm Dante Simone(ph). I was wondering how many microexpressions do people do, like, every hour or every minute or et cetera?

FLATOW: How many do you think you're doing now?

DANTE: Thousands.

FLATOW: Thousands, OK. Is he close? Can they go by, that many that fast?

MATSUMOTO: Well, probably people are not doing thousands per minute. And we don't have any data about that. No one's ever done a study where they've just counted, you know, the number of times that people generally have different types of expressions in normal, everyday life. And again with - I mean so with regular expressions, they're always on our faces, and most of the time we don't know what we're doing on our faces.

So with regular expressions, they're always there, and yeah, that gets to be a large number. But with micros, again these are very special classes of behavior. These are things that are occurring in an emotional situation because you're trying to control yourself. And so I think unless you're in that situation all the time, you won't see them as much.

FLATOW: And you would not have any reason to learn them, and the average person has no reason to learn the microexpression.

MATSUMOTO: Learn to read them?

FLATOW: Yes, to read them.

MATSUMOTO: Yes, I believe that's true.

FLATOW: Yes, unless you had to. Thank you.

DANTE: Do I have any on my face right now?

(LAUGHTER)

MATSUMOTO: I can't tell because you've got that big mic right in front of you.

(LAUGHTER)

MATSUMOTO: You look like a good guy.

FLATOW: There you go.

MATSUMOTO: There you go.

DANTE: Thank you.

FLATOW: Yes, step up.

JANE: Hello, I'm Jane(ph). I was wondering if you could tell us a little bit of the lingo, how you describe a microexpression. I'm thinking of description of courtship in birds, where they talk about head up and tail up and all those sorts of cute little words. What do you say?

MATSUMOTO: Well, the description of a microexpression or the morphology of expressions is the same as any facial expression. So if it's disgust, for example, I'd say that there's the nose wrinkle, or you're lifting your upper lip. Or if it's anger - or if it's surprise, it would be the brows are up, the upper eyelid is raised, the jaw is dropped. So it's those kinds of behavioral descriptions on the face. But those are the same behavioral descriptions that you'd see in a macroexpression. The only thing that differentiates is the timing. It's just that those things go on and off the face incredibly quickly.

FLATOW: And the timing is important because there's a conflict, you're saying, going on there about trying to hide something?

MATSUMOTO: Yes or trying to conceal your expressions. Whether you go - whether you make the interpretation that the person's hiding something other than their feelings is the next level, another level.

FLATOW: Speaking of the next level, in the couple of minutes we have, where would you like to take this research to the next level? What would you like to know or need to know?

MATSUMOTO: Is this the blank check?

FLATOW: Yeah, you can have the blank check, sure.

(LAUGHTER)

MATSUMOTO: I think that there's so much to learn still about expressions, facial expressions, microexpressions in particular but facial expressions in general and how we communicate. We know a lot about words, and we know a lot about faces, but what we don't know a lot about is how we create thought and how the thought, implicit or explicit, gets translated into these words that come out of my vocal cords and how other thoughts and feelings then get elevated to these nonverbal expressions and what's the relationship between all of that in my mind as it's occurring.

That complex system we don't have very much information about, and it's a really basic science question about how thoughts, words, emotions all are put together and expressed. And I think I would love to be able to capture that somehow.

FLATOW: Let's see if I can get one final question.

UNIDENTIFIED MAN #2: So we go through airport screening quite a bit when we travel, and so we're training TSA personnel to check us when we're sometimes very fatigued. Maybe we've lost a loved one. We're traveling to the funeral. And so how do those types of emotions affect someone's microexpressions - the fatigue, death in the family, divorce, all the things?

MATSUMOTO: Right. Forgot whether you turned off the lights in your car in the parking lot, forgot whether you turned off the stove at home before you went to the airport, et cetera, et cetera. There are many things that make us emotional. And I think - this is related to what we talked about earlier.

Everybody that we train, we always say that learning the skill - when you read the behavior, it's just a clue that something may be going on, and you have to talk to people to figure it out. And when you talk to people, you - and you can guide your questions with a well-thought-out plan of an interview or questions - you figure it out. And so then you ferret out whether - what's the situation there and hopefully make good determinations.

FLATOW: Thank you very much, Dr. Matsumoto. It's terrific to have you.

MATSUMOTO: Thank you.

FLATOW: David Matsumoto is director of Humintell and professor of psychology at San Francisco State University. Transcript provided by NPR, Copyright NPR.