The Famous Big 5 Personality Test Might Not Reveal The True You

What kind of person are you?

That's the question that a personality test called the Big Five seeks to answer. You respond to a series of statements about yourself – everything from "I have a kind word for everyone" to "I get chores done right away" – by agreeing, disagreeing or being neutral. Your final score gauges you on a quintet of characteristics: Openness, Conscientiousness, Agreeableness, Extroversion and Neuroticism (or Emotional Stability, depending on which version of the test you take).

Why these five traits?

Starting in the 1940s psychologists began asking people how they'd describe themselves or another person's personality. "They arrived at pretty much the same set of five dimensions," says Christopher Soto, a psychologist at Colby College, who studies personality traits.

The term "Big Five" was coined by a psychologist named Lew Goldberg in 1981, according to Soto, and came into common usage among psychologists by the 1990s. (You can try the test for yourself here.)

And it's not just pop psychology. Studies show that whether you are more conscientious and open or more extroverted and agreeable can influence the career you choose, how well you do in your job — and how much money you make.

"A lot of companies use this [test] for hiring decisions or for allocation of different workers to different types of tasks," says Karen Macours of the Paris School of Economics.

The Big Five test began in the United States and then spread to Europe. It's commonly used in what are known as WEIRD countries, an acronym that stands for "White Educated Industrialized Rich Developed."

In more recent years, it's been introduced in developing countries, administered by researchers and some development organizations to get a sense of people's skills in emerging labor markets.

For example, "the World Bank has a big study that looked into the match between what kinds of skills the employers are looking for and what kinds of skills the population has," says Rachid Laajaj, economist at Universidad de Los Andes, Bogota, Colombia.

But in a new study published Wednesday in Science Advances, Macours, Laajaj and their colleagues found out that something weird is going on with the test in non-WEIRD countries.

The researchers looked at data from previous studies that included online surveys as well as in-person tests conducted in 23 middle- and low-income countries and often conducted in the local language. They found that the test wasn't always a reliable measure of personality traits in these places.

One problem has to do with contradictory answers. The survey has some questions that measure the same thing. For example, "Is systematic, likes to keep things in order" and "Can be somewhat careless" both assess conscientiousness. Someone who thinks of themselves as organized and conscientiousness would "Strongly Agree" with the first statement and "Strongly Disagree" with the second statement. And that's what researchers have previously found in WEIRD countries.

But on analyzing all the data from the non-WEIRD countries, Laajaj and his colleagues found that people sometimes chose "Strongly Agree" to both those statements.

They also found that in many of these developing countries, people tended to agree more than disagree with the Big Five statements.

"This is what we call 'acquiescence bias,' or their willingness to agree more than disagree," says Macours.

"It happens more for some questions than others," says Laajaj. "And some people agree more than others. In countries with less education, they tend to agree more."

The researchers aren't sure why this happens, but it could be because people in these countries aren't used to taking these kinds of surveys or thinking about their own personalities in the terms described in the survey, speculates Laajaj. It could also be that if the question or concept is foreign, "it's harder to say no," he adds.

And someone who isn't used to taking such surveys, say a farmer in India, might think "What do these guys want from me?" or "How will this information be used?" he says. If you're already worried about the test, you might be less willing to disagree with its statements. And some people might agree because they want to look better, he says.

Then there's the issue of inconsistency. Sometimes the test didn't yield the same results when taken by the same person twice. For example, in Kenya, more than 900 farmers took the test two times. Some farmers were interviewed by the same person both times, whereas others had two different interviewers.

One would assume that each person would score similarly for the two tests regardless of who was interviewing them, says Soto, who was a consultant on the Kenya project, the results of which are used in the new study.

"The most surprising thing to me was if someone was interviewed twice by the same interviewer then their responses across the two tests were pretty consistent," he says. "But if they were interviewed by two different interviewers then their responses were often completely unrelated to each other."

So something about the difference in social interactions between the farmer and the two interviewers affected what people said about themselves, he says.

"It could be very mild differences in how these questions are asked" that influence someone's answer, says Laajaj.

There's another complication in the findings of the team. When people in developing countries took the online version of the test, their answers tended to be more consistent and mirror findings in the Western world. Perhaps that's because the anonymity of the online surveys is an incentive to answer truthfully, he says.

But he admits that the population of online test takers is likely to be more urban, more educated and better off than the ones taking the in-person survey.

The new study "forces us to take a step back and say 'OK if we want to use this stuff, we need to really think about why we're using it and then develop better tools for measuring it,'" says David McKenzie, the lead economist at the development research group at the World Bank.

That's obvious to Esther Ngumbi, born in a village in Kenya and now an entomologist at the University of Illinois Urbana-Champaign. She wasn't involved in this study.

"Duh!" she says. "You can't just copy and duplicate your experiment in different environments without really adapting to the local setting."

The environment is a key factor in all science experiments, says Ngumbi. And people's environments shape their behaviors and how they think about themselves.

"For example, I grew up in Kenya," says Ngumbi. "Now I live in the United States and my thinking and actions have changed because my environment has changed. Now I'm more conscious about time, because time is of the essence here."

So back in Kenya, Ngumbi might not have given much thought to questions about being punctual. But now being punctual is something she values.

"It's a fascinating and important study," says Ara Norenzayan, a psychologist at the University of British Columbia. He says the findings show that psychology studies created and tested in developed countries don't necessarily "travel well" to other regions.

In fact, Norenzayan and his colleagues coined the term WEIRD in 2010. They published a review paper titled "The Weirdest People In the World?" to call attention to the fact that the vast majority of psychology studies are done in rich Western countries with mostly white people.

"Ninety-six percent of psychology studies are in Western populations, which is something like 12% of the world population," he says.

Soto agrees. "As research psychologists we mostly work with samples that are easy to get for us, which is mostly because they self-select themselves to do things online or students at colleges and universities, not just in the U.S. but around the world," he says.

That's why educated, middle-class and liberal people are over-represented in these studies, says Norenzayan, and psychologists often assume that their results apply to other parts of the world as well. "We often get into trouble with that assumption." And the new study illustrates why.

Copyright 2019 NPR. To see more, visit https://www.npr.org.