Pop Quiz: 20 Percent Chance Of Rain. Do You Need An Umbrella?
This week, All Things Considered is exploring how people interpret probability. What does it mean to us, for example, when a doctor says an operation has a 70 percent chance of success?
One of the most common encounters with percent probabilities has to do with weather. Take a moment to consider the question below. It comes from a survey developed by scientists at the National Center for Atmospheric Research who were researching how people understand weather forecasts.
(Intrigued? We've included more of their survey questions below).
We put the question above to a few folks across the country, and many of them came to different conclusions. Some thought a 20 percent chance of rain means you should definitely bring an umbrella, while others said they would be surprised if it even drizzled. And at least one person looked at the question the other way: There was an 80 percent chance it wouldn't rain.
But ask a math professor, like Jordan Ellenberg at the University of Wisconsin, and you might get a different response.
"If I had an answer, I would be a famous philosopher instead of a mathematician," he says. "It's a very difficult question, and people have been arguing about it for a really long time."
Ellenberg is the author of How Not To Be Wrong, but on this question, it's evidently hard to be right. The 20 percent chance of rain forecast, he says, could mean a few things. Imagine you have a database of days where the weather conditions were similar. If you find 1,000 days and notice that it rained the next day on 200 occasions, you might say there's a 20 percent chance of rain.
Easy To Get Confused
Forecasts are less philosophical at the National Oceanic and Atmospheric Administration's National Weather Service, where Vankita Brown works. She's not a meteorologist, but a social scientist whose job it is to try to better understand how the public interprets weather forecasts.
Brown herself isn't sure how to interpret a 20 percent chance of rain.
"We think people know what it means," she says. "I have conversations with my colleagues in meteorology all the time about what that means. And, in fact, I challenged one today to tell me, in less than five minutes."
Brown walked away without a clear answer — confirming her suspicions that much of the public is also likely confused.
For the National Weather Service (NWS), of course, this is not a question to puzzle over for fun. It's serious stuff. They need the public to not only understand forecasts, but to have confidence in them so that people will respond appropriately to weather threats.
And it's not just a numbers game — words used to describe weather can be just as confusing. Take "watch" and "warning," for example.
" 'Watch' means that conditions are ripe for something to happen. 'Warning' means that it is happening — it is imminent," Brown says. "It's easy to get them confused."
Brown says the NWS has asked the public if other terms might make more sense. People they've surveyed have suggested words like "emergency," "imminent," "dangerous" and "caution."
Brown is also trying to figure out what works best to describe less dramatic forecasts. Should the weather service use descriptive terms, as in "rain is likely," percent probabilities, as in "70 percent chance of rain," or icons — like a picture of a cloud with a few raindrops?
"Honestly, I think all three," she says. "We find that alone, they don't work so well. But in tandem, they all work well to tell a story."
Confidence In Forecasting
Weather forecasting technology has gotten a lot better in recent years. It used to be that the fourth-day prediction in a four-day forecast was about as accurate as a fortune cookie. Now meteorologists can look almost a week ahead with some confidence.
The concept of confidence can be another important element of a forecast. Jason Samenow, chief meteorologist with The Washington Post's Capital Weather Gang, includes it in his forecasts. He may say: There's a 20 to 30 percent chance of evening showers and storms. Confidence: medium to high.
"The forecast models give you these probabilities, but obviously the further you go out in time, the less skill or accuracy these models have," he says. "Once we get to seven to 10 days, we have little to no confidence."
Samenow says his readers respond well to that statistical humility — and especially appreciate it during snowstorms.
"We've developed what we call 'boom and bust probabilities,' " he says. They "give people a sense of what we think the chance is [that] a snowstorm is going to overachieve or underachieve our best-guess forecast."
More information is power, he adds.
"I think the more information you can give people when the weather is complicated, or when the weather is highly impactful and has a potential to make a difference in someone's life and their decision-making, the better."
Given that weather forecasts can feel more subjective than objective, we want to know how you interpret them. If you selected "Other" for the questions above, share your reasoning in the comments section below.
(And in case you're dying to know, the technically correct answers to the first two polls above are option "C," according to the survey creators. The last one is truly subjective.)
This story is part of an All Things Considered series on Risk and Reason.
9(MDE1MTIxMDg0MDE0MDQ3NTY3MzkzMzY1NA001))
Transcript
ROBERT SIEGEL, HOST:
This week, we're considering how we interpret probability in everyday and not-so-everyday life. What does it mean to us when we're told in operation has a 70 percent chance of success or that there's a 60 percent likelihood that Osama bin Laden is hiding in a particular compound in Pakistan? Well, here's one of the most common encounters with percent probabilities. What does it mean that there's a 20 percent chance of rain? We put that question to a few folks on the streets of Phoenix and Milwaukee. Milwaukeeans Linda Gee and Paul Hess drew different inferences.
PAUL HESS: Well, 20 percent chance of rain - I think that's a reasonably high percentage - that you should prepare. You take a jacket or an umbrella or whatever.
LINDA GEE: If it's higher, I would think that it would rain. Twenty percent is not that much.
SIEGEL: In Phoenix, Erik Noland would batten down the hatches.
ERIK NOLAND: It means that it probably is going to rain.
SIEGEL: Chad Cashin and Angel Denison split the difference...
CHAD CASHIN: I think that means it's going to drizzle a bit in the area somewhere.
ANGEL DENISON: It may sprinkle here and there, and it might get a little humid. Might have some clouds. You may or may not.
SIEGEL: ...While Ahmed Khan, presented with a 20 percent chance of rain, did the subtraction.
AHMED KHAN: That means 80 percent is no chance.
SIEGEL: We put the same question - what does it mean to say there's a 20 percent chance of rain? - to a math professor.
HESS: Well, if I had any answer, I would be a famous philosopher instead of a mathematician. I mean, it's a very difficult question. And people have been arguing about it for a really long time.
SIEGEL: Jordan Ellenberg teaches mathematics at the University of Wisconsin. He wrote the book, "How Not To Be Wrong." But on this question, it's evidently hard to be right.
HESS: There's a few things it could mean. And I think customarily in weather prediction, one thing it means is that if you look at a big population of days where the conditions were similar to what they are right now - not exactly the same because no two days are the same, right? I mean, every day is a new thing. But let's say you sort of find a thousand days where the conditions that you can measure are close enough to the conditions obtained right now. And you find that out of those 2,000, on 200 of them, the next day was a rainy day. Well, that might be one thing you mean when you say there's a 20 percent chance of rain.
SIEGEL: It's a less philosophical question at the National Weather Service. Vankita Brown works there. She is not a forecaster. She's a social scientist whose job is to try to better understand what the public understands about the weather forecast. I put it to her. A 20 percent chance of rain - does she have a good understanding of what that means?
GEE: Absolutely not. (Laughing).
SIEGEL: You don't have a good idea of what it means.
GEE: No, and I work at the Weather Service. But I am not a meteorologist, and I am not trained in providing forecasts, so...
SIEGEL: But we hear something like that every day.
GEE: We do. We think people know what it means. I have conversations with my colleagues in meteorology all the time about what that means. In fact, I challenged one today to tell me in less than five minutes what does it mean.
SIEGEL: You were preparing for this conversation.
GEE: I was preparing for this conversation.
SIEGEL: And did the meteorologist have a very clear, concise...
GEE: No.
SIEGEL: No.
GEE: (Laughing).
SIEGEL: Of course, for the National Weather Service, this is not something to puzzle over for fun. It's serious stuff. They need the public to not just understand weather forecasts, but to have confidence in them, so that people will respond appropriately to weather threats. And they don't just use numbers. They also use words - words like watch and warning.
GEE: Watch means that conditions are ripe for something to happen. Warning means that it is happening. It is eminent.
SIEGEL: But Vankita Brown says, people get those two confused all the time.
GEE: It's easy to get them confused. They both start with W. They both start with W-A. So it's understandable that people are getting them confused.
SIEGEL: Is the Weather Service thinking of more words - preferably ones that don't begin with W-A, too - that would flesh out a - kind of a scale of thunderstorm or tornado information?
GEE: We are. Yes. Well, we're asking people what words would work for them. Instead of watch, instead of warning, they've offered things like emergency.
SIEGEL: Emergency.
GEE: Imminent, dangerous, caution - things like that.
SIEGEL: Perhaps take action. When it comes to less dramatic forecasts, Brown is also trying to figure out what works best. Should the Weather Service use descriptive words, as in rain is likely, or percent probabilities, as in 70 percent chance of rain, or icons - say, a picture of a cloud with a few raindrops?
GEE: Honestly, I think all three. We've been doing some research lately. We've hosted focus groups. And we find that, alone, they don't work so well, but in tandem, they all work well to tell a story.
SIEGEL: Kind of a mixed media presentation of the likelihood of, in this case, rain.
GEE: Absolutely.
SIEGEL: Weather forecasting technology has gotten a lot better in recent years. It used to be that the fourth day of a four-day forecast was about as good as a fortune cookie or a horoscope when it came to accuracy. Now we can look almost a week ahead with some confidence. How much confidence? Well, Jason Samenow, the chief meteorologist with the Washington Post's Capital Weather Gang, says, that varies. So he attaches a confidence level to his forecasts. Samenow may say, there's a 20 to 30 percent chance of evening showers and storms - confidence medium to high.
NOLAND: The forecast models - they give these probabilities. But obviously, the farther you go out in time, the less skill or accuracy these models have. And so our confidence in those probabilities tends to, over time, especially as you get out to three, four, five days in the future - tends to decrease. And sometimes with particularly complex weather set ups, our confidence in those probabilities are even lower. So, you know, for complicated snowstorms, we might have just a low to medium confidence of what the models are simulating within 12 to 24 hours. But typically what happens is we have fairly high confidence out to 36 to 48 hours, and then, gradually, our forecast confidence declines. And basically, once we get to seven to ten days, we have very little to no confidence.
SIEGEL: Jason Samenow says, his readers in Washington Post respond well to that statistical humility.
NOLAND: We've got a lot of feedback from readers that they appreciate having that information. And especially, it's true during snowstorms. And we've developed what we call boom and bust probabilities to give people a sense of what we think the chances a snowstorm is going to overachieve or underachieve our best guess forecast. I think the more information you can give people when the weather is complicated or when the weather is highly impactful has the potential to make a difference in someone's life and their decision-making the better. More information is power, in that sense.
SIEGEL: Most weather forecasts are not about blizzards, hurricanes and tornadoes. Lives are typically not riding on a 20 percent chance of rain. Tomorrow, the stakes are higher. How would you deal with a number that tells you the probability that a particular terrorist is in a particular place? Transcript provided by NPR, Copyright NPR.
300x250 Ad
300x250 Ad