Audio Quality Quiz Results: You Did Slightly Better Than Guessing Randomly

Last week, we published a quiz testing listeners on their ability to discern between lossless audio and compressed mp3s. We picked six songs from different eras and genres: an early digital recording of a Mozart piano concerto, an a cappella version of a pop song, the billionth song ever sold on iTunes, the most-streamed song of 2014 and two songs from musicians who happen to own digital music services. We cut a short clip from each song and then offered that clip three ways: as a 128 kbps MP3 file, a 320 kbps MP3 file and a higher-quality lossless WAV file. Then we asked you to guess which one was the lossless file.

These three options, in aggregate, formed a rough approximation of the different audio qualities that popular streaming services offer. The 128kbps MP3 file is close to what Spotify and others offers in their standard, free service. The 320kbps MP3 file is close to what Spotify and others offer with their paid service. The lossless WAV file is close to what Tidal offers with their $19.99/month HiFi service.

As users took the quiz, we collected anonymous results* of the quiz so we could see how our audience performed. The high-level summary: On average, our users did slightly better than guessing randomly. As a whole, users correctly identified the lossless WAV file 36.0 percent of the time. Guessing randomly, because there were three choices to choose from, would have users identifying the WAV correctly 33.3 percent of the time. Given the size of our sample, we can say that the difference is statistically significant. So congratulations, you are better than a random number generator at identifying lossless audio, but only slightly.

When we break down the results by individual song, not much changes. Users performed about the same for every song — between 35-37 percent of users chose the WAV file, 32-34 percent chose the 320kbps MP3 and 29-30 percent chose the 128kbps MP3. Given the diversity of our song choices, this was surprising. A common audiophile complaint is that modern pop music is so compressed that high-quality audio does not make a difference. However, Neil Young's "There's A World" and the digital classical recording performed just the same as Katy Perry and Jay Z.

There was one anomaly. More people chose the lowest quality version of Coldplay's "Speed of Sound" than the two higher quality versions. 35.6 percent of users chose the 128kbps MP3, 33.7 percent chose the WAV and 30.7 percent chose the 320kbps MP3. We chose "Speed of Sound" for its historical significance — the song was the 1 billionth song downloaded from iTunes — but perhaps it was not the best song for testing audio quality. It was the only song of the six to perform this way.

Overall, the majority of users failed this quiz. 29.7 percent of users got two right, while 25.8 percent got three right. Only 1.6 percent of users correctly identified all six WAV files, and only 4.5 percent got five correct.

At the end of the quiz, we asked a bonus question to determine what people used to listen to the quiz. A user's setup is just as important as a user's ears or the quality of the audio file, and we wanted the user's device choice to inform our feedback at the end.

Over half of users who answered the question reported that they used headphones plugged directly into their device. One in 10 users reported using an amplifier or DAC (digital-to-analog converter) in their setup. And one in five users used their built-in device speakers, which is a really poor way to hear audio quality.

So what does this all mean? It is hard to make too many conclusions — this wasn't the most scientific of tests — but it seems clear enough that most listeners have trouble discerning between lossless audio and a high-quality mp3. Regarding Tidal's HiFi option or Neil Young's PonoPlayer, our quiz results suggest that most people would not be able to hear the difference in the higher quality audio they offer.

If you haven't taken the quiz yet or want to try it again and do better, you can try it below. But first, here are some tips:

• Get the best audio equipment you have available. Try a good pair of headphones, external speakers you can plug into your device, or an amplifier or digital-to-analog converter if you have one.
• Turn up your volume and keep it consistent. It will be easier to hear the subtle differences in audio quality if you're listening to it at a loud volume, though if you've got low-quality headphones or speakers, the improvement will be limited. Also, make sure you keep the volume consistent throughout the test, as it is easy to mistake increased volume for increased quality.
• Listen high and low. The easiest places to hear differences in audio quality is in the low end and the high end of the frequency spectrum. That means focus on the bass in "Tom Ford" or the breathing in "Tom's Diner".

* All user data was collected through Google Analytics, a tracking platform that we use across NPR. Everything we collected came to us anonymously, and we have no way of seeing individual quiz results or knowing anything about an individual who took our quiz. We can only see data in the aggregate.