Internet Explorer doesn't support the Web Audio APIs required to complete this test. To do the test, please download one of the supported browsers.
A p-value is the likelihood that the result obtained would occur by chance if there were no difference. That is, a p<.05 means that if you randomly selected A or B each time, there is a 5% chance that you could get the result you got or one more extreme (i.e. closer to 0% or 100% right).
In psychometric testing a result with p<.05 is often interpreted as the individual detecting the difference between samples.
Because the likelihood of getting one or more extreme results by chance increases when doing multiple tests (i.e. 1 sample = 5% chance, 2 samples = 2 x 5% chance = 10% chance), Bonferroni correction is applied to compensate. Bonferroni correction is done by dividing the cut-off (i.e. p<.05) by the number of samples (e.g. for 2 samples the cutoff becomes 0.05/2, thus the new cutoff is p<0.025)
This percentage is the likelihood that the result obtained, or one more extreme (i.e. further from 50% correct), would occur by chance if you were completely unable to detect a difference. For example, if you get 18 of 25 correct, then the percentage will be 3.2%. That means that if you were to choose the answer to 25 trials randomly, you would get 18 or more right, or 7 or less right just 3.2% percent of the time.
To say that you can hear a difference with any confidence this percentage typically needs to be less than 5%.
To say you can hear the difference for an individual track, the percentage is required to be even lower. This is because if you do multiple calculations (one per track) and each has a 5% chance of being wrong, then doing 5 calculations leads to a 25% chance of being wrong. To correct this, the cut-off is divided by the number of calculations: 5 tests = a cut-off of 1%.
Is your audio system really ready for lossless sound?
Recently, a lossless music streaming provider offered a quick online test to let you check if you and your equipment were ready to hear losslessly compressed music. There was just one problem.
The test didn't answer the question.1 2
To help you work out if you really can tell the difference between lossless and lossy3 music compression...
VALIDATE THE TEST- The Tidal test would tell you how many tracks you got correct, but that itself doesn't tell you whether you can tell the difference. The two-option forced-choice format used by the Tidal test means that even chance performance leads to the response being correct on average 50% of the time: precisely like guessing heads or tails for a coin flip. Further statistical analysis is required to determine whether you can actually tell the difference. The small number of trials (i.e. 5 tracks, once each) meant that a participant would need to get all 5 right answers correct for the performance to be considered statistically significantly different from guessing.
- Actually, there was another problem. The lossless and lossy samples used by Tidal were not perfectly in sync. This meant that when switching between lossy and lossless there would be an audible - and distracting - click. The files used in this test have been modified to correct the sync.
- According to the about page, the lossy encoding used by Tidal in the samples was AAC 320 kbps. However, it is not clear what encoder was used.
How does it work?
The Tidal test had you try to identify which of two versions of a track was lossless for each of five tracks.
This test will instead test whether you can tell the difference between the lossless and lossy version, without choosing which is which. It does this using an ABX test.
You will be presented with two reference samples (A and B), and a target sample (X). You have to decide whether sample X matches sample A or sample B. You will be administered multiple trials for each of the five tracks used in the original Tidal test.
The accuracy of the test will increase markedly as the number of trials increases. Although 5 trials is sufficient to estimate whether you can tell the difference between lossy and lossless, to work out which tracks you can tell the difference on will require 20 trials per sample4.
These files are not cached due to limitations of javascript FLAC audio libraries.
- The reason that you need to do more trials to work out which tracks you can tell the difference on is because of the multiple comparisons problem. The statistical cut-off typically used to decide whether something is 'real' - in this case, your ability to tell the difference between lossy and lossless music - is set so there is a 5% likelihood of thinking there is a difference when there isn't. In this case, that means that for each track there would be a 5% chance of thinking you could hear a difference when you in fact couldn't. When you then test five tracks, the change of that error happening for at least one track is 25% (5% x 5 tracks = 25%). To compensate for this, the metaphorical bar for determining that you can tell the difference for an individual track (in contrast to telling the difference overall) is set very high - five times higher. To reach that bar, you need to do more trials.
In this test, you will be presented with three samples: A, B and X. A and B are consistent, one lossless and one lossy. Each trial, X is randomly set to either A or B. You have to work out which one it is.
Start one of the samples by pressing the relevant button: A to start sample A playing, B to start B, and so on. Once the track is playing, you can switch between the samples by pressing A, B, or X.
You can seek through the tracks using the -5s, << (rewind), and +5s buttons.
Once you think you know whether sample X is matches sample A or B, enter your choice by pressing X is A or X is B.
Then enter your choice by pressing the Next button.
Your progress through trials and tracks is shown at the bottom.
All buttons also have hotkeys. For QWERTY and DVORAK users, you should orient your hand left hand with your little finger on the A key, and your right hand with your index finger on the 8 key
| Action | Hotkeys |
|---|---|
| Play or switch to sample A | A |
| Play or switch to sample X | X, S, or O |
| Play or switch to sample B | B, D, or E |
| Choose that X is A | Z or ; |
| Choose that X is B | C or J |
| Seek back 5 seconds | 8 |
| Rewind | 9 |
| Seek forward 5 seconds | 0 |
| Enter your response | Enter |