Blind Testing.....decisive and clear cut!!

[jkeny]

Blind testing seems such a logical & simple test to perform.

But as other here have already pointed out there are many variables to be controlled before any conclusions can be drawn from such tests. These variables are only ever attended to in truly scientifically conducted tests & I would suggest that none of the blind tests conducted on forums would qualify.

I will be interested in the upcoming blind test on PFM organised by Vital, not so much to see if differences are heard between DACs but if differences are heard between known differences. This is something that is the first qualifier for any test - prove that the test itself is capable of revealing differences of known, uncontested differences - for instance inverting the phase in one channel or changing the volume by 1dB, 0.5dB. Some calibration of the listeners & the test system itself is needed to establish what degree of differences are perceptible by the system before it is then used to test mildly perceptible differences in the devices under test - DACs in this instance.

So, hopefully, this PFM test will do some testing of the test before testing of the DACs & after testing of the DACs. The reason for before & after is to ascertain if the position of a test has some influence on it's outcome (I remember listening to the same track over & over in some of our tests & really by the end it took a lot more will power to really apply myself to the task - frankly by the end I couldn't care less, I just wanted it to stoppppppppp).

So before thinking blind testing is a useful method for teasing out small perceptible differences between devices, cables, etc. I would prefer to calibrate & qualify the test environment as suitable for doing such a test.

I find this equally as logical & simple as saying "blind testing makes sense & removes sighted bias"

[jkeny]

Isn't life strange. I posted this and the very next thing I saw was a link to the following article:
http://www.pnas.org/content/early/2012/ ... 9.abstract

I read this before but found the test to be flawed as a scientifically rigorous test. Again, at the risk of repeating myself, it is a good example of a test without any self-testing or calibration - no attempt at controls that might verify if the conditions that the violin players were subjected to (wearing welders goggles, having some disguise scent & playing in a hotel room) might interfere with their ability to differentiate.

I'm not a musician myself but I wonder if some musicians here might have a view as to whether their powers of dscrimination might be somewhat hampered by wearing welders goggles & smelling a perfume up close while trying to play in a room whose acoustics they were unfamiliar with?

[Ivor]

it's a pity humans have to be involved really.... bloody variables!

[jkeny]

it's a pity humans have to be involved really.... bloody variables!

That's it, Ivor & as you said before, it's pseudo-science unless these variables are controlled.
To my mind these sort of tests, being put forth as somehow superior to sighted tests is simplistic.
Eliminating one known variable - sighted bias, does not make a better test.

[jkeny]

Just as an example of how misdirected & pseudo-science these forum blind tests are - look at the latest post on the PFM thread I mentioned earlier, Vital's blind test thread http://www.pinkfishmedia.net/forum/show ... stcount=81

"At the first bake-off I thought I'd be able to hear a significant difference between the Sonos and Vega, at least sighted, but I couldn't."

The first bake-off was to "test" the if the premise "all DACs sound the same" was correct.

So this guy couldn't hear a difference sighted with two very different devices - he's the only one, I've seen, that has reported sighted results (I stand corrected 3 others DID report hearing sighted differences) - the focus was on the blind results which of course was interpreted as conclusive proof of the premise.

We probably all have heard one or the other of these devices & have an opinion on this. I have heard the Sonos & my opinion is that it sounds OK, not great. Don't think I have heard an Auralic Vega?

I have heard a number of DACs that sound different, when sighted. But when this view is expressed by anyone on PFM the stock quip is "did you do a blind test".

Ah, good ol' Maxi with his usual knee-jerk reaction - not a scientific bone in his body, yet he tried to espouse science in every PFM quip

[maxflinn]

I have heard a number of DACs that sound different, when sighted. But when this view is expressed by anyone on PFM the stock quip is "did you do a blind test".

Perhaps the new stock quip will be - did you calibrate the system/listener combination beforehand.

Ah, good ol' Max- not a scientific bone in his body

I'm not the one holding some irrational belief that USB cables sound different!

[jkeny]

I have heard a number of DACs that sound different, when sighted. But when this view is expressed by anyone on PFM the stock quip is "did you do a blind test".

Perhaps the new stock quip will be - did you calibrate the system/listener combination beforehand.

Well, Max, if you read this thread you should realise that an ill-conceived test is worse than no test because it leads people up the wrong path

Ah, good ol' Max- not a scientific bone in his body

I'm not the one holding some irrational belief that USB cables sound different!

Maxi, science begins with observation - you can try it - we can demonstrate different sounds of USB cables if you want to visit

[Ivor]

I'm not the one holding some irrational belief that USB cables sound different!

An awful lot of the beliefs held by hifi enthusiasts are irrational. Irrational does not equal wrong.

[jkeny]

Looks like I was correct in my diagnosis of a potential problem in the first blind test in Lenehan Audio - they have a ground loop in the system
http://www.stereo.net.au/forums/index.p ... try1079620

I went down Tuesday evening to have a listen. During the initial setup and volume matching one of the local DACs exhibited a very audible hum and nothing we tried would get rid of it.

[sima66]

I have heard a number of DACs that sound different, when sighted. But when this view is expressed by anyone on PFM the stock quip is "did you do a blind test".

Perhaps the new stock quip will be - did you calibrate the system/listener combination beforehand.

Ah, good ol' Max- not a scientific bone in his body

I'm not the one holding some irrational belief that USB cables sound different!

Just out of curiosity, how about the other cables (power, interconnects, speaker, etc)? Are they all sound the same?
Or, what cables do make difference and what don't?