Blind Testing discussion - split from Quad Lamp7 thread

[DaveF]

Long term listening seems to me to be the way to evaluate audio for many reasons not least because it averages out the many variables that can affect our auditory perception & we are listening to the system naturally, not in some enforced, analytical way.

Yip this goes along pretty much with my suggested test above where I emphasised that it be done on a system that the person really knows well for a long time. I went further with the software script idea so as to eliminate any possible influence from other people in the room as Ivor touched on in his post above. With the script a person could do it several times over so as to average out the variables as you said above John.
What I'm basically trying to do is to get to a situation that gets rid of any forces that might skew or spoil the test. Basically some form of unsighted test in the most natural way possible for the listener. I'd be hoping for some constructive responses on ideas or methods as opposed to the "go do it yourself Dave" response I got above. ;-)

[jkeny]

So, to continue....
Our sense of hearing is not a measuring device nor a simple translator of eardrum vibrations into brain signals - it is a pattern matching, prediction engine that maps the vibrations occurring at the ear to objects in the physical world.

The goal of our audio systems is to produce a believable illusion of a a real world audio event. So, in my estimation, it's job is to provide as best it can, signals which our auditory system is happy to accept as coming from a real instrument, real singer, real orchestra, etc. In other words it will go through the same processes that it does with any audio stream & use all the techniques it has to provide a best guess analysis of the signals.

We don't know the full workings of our auditory system & therefore we don't know how to build audio systems which will provide the necessary signal streams that will best convince us the illusion is real. Some audio systems get some parts right & others get other parts right - there is no system that gets it all right - it may not be possible with 2 channel audio, who knows?. So the situation we are left in is that because no audio system gets it completely right we are forced to choose aspects which we consider of importance for the portrayal of realism - maybe sound stage, maybe dynamics, maybe timbre, etc. & choose the system that has the best mix of these attributes.

It's no wonder there is such a difference of opinion between people

There's been a great deal of controversy about audio since digital audio was introduced & it's a good thing because I think it reveals an underlying truth. Digital audio gave us the ability to produce audio that had no equivalence in the physical world - things like pre-ringing in digital filters where the ripples of sound that usually occur after a sound were now occurring before the sound - it's like a bell knew it was going to be hit & started to vibrate before it was hit. That and other types of distortions that cannot occur in nature were & are possible in digital audio.

I believe this causes issues with our auditory perception as we have no stored model for these distortions, no previous experience built up for them & s a result we are like new-born babies hearing this for the first time without a frame of reference to analyse it with - a tiring & confusing situation. This is happening at the subconscious level & we are not conscious of it per-se, just of it's repercussions. Perhaps this is why people report an unease & tiredness with some digital?

Anyway, back to blind testing........

[jkeny]

Long term listening seems to me to be the way to evaluate audio for many reasons not least because it averages out the many variables that can affect our auditory perception & we are listening to the system naturally, not in some enforced, analytical way.

Yip this goes along pretty much with my suggested test above where I emphasised that it be done on a system that the person really knows well for a long time. I went further with the software script idea so as to eliminate any possible influence from other people in the room as Ivor touched on in his post above. With the script a person could do it several times over so as to average out the variables as you said above John.
What I'm basically trying to do is to get to a situation that gets rid of any forces that might skew or spoil the test. Basically some form of unsighted test in the most natural way possible for the listener. I'd be hoping for some constructive responses on ideas or methods as opposed to the "go do it yourself Dave" response I got above. ;-)

Yes, Dave, it's not a bad idea - it's basically long-term blind testing so it gets rid of a lot of the issues with short-term, analytical style listening for differences.

I wonder can an easy script be done which would randomly & seamlessly flip between different playback software while also recording your choice against which pb software was used?

[DaveF]

Good post above John. I certainly wouldnt disagree with any of that.

Perhaps this is why people report an unease & tiredness with some digital?

Electronics and equipment aside for one moment, I would think a lot of that comes from the mastering of the recording too. Very easy for the mixer to make a dogs dinner out of a digital recording. The options for vinyl are more restrictive which is why in general they are not as bad. That said, some of the really good classical recordings on CD from the likes of the Jordi Savall or Naive labels are some of the best I've heard.

[nige2000]

lol
starting to sound like a lot of stressful work
probably much of the reason why attempts of pfm style blind tests arent particularly conclusive

just for fun
30 second test track playback per (unknown)device (meridian cdp, lampi4 ,lampi 7)
guess the source
all done and dusted in 10 min stress free

only reason i suggest it is because i believe the differences are large enough not to require exact level matching or long term listening

[nige2000]

Perhaps this is why people report an unease & tiredness with some digital?

i think this seems to ease with better timing and power supplies

[jkeny]

Good post above John. I certainly wouldnt disagree with any of that.

Perhaps this is why people report an unease & tiredness with some digital?

Electronics and equipment aside for one moment, I would think a lot of that comes from the mastering of the recording too. Very easy for the mixer to make a dogs dinner out of a digital recording. The options for vinyl are more restrictive which is why in general they are not as bad. That said, some of the really good classical recordings on CD from the likes of the Jordi Savall or Naive labels are some of the best I've heard.

Yes, indeed, it seems mastering can be done with much more abandon (& much less knowledge) than was possible in analogue days - the loudness wars for example

[jkeny]

Perhaps this is why people report an unease & tiredness with some digital?

i think this seems to ease with better timing and power supplies

Yes, perhaps this a major source of some of the unnatural distortions in digital audio? Jitter & the distortions it creates are often not harmonically related to the primary signal itself & hence an unnatural distortion.

But, in general, digital audio is getting somewhat better in this area anyway - generally less unease & tiredness than was evident a number of years back & almost universal when digital audio was first created

[Diapason]

only reason i suggest it is because i believe the differences are large enough not to require exact level matching or long term listening

I know you don't want to get into science here, but this is a good example of the problem with these discussions! You have stated yourself that this test is predicated on your (personal) belief that the differences are large enough. To be honest, that renders the test pretty much useless, since if we're going to start by assuming exactly that which we're trying to prove, there's no point in doing the test at all.

If I take the opposing point of view (that the items all sound identically the same) then if people hear differences/have preferences I can easily claim that it's due to different levels. And so we go on and on and on as people on the internet have always done.

Part of the problem is that the opposing sides disagree on what's "self-evident".

[nige2000]

i thought its quite simple either they can be correctly identified or they cannot