20.2 What is blind testing? Non-blind? Double-blind?
Description
This article is from the rec.audio.* FAQ, by with numerous contributions by Bob Neidorff others.
20.2 What is blind testing? Non-blind? Double-blind?
If you want to compare pieces of equipment, recordings, or
people, you could run an experiment. You could select an
experimenter to initiate various trials, select some subjects
to listen to the sounds, and then ask the subjects questions
about what they hear. However, if you want meaningful
results, it is necessary to set up the experiment correctly,
and ask the right questions.
One of the major problems with any experiment is that the
subjects may become aware of the experimenter's hypothesis
and allow this awareness to influence their behavior. One
technique for preventing such bias is to keep the person
who conducts the experiment unaware of the hypothesis of the
research. Unfortunately, experimenters invariably form SOME
hypothesis of what's going on, and these hypotheses affect
how they deal with subjects.
A more reasonable solution involves allowing the experimenters
to know the true hypothesis but somehow keeping them ignorant
of the specific experimental condition of each subject. This
is known as a Partial Blind Experimenter technique. An example
of this is that the person running an experiment knows that the
main experimenter wants to determine which connecting cables
are best at signal carrying, but would not know which cables
are being used at any given time during the experiment.
It is also important for subjects not to become aware of
the experimenter's specific hypothesis. Subjects often
become highly responsive to any cues, intended or unintended,
in the research situation that suggest what they are supposed
to do to appear normal or "to make the study come out right."
This problem can be present in judgment experiments,
particularly those in which each subject is exposed to more
than one variation of the stimulus. Such a procedure, by its
very nature, increases the probability that the subject will
begin to guess which aspects of the experiment are being
systematically varied by the experimenter.
Many studies avoid this problem with what is called a Blind
Subject technique. Using this approach, subjects are not told
specifically what the hypotheses are. Additionally, subjects
are not told what specific experimental conditions they are in.
For example, a subject might be told that he/she is supposed to
determine which stereo system sounds better, when in fact the
experimenter wishes to examine which color or appearance of the
same components looks better to subjects.
When both a Partial Blind Experimenter technique and a Blind
Subject technique are used at the same time, this is called a
Double Blind experiment. Double Blind experiments have higher
probability of producing statistically valid results than
Partial Blind Experimenter alone, Blind Subject alone, or other
techniques. Double Blind experiments are highly recommended.
Continue to:
My Books
