[Elecraft] Sherwood on ARRL Testing Methodology

[Don Wilhelm]

As measurements become more precise, it becomes more and more important 
to recognize what Bill is pointing out.  A slight difference in test 
setup can result in a different result.  These measurements are done at 
the sub-microvolt level and it does not take much to create a difference.

Test equipment must be calibrated, and the calibration tolerance should 
be known.  Traceable calibration is one thing, but the tolerance limits 
of that calibration are also important - not all calibration labs are equal.

Even with calibrated equipment and the same test setup, two different 
equipment operators may yield two different results.  As an example, 
consider an instrument having a display for readout (like an 
oscilloscope), the trace has a finite width, and one operator may place 
the cursor on the midpoint of a trace width while another may place it 
at one edge yielding two different values - how much they differ depends 
on the resolution used, brightness of the trace, scale illumination, how 
well the display was focused, etc.

One good step in the right direction would be to report the region of 
uncertainty for all measurements. For me, that is a piece of information 
that becomes more critical as the measured values become smaller.  The 
ARRL lab may do that calculation in-house (I haven't asked), but they do 
not state it in their published reports.

So for now, when I see comparison data between two receiver that vary 
only by a dB or so, I usually figure that is close enough to ignore the 
difference (I usually do consider 3 dB or more difference to be 
significant).

73,
Don W3FPR
 
Bill Tippett wrote:
> snip...
>         It's also dangerous to assume Elecraft's measurements
> will be identical to ARRL/Sherwood.  There are often differences
> due to different test methodologies, people and equipment. 
>