[Elecraft] Sherwood on ARRL Testing Methodology (LONG!)
Ed Rodriguez
erodriguez26 at tampabay.rr.com
Sun Sep 30 19:31:04 EDT 2007
Thats is why new is not always the best, I keep my ft 736r for 6 meters and
my knwd ts 950sdx, loaded with all the options for Hf.. and I hope the K3
will out preform them all. as I expected with the FT 2000 which was a flop
for me..
Keeping my fingers crossed, on the K3 , I am on the third wave....Bill
thanks for your input..really appreciated...
de
Wp4o, Ed
----- Original Message -----
From: "Bill Tippett" <btippett at alum.mit.edu>
To: <elecraft at mailman.qth.net>
Sent: Sunday, September 30, 2007 7:08 PM
Subject: [Elecraft] Sherwood on ARRL Testing Methodology (LONG!)
>
>
> Rob Sherwood gave his permission to post this on the
> FT-2000 list. Since that is public information I am posting
> it here also. BTW I agree with his comments on the "new"
> methodology.
>
> 73, Bill W4ZV
>
>
> What has gone wrong with the ARRL's new Product Reviews in QST?
>
> For several years I participated as part of a group of hams who were
> trying to improve the testing of radios reviewed in QST. Several reviews
> in the past had included questionable data on receiver performance. A
> sincere effort was initiated to correct testing problems within the lab,
> and hopefully to also improve the "hands on" portion of the report.
>
> As time went on, however, it appeared to me that the group had become
> fixated on minutia, and at the same time the League was unwilling to look
> at real problems in new radios being offered to the amateur radio
> operator. I have not contributed lately in the steering committee, as I
> felt I was banging my head against the wall.
>
> Some of the nonsense coming out of the League has been around for a long
> time, like the following quote from the 2004 review of the Icom IC-7800.
> "I was able to hear calling US stations on back scatter that I don't
> believe I would have heard on the '930." Did the reviewer bother to turn
> on his TS-930? No, he just assumed he was hearing something unusual on
> the receiver being reviewed (or hyped) in QST.
>
> Has anything improved in 2007? The latest October review of the FT-2000D
> (200 watt version of the FT-2000 that was earlier reviewed in February)
> states the following: "Why would I need a 200 W transceiver? After using
> it for a while, I was quite impressed with the extra punch the '2000D
> offered during routine CW and SSB contacts compared to the 100 W version."
> How could anyone tell a difference of 3 dB, especially compared to
> operation of the FT-2000 eight months ago? This kind of reporting is
> drivel.
>
> What has changed in the ARRL reports?
>
> Some of the changes are of minor interest, like measuring the noise figure
> of an HF radio. Noise figure is generally used by VHF and UHF
> enthusiasts, but adding these data points certain hurts nothing. Is noise
> figure, or noise floor, or sensitivity a significant issue in today's
> receivers? The 75A-4 has an excellent noise floor, as reported in the
> January 2006 QST Annual Vintage Issue. Few of us have such a quiet
> location that atmospheric and galactic noise don't overshadow the noise
> floor of a modern receiver.
>
> What we did get was additional confusion in the ethereal world of
> third-order intercept (IP3), in place of real dynamic-range data. The
> League used to measure it one way, then a second way, and now three ways.
> Is this supposed to be helpful? The old way (measured at the noise floor)
> was acceptable. The second way referenced an imprecise S5, now defined
> as -97 dBm, and a third new way at 0 dBm. Zero dBm is really strong,
> something we don't likely ever see, unless we are working Field Day or
> Multi-Multi contests from near-by transmitters. (I am assuming we are not
> living in Europe with their 5 megawatt AM broadcast transmitters.)
>
> 0 dBm is S9 + 73 dB, assuming any S meter reads that level accurately.
> (The Flex 5000A would actually do that.) On my IC-781, 0 dBm reads S9 +
> 50 with 30 dB of internal attenuation, or something like S9 + 80 dB with
> the attenuators off, if the S meter would read that high, which is does
> not. What happens when you put two 0 dBm signals into an IC-781 at 20 kHz
> spacing? The IMD reads S9 + 18 dB. At 2 kHz spacing the IMD reads S9 +
> 60 dB! The 781 is not a radio with performance problems, so what do these
> new and improved measurements really mean?
>
> If you look at the FT-2000 chart for IP3 at 2 kHz with the preamp off, you
> see the IP3, measured at the noise floor, is -19 dBm. This is not a good
> number, particularly since a Yaesu radio with "IPO" enabled (no preamp) is
> similar to most other radios with the 10 dB attenuator enabled. Yet if
> you measure the FT-2000 at 0 dBm, the IP3 calculates out to +15 dBm, which
> sounds good. This new information is meaningless at best, or misleading
> at its worst. Why is the IP3 so high at 0 dBM? Because the
> inter-modulation is so strong (S9 + 60 dB) the AGC has basically turned
> the gain of the radio off.
>
> Most operators will run an FT-2000 with preamp 1 enabled, since it gives a
> reasonable noise floor, sensitivity and AGC threshold. Yet no information
> is available with this typical setting for the newly touted IP3 reporting
> method, which at 2 kHz would be about -30 dBm for the League's sample.
> (The FT-2000 I measured was considerably worse.) To get a meaningful
> dynamic-range number, the reader now has to subtract two numbers. Why is
> this important data now missing, or at least obfuscated? Could it be the
> big advertisers in QST didn't like seeing 2 kHz dynamic-range numbers that
> are typically around 70 dB? Only the League could take a measured 2 kHz
> dynamic range of 69 dB at 2 kHz and calculate it into a +15 dBm intercept
> at 0 dBm. Talk about smoke and mirrors!
>
> The League is also going to differentiate between blocking (gain
> compression) and phase noise limited (a typical problem with synthesized
> radio). A narrow band audio spectrum analyzer is needed to measure
> blocking this new way. (I used this method on my Flex 5000A report
> because of the phase noise.) The ear is not going to hear what the
> analyzer sees, but the League may have made an improvement here. At least
> the two measurements now will be differentiated.
>
> What is the League completely missing?
>
> Most new DSP radios have serious problems in QRN, and with any kind of
> transient impulse noise. Has QST reported on these problems? They have
> not said a word. I gave a talk on this subject at the 2007 Dayton
> Hamvention, to try to point out that all is not well in the current state
> of radio design. The IC-7000 is a prime example of a radio that is nearly
> useless in QRN, as is the FT-2000. Every DSP-chip based radio designed in
> the last few years has an AGC problem to some extent. Fast rise-time
> noises are improperly handled by the AGC, drastically exaggerating the
> impulse noise.
>
> I recently finalized an AGC test, using an HP fast-rise-time pulse
> generator. It basically approximates an electric fence. The generator
> was set for one pulse per second. The rise time was < 10 nanosecond, with
> a duration of around 1 microsecond. The level was set to 1 volt peak, to
> propagate a pulse well into the HF spectrum. The first radio tested with
> this new method was the FT-2000, with preamp 1 enabled. This produces a
> rather typical CW noise floor of -124 dBm, an SSB sensitivity reading of
> 0.3 uV, and an excellent AGC threshold of 1.3 uV. With a reference non-DSP
> IC-781 that has similar specifications, the S meter on the pulse test read
> less than S1, barely moving the S meter. On the FT-2000 the impulse noise
> read S7, pulse after pulse after pulse.
>
> While many hams seem oblivious to these AGC problems, at least some
> operators are voicing their concern. I was pleased to hear from a new ham
> at a recent Colorado hamfest describe his observations on his IC-7000.
> Even though he had no past frame of reference from an analog radio, he
> noted how strangely his Icom reacted to the slightest click or tick.
> Merely turning on a light switch would kick his S meter up many S units.
> I noticed the exact same problem two years ago on all the DSP radios
> coming though my lab and ham shack.
>
> When I queried the League on their review of the IC-7000, saying they
> totally missed the AGC problems on transient noise and QRN, they simply
> said they listened to it in December when there was no QRN. What is their
> excuse this time on the FT-2000D? This radio had to have been evaluated
> during the summer of 2007 when there was plenty of thunderstorm static.
>
> What did the FT-2000D review happen to say about the dynamic range numbers
> with the different roofing filters? After giving a full paragraph to
> explaining why narrower roofing filters are usually helpful, the League
> simply said, "We noticed little difference in performance between the 3
> and 6 kHz roofing filers in any of the FT-2000s tested, though, at any
> signal spacing."
>
> On the FT-2000 data recently posted on my web site, the dynamic range
> actually dropped from 90 dB at 20 kHz with the 6 kHz roofing filter to 81
> dB with the 3 kHz roofing filter. At 2 kHz spacing there was minimal
> difference, 63 dB (3 kHz filter) and 61 dB (6 kHz filter). Dynamic range
> numbers in the low 60s are not acceptable for serious operators.
>
> Finally, one more bizarre comment from the "hands-on" QST reviewer. The
> u-Tune unit adds modest selectivity in the front end, and significant
> insertion loss, as seen by the degraded noise floor. Yet the reviewer
> found the u-Tune unit to be helpful "on 20 meters before the band closed
> with the u-Tune unit switched on." One wonders why a little added RF
> selectivity and 10 dB insertion loss would help when the band was fading
> out. If this statement is accurate, which I question, there is something
> seriously wrong with this radio beyond AGC and roofing-filter problems, a
> subject totally ignored by the review.
>
> When will the day come when the information in QST is more than a fluff
> review, and a free multi-page advertisement for the manufacturer?
>
> 73,
> Rob Sherwood
> NC0B
> Rev C1
>
> _______________________________________________
> Elecraft mailing list
> Post to: Elecraft at mailman.qth.net
> You must be a subscriber to post to the list.
> Subscriber Info (Addr. Change, sub, unsub etc.):
> http://mailman.qth.net/mailman/listinfo/elecraft
> Help: http://mailman.qth.net/subscribers.htm
> Elecraft web page: http://www.elecraft.com
More information about the Elecraft
mailing list