[Lowfer] WSPR noise bandwidth, was 74.5495 QRSS 60 as usual ...

[JD]

>>> I apparently missed the section in the User Guide that spelled out in 
>>> detail how WSPR goes about determining the noise level for the SNR 
>>> calculation. Please forward that information to me. Oh that's right ... 
>>> it's not in there.


In point of fact, I stated no opinion previously about how WSPR makes the 
calculation.  What matters to me is that it does calculate it over a wide 
range of frequencies that don't pertain to the signal as it gets decoded in 
the detection band.  Both of those numbers are in the guide.


If I say something that's factually incorrect, why not just quote the exact 
words and explain why it's wrong?  Maybe we could both learn something then. 
Mischaracterizing what someone says to portray him as too ignorant to bother 
with, then using that as an excuse to shut off dialogue, accomplishes 
nothing.


* Now--for anyone who's still interested in the technology of WSPR and not 
too gun shy about this thread's history to read on, let's see if we can 
learn something together.  Might as well start with noise detection.  I'm 
not trying to prove anything with what follows.  I've already made my point 
and I'm simply exploring WSPR along with quite a few other folks, though you 
will probably notice a few places where I emphasize something I've already 
said.  I'll apologize in advance in case I overdo that a bit too often.


At heart, WSPR is a propagation reporting protocol grafted on top of 
transmission and reception routines developed for WSJT--which I _do_ know a 
little about.  I have no interest in propagation reporting--that's just 
me--but I have been following WSJT with varying levels of interest since I 
first saw Joe Taylor's paper in the September-October 2005 issue of QEX.  I 
like to re-read it every so often to refresh myself on digital transmission 
concepts in general, and to try to better grasp some of the tougher aspects 
of coding and error correction that I haven't had to think about on a daily 
basis since retiring from broadcasting.  (Couldn't find my physical copy 
last night, so I rummaged around at K1JT's page at Princeton and found it in 
PDF through the References link [ 
http://physics.princeton.edu/pulsar/K1JT/refs.html ], along with some other 
papers of his I hadn't read before.)


Just as I thought I recalled, the decoding of WSJT ...and by extension, 
WSPR... starts with detecting the presence of signals through Fourier 
transforms of the digitized audio data (discrete transforms, as it turns 
out).  The reason for starting transmission at accurate times is, as you 
would expect, so that it will be easier to recognize which bins are falling 
into the known pseudorandom sequence that indicates the sync pattern.  The 
sync vector repeats so often that it occupies about half the transmission 
time in regular JT modes.  That's part of what enables the software to 
recover enough information after fades and noise for the error correction 
routines to extract a legitimate data payload.


The pseudorandom sequence distributes the transmitted energy nearly equally 
over the communication bandwidth, which is why it's legitimate to compare 
signal with random Gaussian noise.  And of course, Gaussian distribution is 
scalable for bandwidth AS LONG AS the randomness remains true.


JT modes do *not* attempt to make that true of received noise, however.  In 
fact, the detection routine expressly assumes everything in the background 
is random, and relies on that assumption to statistically analyze what it 
sees.  On average, a bin must contain 2 to 6 dB more energy than those 
around it to be considered signal.  But if you were to require that as an 
absolute threshold of every data tone, you'd miss a lot of them.  Instead, 
you can "help" the process along with knowledge of the properties of noise. 
As Dr Taylor put it:
"Many of the individual data tones may not be detectable above the noise. On 
average, however, in each tone interval the one frequency bin containing 
signal will have greater amplitude than the others. Using the known 
statistical properties of random Gaussian noise, WSJT computes the 
probability that a symbol was transmitted with each one of the possible 
values. This probabilistic information, based on measured spectra of the 
synchronized symbols, is the basic received information. After Gray coding 
and symbol interleaving have been removed, the probabilities are passed on 
to the decoder."  (The decoder being the set of routines where 
de-interleaving and error correction takes place.  The error codes chosen 
are so unique out of the possible combinations of received data, that if the 
bits come through at all, there's only the tiniest chance they will be 
displayed incorrectly regardless of corruption).



While we're talking about statistics and probability, it turns out 
probabilities play a role in WSJT's Deep Search mode, too.  I hadn't given 
that much thought before, but even when you are missing a lot of valid bits 
of the message, there's usually still a certain greater probability that an 
incomplete call sign or grid is this one rather versus that one.  Out of the 
calls that have previously been seen and reported to the database, one will 
be deemed to be more likely out of the bits that are available.  It's 
analogous to viewing a pair of dashes in Argo, followed by a dot-(broken 
dash)-(missing element of three dot length)-dot, and mentally concluding 
you've seen MP because (a) the next repeat contains missing information too 
but appears to fill in some of the pieces, and (b) you already know MP 
operates in that band.  If you allow Deep Search, it makes assumptions based 
on statistics in its database, and is relatively generous in its pattern 
matching.  Unlike the normal modes which require pretty strict 
all-or-nothing decoding, it is possible for Deep Search to produce false 
decodes sometimes.  (Based on something I witnessed myself a few weeks ago, 
I wonder if WSPR 2.0 has a form of Deep Search on by default.  Anyone know 
for sure?)



A final note to conclude this little venture into noise and WSPR.... Someone 
reminded me off-list the other day of the term SNRr to represent noise 
bandwidth from the receiver, regardless of the actual signal bandwidth. 
Personally, I like that much better than simply SNR, which used by itself in 
an engineering context implies at least _some_ type of bandwidth 
relationship between the signal and noise.  SNRr can be relative to whatever 
bandwidth you specify, and seems much more applicable to threshold detection 
comparisons in amateur modes.  It's been a long time since I saw anyone use 
the extra R, but it would certainly remove my objections to a supposed "SNR" 
value that is not intrinsically related to the signal.



Is there something else someone would like to explore about WSPR or WSJT?



73

John