[Dx4win] Wish list, again -- LOTW Import

[Randy Farmer]

Having just done a major cleanup of a log with around 55K QSOs, many 
of them imports from contest logs, I would like to revive the "Wish 
List" thread in hopes of influencing the development of future 
versions of DX4WIN. Specifically, there is room for major improvement 
in how DX4WIN handles data imports from downloaded LOTW reports.

About a year ago, I bit the bullet and decided to import all the 50K+ 
QSOs from my electronic contest logs, going back to 1989. When the 
import was complete, I uploaded the log (in several installments) to 
the LOTW server and was rewarded with around 4000 QSLs. I just did 
the import of the LOTW report ADIF files back into DX4WIN with the 
"Imported QSO sets Upload Flag" option set, and have continued to 
follow that procedure with succeeding updates ever since. This worked 
fine, and I have picked up lots of DXCC credits from this process.

A couple of weeks ago, I happened to be scrolling through the logbook 
window and realized that the log file was terribly cluttered with 
large amounts of redundant information in the Notes for this QSO 
field. I decided to see if I could clean up a lot of this stuff. To 
my horror, I found in excess of 3000 records that needed to have this 
material expunged.

What do I mean by redundant? If you look at what happens when the 
program imports a LOTW report, you'll understand. To give a near 
worst-case example, let's say I get a LOTW QSL for a QSO I made with 
my friend KU8E in the CQ WPX contest. Since the only exchange in WPX 
consists of a 599 and serial number, the original contest QSO record 
will have only the received serial number in the Received log field 
and the Notes for this QSO field will say "Freq=xx,xxxKHz, WPX CW 
YYYY". (This is the result of using AD1C's very nice ADIF2DXQ 
software -- thanks, Jim!). When I do the import of the LOTW record, 
assuming Jeff has provided all the possible information, the 
following will be appended to the text in the Notes for this QSO 
field: "LOTW: State=GA County=Harris Grid=EM72 CQZone<>5".

What's redundant here? Practically everything! The "LOTW:" part isn't 
needed, since the record import sets the Upl Cnf flag. The 'State=GA 
County=Harris" aren't needed, since the record import populates the 
State and County fields. Similarly, the Grid field is also populated, 
so the "Grid=EM72" isn't needed.

The "CQZone<>5" part IS useful information, given that the initial 
import of the contest QSO caused DX4WIN to populate the Zone field 
with a 4 based on the "8" in the call. Unfortunately, DX4WIN does NOT 
update the Zone from the imported data. This may not be all bad, 
since there are a good many LOTW users who mistakenly enter their 
IARU zone in the CQ zone box when they upload their log files and 
thus provide the wrong zone in their QSL record.

The details of the log changes are really good to have, but they 
certainly don't belong in the Notes for this QSO field. This 
information should instead be placed in a transaction log that is 
generated (or appended to) by the import process. Some sort of 
indicator in the log that there is updated data that conflicts with 
previously logged information is a good idea, but this would be much 
less intrusive than filling the log file with what amounts to a 
transaction log of the import process.

By the way, manually editing thousands of records in the log to get 
rid of the spurious text was simply too overwhelming to contemplate, 
as the program provides no tools to manipulate the notes field. What 
I ended up doing was to export the log in chunks as a .dxq file 
(choose File > Import/Export from the main window, and then select 
DX4WIN6 format > File > Export in the Import/Export Filters window) 
and use WordPad to edit the .dxq file.

Editing the .dxq file with WordPad is still a major pain, since the 
text following the "LOTW:" in each record is unique and you can't 
just do a search and replace to remove the unwanted text. But it does 
go much, much faster working with the text file to select and delete 
the spurious content from each record. After fixing the .dxq file, I 
then re-imported it into the original log file and let the imported 
dupes replace the existing records.

I broke the project into manageable chunks and finally got all the 
log entries fixed. It was a monumental job. You can bet I'll stay on 
top of it in the future.

Lest this sound like too much like whining, I want to be clear that 
I'm still extremely pleased with the comprehensive capabilities of 
DX4WIN. It does a great job of keeping track of award status and 
gives the user unparalleled flexibility in handling log data. But 
like any software, there's always room for improvement. In my book 
fixing the LOTW import process would be a great leap forward.

73...
Randy, W8FN