[Dx-qsl] LotW fix?

[Joe Subich, W4TV]

The following is from Dave Summer, K1ZZ at ARRL.  I had been in contact
with him late Monday and early Tuesday forwarding may of the reports
from here concerning "lost" uploads (along with my own).

------------------------------------------------------------------------

Joe, thanks for your help.

The following explanation and apology is being distributed this
afternoon.

Dave


ARRL's Logbook of The World: Bug Fix is on the Way

The past few weeks have been a frustrating time for many users of the
ARRL's Logbook of The World (LoTW) QSO confirmation system. The problems
began with a 3-1/2 day outage during the first week of November - a
particularly busy time of the year for log uploads. When the system was
brought back on line a large queue of logs awaiting processing
developed, and processing times lengthened well past normal
expectations.

Another, much more subtle problem cropped up at about the same: some
logs began disappearing from the queue, apparently at random. Users had
been advised at the time the log was uploaded that it was "queued for
processing" and so they were understandably upset when the log was not
processed, even after the several-day delay that most logs were
experiencing.

Because of its random nature it took the ARRL IT staff a while to figure
out what was happening. When LoTW was designed more than a decade ago -
long before the present IT staff was here - an assumption was made as to
how many logs could possibly be in the queue at a given time. The
assumption was based on users uploading their most recent QSOs perhaps
once a week or once a month.  The environment in which LoTW now operates
is quite different from that assumption in that many users now upload
logs with small numbers of QSOs in them, almost in real time. This
creates a much larger number of separate logs.
When a log is uploaded it is identified by a file name that is assigned
by the user. Because there is no way to avoid duplication of file names
that are assigned in this fashion, the LoTW system renames each file.
Because of the unusual processing delay combined with the dramatic
increase in the number of submitted logs, the system began to run out of
unique identifiers for the log files. This resulted in a file sometimes
being renamed with an identifier that had already been assigned to a log
that was still in the queue, causing the earlier log to be overwritten.

Once the problem was identified, designing a fix was relatively easy. It
should be in place by 2359 UTC November 28. Because the number of
overwritten logs is relatively small, we have decided to keep the system
available for use even though this may result in a few more logs being
lost until the fix is in place.

We apologize for the inconvenience that users have experienced, and
especially for being unable to explain what was happening until now. We
want to emphasize that no data from processed logs has been lost. That
data is secure and backed up. If you have had a log disappear after it
was "queued for processing" the solution is to upload the log file
again, preferably after the bug fix is in place. We will announce when
that occurs.

73,
David Sumner, K1ZZ
Chief Executive Officer, ARRL
November 28, 2012


-- 

73,

   ... Joe, W4TV