[TheForge] searchable theforge archive draft design

[terry l. ridder]

hello;

since there is still confusion about what the searchable theforge
archive is i have put together this draft design document.

theforge mailing list has basically three 'epochs'.
0. wugate epoch
1. majordomo qth.net epoch
2. mailman qth.net epoch

how epochs relate to time.
0. wugate epoch                1995 - dec 1998
1. majordomo qth.net epoch dec 1998 - dec 2001
2. mailman qth.net epoch   dec 2001 - present

each epoch has distinct message formats which are processed very
differently.

for each epoch theforge messages will be sorted by month and year.
each message will have non-blank lines numbered.
example:
     0 date:
     1 from:
     2 to:
     3 subject:

     4 hello;

     5 this is an example.
     6 > forge_member at example dot com wrote
       >
     7 > what is a good make of tig welder?
       >
       --
     8 forge_member

each numbered line of a message will be inserted into a mysql database.
the table layout is roughly.
epoch month    year       msg_number line_number line_text
  0-2  0-11  1995-present   0-?          0-?      the line text

msg_number is the message received for a given month.

a search for a phrase would find those lines which match.
by knowing which lines match the phrase the original message can be
found and displayed.

options for searching the archive.
all epochs
a single epoch
a single month
a single year

poor ascii art

+--------+     +-------+     +-------+
| apache |<--->|  perl |<--->| mysql |
| httpd  |     |  cgi  |     |  db   |
|        |     | script|     |       |
+---+----+     +-------+     +-------+
     |-----------------+----------------+
     |                 |                |
+---+----+     +------+----+     +-----+---+
| wugate |     | majordomo |     | mailman |
| epoch  |     | epoch     |     | epoch   |
| msgs   |     | msgs      |     | msgs    |
+--------+     +-----------+     +---------+

-- 
terry l. ridder ><>