[Boatanchors] e mail traffic

[mikea]

On Wed, Jul 13, 2005 at 04:12:22PM -0400, BSugarberg wrote:
> Hello Jeff,
> 
> I am curious - what is Echelon?

Google on (without the quotes) "NSA echelon". I doubt that much of it is   
hollow state, though, and so it's seriously off-topic.                     

As to the possible censorship of attachments, my job (for my sins, no
doubt) involves running the inbound and outbound mailfilters for a large
state government agency in the midwest. There are a lot of attachment types
that Just Don't Get Through, Period. I also run ClamAV to get rid of worms
and other executable nasties.

Once I've filtered those out, I scan the headers and body of each mail,
using pattern recognition techniques and about 12000 patterns in a
database, for "spamminess", where each pattern has a score attached to it,
and all the scores for matched patterns get added together. I also use
Bayesian recognition techniques, where earlier spam and ham (non-spam)
mails are broken down into tokens, each token having its own rating
(negative for ham, positive for spam) and stored in a database, and
incoming mail is tokenized and matched against the database contents.

The bayesian probability that a piece of mail is spam produces a score that
is added to the previous pattern-match score, and if the total score goes
over a threshold, we call the mail "spam"; otherwise, we call it "ham" and 
deliver it to our internal addressees.

Lots of folks nowadays are running similar filters or milters in their 
inbound mailstreams, because the spam and worm problems are so damnably
bad. That probably -- almost certainly, I'd say -- is where you are having
your attachments removed. 

The four IBM x335 servers I use for this are necessarily sand-state, with
140 gigabytes of Ultra320 SCSI disk, 2 Xeon 3.2 GHz CPUs, and 2 gigE NICs
each. I couldn't do it with hollow-state devices.

But I have an R-390 and 3 R-390A and some other hollow-state stuff, and
doing this provides a paycheck so that I can play with the glowbug stuff
and keep it lit, and so it's all relevant, see!

-- 
Mike Andrews, W5EGO
mikea at mikea.ath.cx
Tired old sysadmin