Holidays Enough

As usual for when I'm out of town one or more of the computers are running — fetching mail, sometimes monitoring the house via webcam, that sort of thing. During the break a curious thing seems to have happened: the rate at which I was receiving spam actually started to exceed the rate at which Eudora could accept mail. At least the way I had Eudora set up. The result was that Eudora crashed sometime after Christmas and when I checked Botzilla again on New Year's Eve, I had close to 80,000 messages awaiting receipt.

It was more than my Eudora could handle at that point, and it just croaked when trying to load that many messages. Or maybe some message was killing Eudora. Either way, I couldn't proceed and by my estimation I was receiving another message on the server every second or two. So I did two things:

1. Made my mail handling on MUCH more strict about sending undefined mail into /dev/null

2. Switched my mail handling at home from Eudora to Apple Mail.

The switch was precipitated by Apple's famous anti-spam techniques, which Courtney's been using for some time. I found that they were at first ineffectual but as the collection of junked messaged grew it became very effective indeed — easily hitting the promised 98% tag rate. Of course, 98% of 10,000 messages still leaves 200 spams in the inbox, but 9800 dismissed messages about cut rate pharmaceuticals.

Unfortunately, looks like it tends to bog down when mail boxes get really large — including the "Junk" mailbox. With some 20,000 junked mails under its belt, did well on filtering but was reading very slowly. So I followed some errant web advice and set the Junk mailbox to start clearing itself. Big mistake! The mail-reading speed increased, but the spam-filter success rate when right into the floor again. So I spent another good chunk of a morning hand-junking mails, until the filter was back up to fighting strength.

It's still not done. At the time I started this post still had 35-40,000 emails left to process, after a long full day's run. Oh joy.

Comments on "Holidays Enough"

January 4, 2005 06:21 PM

In my previous job I had around 10.000 emails a day. Regularly and legit. It was madness. Then I went on a two week holiday... you can guess the rest. Anyway, I found that Apple Mail would slow down after a while, so I would quit the app and start it again, which sped up things considerably.

January 4, 2005 10:45 PM

It got to the end! After a 30+ hour run it read the last 68,000 or so mails. And then DIDN'T DELETE THEM FROM THE SERVER. So I went in as root and deleted the whole $%^%!!! mail file myself. Since then it's been deleting okay, the server must have croaked on the massive request for deleting individual mails (Berkeley unix "mail" and "pine" both also refused to read such a large email spool file, they'd crash during an fseek()).

So I probably lost mails during the last 30 hours or so (tried to search for "Subject" headers in the tail of the mail file, but they were still mostly spam). Ugh.


All content on is 1994-2017 by Kevin Bjorke. All Rights Reserved.