Mailing List SIMS@mail.stalker.com Message #13820
From: <billc_lists@greenbuilder.com>
Subject: SIMS Content filtering with TOLD
Date: Tue, 7 Oct 2003 01:58:14 -0500
To: SIMS Discussions <SIMS@mail.stalker.com>
Hi All,

It's working!

I was able to get Trebor's Obnoxious Lunchmeat Detector (TOLD) working on incoming mail, with an assist from the StripMime MacPerl script. TOLD uses a point system to determine whether a piece of mail is likely to be spam or virus, and marks it accordingly so that it can be easily filtered by the mail client.

Still to do:

a) test the routing for common account names held by more than one domain hosted by the server, such as info@, webmaster@, etc.  I suspect that the account routing will need wildcards in the second line (ie, TOLD.account1*=<local>account1* ).  I haven't gotten my info@greenbuilder.com account to work yet...

b) figure out how to automatically launch the MacPerl script. I'm a Perl newbie.

Here's how to do it:

Requirements:
Mac OS 9
Memory: TOLD: 11mb MacPerl: 17.3 mb

1) download TOLD from http://www.madoverlord.com/projects/told.t . Install it in your SIMS folder instead of the EIMS folder as directed.  All other instructions apply.

2) download MacPerl from http://www.macperl.com

3) Create an account called TOLD on your SIMS.  Set it to save mail in a folder only.

4) In SIMS Folder, create a folder named Mail Folder.

5) In Mail Folder, create folders named Incoming Mail and Received Mail

6) Make an alias of Received Mail and place it in SIMS Folder:Accounts.  Rename it TOLD@F

7) In your SIMS Router, enter the following line:

<TOLD.*>=FilteredByTOLD@TOLD.* ; the part before the @ can be whatever you want.

After that, enter the following lines for each account that you want to filter:

<account1> = TOLD
TOLD.account1=<local>account1

<account2> = TOLD
TOLD.account2=<local>account2

etc.

You'll probably want those before most other router entries...

What the first of those lines does is route all mail for account1 to TOLD's folder.  TOLD monitors that folder, processes anything that lands there, and plunks it into the Incoming Mail folder.  TOLD simply adds some X-headers that identify potential spam (and if desired, the adds [SPAM] and [VIRUS] to the subject line of likely suspects).

The MacPerl script below monitors the Incoming Mail folder, pulls whatever lands there, and prepends "TOLD." to the account names on the SIMS server that the mail is being delivered to. (These are in the mail file's resource fork, so the stuff in the text body isn't changed). It then dumps the file in the SIMS Folder:Submitted folder for delivery by SIMS.  The first router line you added <TOLD.*>=FilteredByTOLD@TOLD.* changes TOLD.account1@yourserver.com to FilteredByTOLD@TOLD.account1@yourserver.com, and the second line you added for each account (TOLD.account1=<local>account1 ) changes *that* back to the original account name.

8) Create a text file with the MacPerl script below. (if someone here knows how to make that script into something we could drop into the Startup Items folder to launch it and MacPerl, please let me know. I'm completely new to Perl.)

9) Launch SIMS, TOLD, and the MacPerl script.  Built appropriate filters in your  mail client (see the TOLD docs).



Here's the script as it currently stands (watch for text wrapping by your email client!):

#!/usr/bin/perl
#
#Intent of this TOLDHelper script is to take a message from the Incoming Mail folder (after it's been processed
#by TOLD), prepend an identifier to the recipient address(es), and drop it in the SIMS Submitted folder.
#
#Portions of this code were derived from Stripmime by Alex Wetmore.
#BEGIN required Alex Wetmore acknowledgement
#
# (portions of) This code is Copyright 2000-2001 Alex Wetmore.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1.  Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2.  Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided with
# the distribution.
#
# 3.  All advertising materials mentioning features or use of this
# software must display the following acknowledgement:  This product
# includes software developed by Alex Wetmore.
#
# 4.  The name of Alex Wetmore may not be used to endorse or promote
# products derived from this software without specific prior written
# permission.
#
# THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS''
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
#END Alex Wetmore acknowledgement.

#Some portion of this script may also have been modified by Jason Simpson
#
#Minor tweaks added by Bill Christensen
#
#define delay, startup disk, sims directory, and folder to watch

$delay = 20; # number of seconds between folder checks
$startupdisk = MacPerl::MakePath(scalar(MacPerl::Volumes));
$simsdir = "${startupdisk}System Folder:SIMS Folder";
$account = "$simsdir:Mail Folder:Incoming Mail";
$submitfolder = "$simsdir:Submitted";

package main;
# start things going at the top level

use Mac::Resources;
use Mac::Memory;
use Mac::Types;
use Mac::Files;

print "TOLDhelper\n";

print "Monitoring $account every $delay seconds.\n";

$last_time = -1;

while (1) {

ScanMailbox($account, "$submitfolder");
if ((localtime)[2] != $last_time) {
print scalar localtime(), "\n";
$last_time = (localtime)[2];
}
print "Sleeping for $delay seconds.\n" if $debug;
sleepWNE($delay);

}

sub sleepWNE {
use Mac::Events;
my $delay = shift || 1;
my $gotime = time() + $delay;
while (time() < $gotime) {
sleep 1 if $WNE_OncePerSecond;
WaitNextEvent();
}
}

sub ScanMailbox {
my ($mailbox, $submitfolder) = @_;
chdir $mailbox or die "Couldn't chdir to $mailbox account folder";

$lockfile = "TOLDhelp.working";

if (-e $lockfile) {
open LOCKFILE, "<$lockfile";
my $unlocktime = <LOCKFILE>;
close LOCKFILE;
if (time() < $unlocktime) {
printf "lockfile $lockfile already exists, valid for %d more seconds\n", $unlocktime - time();
return 0;
} else {
UnlockMailbox();
}
}

LockMailbox();

my @waiting = <F.*>;

my $processed = 0;
printf "%s %d waiting file%s\n", scalar localtime(), scalar @waiting, (@waiting == 1 ? "" : "s") if @waiting;
foreach my $msg (@waiting) {
$processed++;
if ($msg =~ /\.out$/) {
print "#$processed: Skipping $msg\n";
next;
}

my ($returnPath, @recipients) = GetResourceData($msg);

print "#$processed: $msg from <$returnPath> to <@recipients> ";
my $outfile = "$msg.out";
unlink $outfile;
open MESSAGE, "<$msg" or die "Couldn't open $msg for reading";
open OUTFILE, ">$outfile" or die "Couldn't open $outfile for writing";
while (defined (my $line = <MESSAGE>)) {
print OUTFILE $line;
}
close MESSAGE;
close OUTFILE;

s/^([^\@]*)/TOLD.$1/ foreach @recipients;
print "=> <@recipients>\n";

AddResourceData($outfile, $returnPath, @recipients);

if (FSpCatMove($outfile, $submitfolder)) {
unlink $msg;
} else {
die "$^E";
}
sleepWNE(1);
}
UnlockMailbox();
sleepWNE(1);
}

END {
UnlockMailbox();
print "Quitting.\n";
}

sub LockMailbox {
open LOCK, ">$lockfile" or die "couldn't create lockfile $lockfile";
print LOCK time() + 60, "\n";
close LOCK;
}

sub UnlockMailbox {
print "removing lockfile $lockfile\n" if $debug;
unlink $lockfile;
}

sub GetResourceData {
my $file = shift;
my $res = FSpOpenResFile($file, 1) or die "$^E";
my $resReturnPath = Get1Resource('STR ', 8192) or die "$^E";
my $returnPath = MacUnpack('STR ', $resReturnPath->get());
my $resRecipients = Get1Resource('STR#', 8192) or die "$^E";
my @recipients = MacUnpack('STR#', $resRecipients->get());
CloseResFile($res);
return ($returnPath, @recipients);
}

sub AddResourceData {
my ($file, $returnPath, @recipients) = @_;
my $resReturnPath = MacPack('STR ', $returnPath);
my $lResReturnPath = length($resReturnPath);
my $resRecipients = MacPack('STR#', @recipients);
my $lResRecipients = length($resRecipients);

FSpCreateResFile($file, MacPerl::GetFileInfo($file), 1) or die "$^E";
$res = FSpOpenResFile($file, 0) or die "$^E";
$hReturnPath = NewHandle($lResReturnPath);
$hReturnPath->set(0, $lResReturnPath, $resReturnPath);
$hRecipients = NewHandle($lResRecipients);
$hRecipients->set(0, $lResRecipients, $resRecipients);
AddResource($hReturnPath,'STR ',8192, 'Return-Path');
AddResource($hRecipients,'STR#',8192, 'Recipients');
UpdateResFile($res) or die "$^E";
CloseResFile($res);
}

# end script
--
Bill Christensen
http://greenbuilder.com/contact/

Green Building Professionals Directory: http://directory.greenbuilder.com
Sustainable Building Calendar: http://www.greenbuilder.com/calendar/
Green Real Estate: http://www.greenbuilder.com/realestate/
Straw Bale Registry: http://sbregistry.greenbuilder.com/
Books/videos/software: http://bookstore.greenbuilder.com/
Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster