Mailing List Message #6676
From: Howard Shere <>
Subject: Re: Stall Problem
Date: Sun, 13 Aug 2000 21:31:24 -0500
To: (SIMS Discussions) <>
X-Mailer: Microsoft Outlook Express Macintosh Edition - 4.5 (0410)

>>We have turned on additional logging and we believe we have found the
>>problem. Below are log fragments around the time of the stall. See anything
>>15:19:47 4 SMTP-131( Input Stream ended
>>15:21:02 4 POP-787([]) Sending +OK 20746 messages in
>>15:13:08 4 POP-533([]) Input Stream ended
>>15:14:13 4 POP-633([]) Sending +OK 20746 messages in
>>This user seems to get a lot of mail.
> Um, yeah. And he is checking it every 7 minutes without deleting any messages.
> See how it is taking exactly 1:05 both times to do anything while it
> counts those messages? That is a sure sign that the machine is doing
> exactly the same thing and NOTHING else during that time.


> Welcome to MacOS's synchronous I/O.
> Without knowing the details of the SIMS code, my guess is that in this
> time SIMS is reading the entire account file and counting up messages. It
> is even doing a pretty darn efficient job of that. It may even be using
> asynch calls, but doing so in a fashion that with this size of a total
> read may end up looking like a synchronous read.

That is (more or less) the reason for my post. It should be possible to read
the data async in small chunks and keep everything else working.

>>The account file seems to be 1.3 GB (no, that is not a typo).
>>What can be done to make this work better?
> Get the user to actually delete their mail after reading. POP3 is not
> designed to provide IMAP-like service where most messages reside on the
> server until the user is permanently done with them. POP3 is designed to
> be primarily used to retrieve and delete massages, not retrieve and leave
> on the server.

In this case it is a user account for someone who disabled their account
with us a few months ago. The mail piled up. I am not certain why they
suddenly had an interest in popping their mail 'cause they couldn't send
mail out through our system from another ISP.

> SIMS *COULD* be more polite about this but at the cost of speed for that
> action (i.e. it could do small reads asynchronously and count the messages
> in a less direct way, but the result would be slower response to that
> client.) SIMS could also stash meta-information for the mailbox in a
> resource so that it doesn't have to directly count all the messages on
> every attempt to access the mailbox, but that would open a path to account
> file corruption that doesn't currently exist. What you are seeing is the
> Mac reading 1.3GB in 65 seconds, which equates to 20MB/s, probably very
> near the practical limits of your system. Short of a radical change in how
> SIMS works or a serious beefing up of your disk subsystems (like RAID
> striping the mailboxes across multiple LVD cards) you won't be able to
> speed this up much. SIMS isn't made to handle this sort of mailbox because
> the protocol isn't intended for such use, and I doubt that Stalker would
> revise SIMS in deep and potentially negative ways just to accommodate such
> a pathological case.

I would rather have the response be slightly slower overall than have any
case where a single operation can stop the server from serving anyone else.

> This user is very likely having serious trouble reading mail. The fact
> that it takes 65 seconds to get the initial response likely means that
> their client is timing out. The fact that they have not brought this to
> your attention makes me very suspicious that they may know a reason for
> their mail problems that they'd rather not discuss with you. Such events
> include things like spam runs that feed large numbers of bounces and/or
> complaints back at them. Since this is clearly causing problems with your
> system, I would think that this is sufficient justification to dive into
> that mailbox and see what all the volume is, and if you don't want to do
> this, you really need to talk to the customer via some means other than
> email, which is likely completely useless to them at this point.

The user is unlikely to be able to get their mail at all at this point since
the account file has been moved out of the accounts folder. We have a policy
of not looking or filtering the content of our customers mailboxes, but they
are not a customer anymore so they don't need a mailbox.

Some mailing lists are going to be very unhappy for a little while.

 Gridz 1.3 ---- ---- NetSpace will never be the same again
Howard Shere       |  Green Dragon Creations  |  Water Valley Interchange
President          |  301 N. Main St.         |  P.O. Box 70
Software Sculptor  |  Water Valley, MS 38965  |  Water Valley, MS 38965
                   |  |
                   |     |
                   |  1-662-473-4225          |  1-662-473-9209

Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster