Mailing List CGatePro@mail.stalker.com Message #106126
From: Nicolas Hatier <nicolas.hatier@niversoft.com>
Subject: Re: New Server Hardware
Date: Fri, 22 Jul 2016 20:08:44 -0400
To: CommuniGate Pro Discussions <CGatePro@mail.stalker.com>

Yes, you're right.

In fact it mostly depends on the ability of the filesystem/OS to efficiently list directories containing thousands of files, and to quickly read each of them to get relevant headers (Subject, To, From, Date...). On a lot of filesystems, this operation is very slow, compared to scan the same information within a single (indexed) mbox.

Since it's possible to combine multiple mailbox formats within a single account, it's possible to get the most of each format by using mslc for common-use mailboxes (inbox, sent items), and mdir for mailboxes connected to an external tool like you describe. Archive-type mailboxes could also use mdir to help reduce the size of incremental backups - a longer load time can be more acceptable for those mailboxes than it would for INBOX.


Nicolas Hatier, ing. <nicolas.hatier@niversoft.com>
Niversoft idées logicielles - http://www.niversoft.com



On 2016-07-22 18:03, Bill Cole wrote:
On 21 Jul 2016, at 18:06, Nicolas Hatier wrote:

mdir has the issue of not being indexed.

In a sense, with mdir the filesystem IS the index. Each message is a single RC822-format text file with its name encoding the message's IMAP UID, multiple status flags, time of receipt (IMAP "INTERNALDATE") and line count.

Opening a large mailbox with several thousands of messages may take a while, it all depends on the filesystem.

Also on the ability of the OS to efficiently cache directory information, which is a mix of how much memory pressure there is, system tuning, and which filesystem is used.

All other operations are very fast. And it's very handy for incremental backups.

Also, because each message is in its own file in standard RFC822 format with a name that carries significant metadata, mdir is especially convenient for external tools. For example, I have a simple script that feeds messages into the SpamAssassin learning subsystem as spam when a user moves them into a folder with a particular name. Without mdir that script would need to figure out how to extract individual messages from mslc or mbox files and how to know which messages are the new ones.


Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster