Mailing List CGatePro@mail.stalker.com Message #96223
From: Christian Ellsworth <k@ellsworth.cl>
Subject: Re: Your experiences - MBOX vs MDIR
Date: Wed, 24 Sep 2008 08:56:15 -0400
To: CommuniGate Pro Discussions <CGatePro@mail.stalker.com>
Rob@Logan.com wrote:
> please ZFS groupies stay out of this one...

why? mdir vs mbox is filesystem dependent... a modern
filesystem will index mdir faster than an indexed
mbox.

% time ls -l > /tmp/t ; wc -l /tmp/t
0.26u 0.66s 0:00.94 97.8%
   48565 /tmp/t

48k messages in one folder indexing @ 51,664.89 msg/sec

how does your box compare?

            Rob
What i run?

a small RHEL4 box, only 1GB ram, 3 years old CPU, with IDE disks no raid (old and crappy 40GB disks), using the slow but trusted ext3... using mbox indexing

~ time wc -l Kernel_devel.bdx
63103 Kernel_devel.bdx

real    0m0.007s
user    0m0.004s
sys    0m0.002s


and there are 63k+ messages on the mbox folder.

you might say that's cheating... but the index files contains something way similar as a mdir format info you will get from a ls -l..

TEXTINDEX 63101 391510743 20080616T161629Z
MSG 3 86576 S_____________ 20070611T212349Z 54 58 2490
MSG 3 86577 S_____________ 20070611T212659Z 375 2549 13978
MSG 3 86578 S_____________ 20070611T212708Z 103 14037 18014
MSG 3 86579 S_____________ 20070611T212943Z 67 18073 21292
MSG 3 86580 S_____________ 20070611T213037Z 68 21351 24740
MSG 3 86581 S_____________ 20070611T213533Z 96 24799 28965

that's the way CGP overcomes an OS filesystem limitation, still some filesystems are faster at large amount of small files, like ZFS, but that one is just one... hfs+,ext3, ntfs, ufs... are not that privileged...


my single cent today ;)

Christian Ellsworth


#############################################################
This message is sent to you because you are subscribed to
 the mailing list <CGatePro@mail.stalker.com>.
To unsubscribe, E-mail to: <CGatePro-off@mail.stalker.com>
To switch to the DIGEST mode, E-mail to <CGatePro-digest@mail.stalker.com>
To switch to the INDEX mode, E-mail to <CGatePro-index@mail.stalker.com>
Send administrative queries to  <CGatePro-request@mail.stalker.com>

Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster