Mailing List CGatePro@mail.stalker.com Message #103281
From: Jonathan Weinraub <jonathan@weinraub.net>
Subject: Re: Can't get cgpav to work with cgpro
Date: Thu, 03 May 2012 15:59:23 -0400
To: CommuniGate Pro Discussions <CGatePro@mail.stalker.com>
X-Mailer: CommuniGate Pro WebUser v5.3.13
I also saw that it does scan outbound mail, as I also tried sending one to gmail, but it just said OK but no other errors were found in the log.

I don't know if this means anything or not, but looking at the mail headers, it appears to look non standard for mbox format.  Its the first from with the angle brackets that maybe is causing some issues.

simpson:/var/CommuniGate/Accounts/homer.macnt# head INBOX.mbox
From <>(S_____________-000000000001) 01-06-2011_18:19:18_
Return-Path: <root@springfield.com>
Received: by mail.springfield.net (CommuniGate Pro PIPE 5.3.13 _community_)
  with PIPE id 40003; Wed, 01 Jun 2011 14:19:18 -0400
Date: Wed, 1 Jun 2011 14:19:17 -0400
From: root <root@springfield.net>
To: marge@springfield.net
Subject: Test
Message-ID: <20110601181917.GA32450@springfield.net>
MIME-Version: 1.0



Looking at another mbox file appears to be normal (though it was an imported Linux mbox, not cgpro)

On Thu, 03 May 2012 15:10:09 -0400
 "Jonathan Weinraub" <jonathan@weinraub.net> wrote:
> Well, I tried sending it using mutt from the command prompt to
>myself, then I got the virus flag added.  
> Since most ISPs will reject the outbound email with even Eicar
>attached, I used webmail to send it to myself.  I guess that is what
>caused the rescan?
>
> 15:05:29.000 2 PIPE [130887] received in
>{Submitted/M13360719248310.sub}, 989 bytes 15:05:29.003 2
>QUEUE([130887]) from <root@there.com>, 989 bytes
>(<20120503190524.GA7882@me.com>) 15:05:29.004 4 EXTFILTER(cgpav)
>out(026): 875 FILE Queue/130887.msg\n 15:05:29.007 4 EXTFILTER(cgpav)
>inp(033): 875 ADDHEADER "X-Virus-Flag: Yes" 15:05:29.007 4
>EXTFILTER(cgpav) [130887] header added: X-Virus-Flag: Yes
>15:05:29.007 4 EXTFILTER(cgpsa) out(026): 877 FILE Queue/130887.msg\n
>15:05:29.017 4 EXTFILTER(cgpsa) inp(098): * 877 Processing CGP header
>line: R W 03-05-2012 19:05:29 0000 ____ _FY_ <jonathan@me.net>\n
>15:05:29.017 4 EXTFILTER(cgpsa) inp(067): * 877 Using default
>SpamAssassin settings for jonathan@weinraub.net 15:05:29.017 4
>EXTFILTER(cgpsa) inp(072): * 877 Processing CGP header line: P I
>03-05-2012 19:05:29 0000 ____ ____ 15:05:29.017 4 EXTFILTER(cgpsa)
>inp(039): * 877   <root@me.com>\n 15:05:29.017 4 EXTFILTER(cgpsa)
>inp(046): * 877 Return-Path: root@weinraub.fbyneserv.com 15:05:29.017
>4 EXTFILTER(cgpsa) inp(052): * 877 Processing CGP header line: S PIPE
>[0.0.0.0]\n 15:05:29.017 4 EXTFILTER(cgpsa) inp(039): * 877
>Processing CGP header line: O T\n 15:05:29.017 4 EXTFILTER(cgpsa)
>inp(036): * 877 Processing CGP header line: \n 15:05:29.017 4
>EXTFILTER(cgpsa) inp(037): * 877 Finished processing CGP headers
>15:05:29.018 4 EXTFILTER(cgpsa) inp(069): * 877 Running SpamAssassin
>with system default settings for 1 address 15:05:29.019 4
>EXTFILTER(cgpsa) inp(057): * 877 State directory is in system default
>home directory 15:05:29.019 4 EXTFILTER(cgpsa) inp(062): * 877
>  (/var/CommuniGate/Settings/SpamAssassin/.spamassassin) 15:05:29.148
>4 EXTFILTER(cgpsa) inp(064): * 877 Identified non-spam (1.2/5.0) for
><default> in 0.1 seconds
>
>
> On Thu, 03 May 2012 14:50:42 -0400
> Nicolas Hatier <nicolas.hatier@niversoft.com> wrote:
>>
>> From your log excerpt, we can't tell why your filter doesn't work,
>>as cgpav seems to remember message it already scanned and won't scan
>>them twice (inp(041): * 674 Previously-scanned message detected).
>>Please try with a different message, and send the log excerpt if
>>relevant.
>>
>> If you decide to purchase CGP-ClamAV, you won't need your existing
>>clamav installation, including clamd and freshclam.
>>
>> Regards
>> Nicolas Hatier
>>
>> On 2012-05-03 13:34, Jonathan Weinraub wrote:
>>>
>>> I actually was considering Niversoft. I already own their skin. Was
>>>also considering their winmail.dat converter too but was under the
>>>impression their filter was just a fork of what I'm using now   I'll
>>>give it a shot but I really like to know why the free one doesn't
>>>work. The very least to satisfy my curiosity...  So if I use Niver's,
>>>do I need to remove the daemons I have now, the freshclam, etc?
>>>
>>> *
>>> *
>>>
>>> *From:*CommuniGate Pro Discussions
>>>[mailto:CGatePro@mail.stalker.com] *On Behalf Of *Nicolas Hatier
>>> *Sent:* Wednesday, May 02, 2012 4:45 PM
>>> *To:* CommuniGate Pro Discussions
>>> *Subject:* Re: Can't get cgpav to work with cgpro
>>>
>>>
>>> I would say the answer is on this line:
>>> inp(041): * 674 Previously-scanned message detected
>>>
>>> Try again with another message.
>>>
>>> The last time I tested it, a few years ago, the cgpav+clamav pair
>>>had one issue processing CGP messages.
>>>
>>> First, a disclaimer, we sell a cgp antivirus helper which relies on
>>>the ClamAV engine but does not have the issue described, so this may
>>>sound like a sales pitch, and probably is, but this is still a real
>>>issue:
>>>
>>> ClamAV (clamd, clamdscan, etc), as installed by default, scans a
>>>whole file. There is magic numbers and detection methods in libclamav
>>>to determine the type of file to be scanned, and perform the correct
>>>extraction action to scan all parts.
>>>
>>> Unfortunately, the CGP envelope information prevents libclamav from
>>>correctly detecting the type of a CGP message. It identifies it as a
>>>plain mbox file and is able to do a shallow scan on it. However, if
>>>there was a virus embedded, for instance, in a zip file contained in
>>>a rfc822 mime part, libclamav wouldn't extract it and wouldn't be
>>>able to detect the virus.
>>>
>>> Due to its architecture, relying on the communication channel with
>>>clamd, cgpav has only one way to fix this issue - by making a copy of
>>>the message file to scan, without the cgp envelope information. I
>>>just re-checked the cgpav code and I didn't find any indication of it
>>>doing so, but I may be wrong. Nevertheless, if cgpav does copy the
>>>file, this means a performance hit on the processing.
>>>
>>> Also, having to perform type detection on the file is quite useless
>>>for a mail scanner as we should already know it's a mail file. Cgpav
>>>connect CGP with a "general-purpose" virus scanner.
>>>
>>> This said, cgpav is, as far as I know, excellent with SpamAssassin.
>>>
>>> We solved these issues and improved the virus-scanning performance
>>>by linking the clamav engine directly in our helper, and by modifying
>>>its entry points to use the correct mail scanning procedure without
>>>detection. No inter-process communication, no magic numbers involved,
>>>we got rid of the clamd/clamav client and just integrated the clamav
>>>engine directly in a dedicated CGP helper, CGP-ClamAV.
>>>
>>> I'm pretty sure other people on the list who run CGP-ClamAV would
>>>agree CGP-ClamAV is fire and forget, as the filter also automatically
>>>updates its virus database as soon as a new one is available. And
>>>it's not even expensive.
>>>
>>> Best regards
>>> Nicolas Hatier
>>>
>>> *Nicolas Hatier, ing.* <nicolas.hatier@niversoft.com
>>><mailto:nicolas.hatier@niversoft.com>>
>>> Niversoft idées logicielles - http://www.niversoft.com
>>><https://mail.weinraub.net/Redirect/www.niversoft.com>
>>>
>>>
>>> On 2012-05-02 16:07, Jonathan Weinraub wrote:
>>>
>>> I had setup cgpav and cgpsa on my web/mail server.  I got spam
>>>assassin working fine, just I can't get ClamAV to actually work.
>>> Well,  it works from the command prompt, but it doesn't work with
>>>cgpro itself,  it just says OK.
>>>
>>> See the below logs for reference.
>>>
>>> Any assistance would be greatly appreciated.
>>>
>>> Thanks.....
>>>
>>> 15:52:14.003 4 EXTFILTER(cgpav) out(026): 673 FILE
>>>Queue/130676.msg\n
>>> 15:52:14.008 4 EXTFILTER(cgpav) inp(006): 673 OK
>>> 15:52:14.008 4 EXTFILTER(cgpsa) out(026): 674 FILE
>>>Queue/130676.msg\n
>>> 15:52:14.016 4 EXTFILTER(cgpsa) inp(072): * 674 Processing CGP
>>>header line: P I 30-04-2012 19:52:14 0000 ____ ____
>>> 15:52:14.016 4 EXTFILTER(cgpsa) inp(038): * 674
>>><jonathan@myserver.net> <mailto:jonathan@myserver.net>\n
>>> 15:52:14.017 4 EXTFILTER(cgpsa) inp(045): * 674 Return-Path:
>>>jonathan@myserver.net <mailto:jonathan@myserver.net>
>>> 15:52:14.017 4 EXTFILTER(cgpsa) inp(093): * 674 Processing CGP
>>>header line: R W 30-04-2012 19:52:14 0000 ____ _FY_
>>><jon@myserver.net> <mailto:jon@myserver.net>\n
>>> 15:52:14.017 4 EXTFILTER(cgpsa) inp(062): * 674 Using default
>>>SpamAssassin settings for jon@myserver.net <mailto:jon@myserver.net>
>>> 15:52:14.017 4 EXTFILTER(cgpsa) inp(052): * 674 Processing CGP
>>>header line: S PIPE [0.0.0.0]\n
>>> 15:52:14.017 4 EXTFILTER(cgpsa) inp(039): * 674 Processing CGP
>>>header line: O T\n
>>> 15:52:14.017 4 EXTFILTER(cgpsa) inp(036): * 674 Processing CGP
>>>header line: \n
>>> 15:52:14.017 4 EXTFILTER(cgpsa) inp(037): * 674 Finished processing
>>>CGP headers
>>> 15:52:14.018 4 EXTFILTER(cgpsa) inp(041): * 674 Previously-scanned
>>>message detected
>>> 15:52:14.018 4 EXTFILTER(cgpsa) inp(006): 674 OK
>>> 15:52:14.018 2 QUEUE([130676]) enqueued
>>> 15:52:14.021 2 MAILBOX(jonathan/INBOX) {558} appended @4557186:
>>>59+1561 bytes
>>> 15:52:14.022 2 MAILBOX(jonathan/INBOX) [130676] stored as {558}
>>> 15:52:14.022 2 ACCOUNT(jonathan) [130676] delivered
>>> 15:52:14.022 2 DEQUEUER [130676] LOCAL(jonathan) delivered:
>>>Delivered to the user mailbox
>>>
>>> web:/var/CommuniGate# ./cgpav
>>> 1 FILE eicar.com
>>> 1 ADDHEADER "X-Virus-Flag: Yes"
>>>
>>>
>>> web:/var/CommuniGate# clamscan
>>> /var/CommuniGate/ProcessID: OK
>>> /var/CommuniGate/cgpav: OK
>>> /var/CommuniGate/cgpsa: OK
>>> /var/CommuniGate/@: OK
>>> /var/CommuniGate/eicar.com: Eicar-Test-Signature FOUND
>>> /var/CommuniGate/spam.msg: OK
>>>
>>> ----------- SCAN SUMMARY -----------
>>> Known viruses: 1208850
>>> Engine version: 0.97.3
>>> Scanned directories: 1
>>> Scanned files: 6
>>> Infected files: 1
>>> Data scanned: 0.21 MB
>>> Data read: 0.14 MB (ratio 1.53:1)
>>> Time: 8.048 sec (0 m 8 s)
>>>
>>> web:/var/CommuniGate# ps aux | grep cgp
>>> root      5566  0.0  0.0      0     0 ?        Z    16:50   0:00
>>>[cgpsa] <defunct>
>>> root      5573  0.0  0.1   1812   572 ttyp0    S+   16:51   0:00
>>>grep cgp
>>> root     26549  0.0  0.2   4368  1056 ?        S    Apr14   0:00
>>>/var/CommuniGate/cgpav
>>> root     31784  0.0  6.2  37088 32828 ?        S    15:44   0:02
>>>/usr/bin/perl /var/CommuniGate/cgpsa
>>>
>>> #############################################################
>>>  
>>> This
>>> message is sent to you because you are subscribed to
>>>  
>>>    the mailing list<CGatePro@mail.stalker.com>
>>> <mailto:CGatePro@mail.stalker.com>.
>>>  
>>> To unsubscribe, E-mail to:<CGatePro-off@mail.stalker.com>
>>> <mailto:CGatePro-off@mail.stalker.com>
>>>  
>>> To switch to the
>>> DIGEST mode, E-mail to<CGatePro-digest@mail.stalker.com>
>>> <mailto:CGatePro-digest@mail.stalker.com>
>>>  
>>> To
>>> switch to the INDEX mode, E-mail to<CGatePro-index@mail.stalker.com>
>>> <mailto:CGatePro-index@mail.stalker.com>
>>>  
>>> Send
>>> administrative queries to<CGatePro-request@mail.stalker.com>
>>> <mailto:CGatePro-request@mail.stalker.com>
>>> #############################################################
>>>
>>> This message is sent to you because you are subscribed to
>>>
>>>    the mailing list<CGatePro@mail.stalker.com>.
>>>
>>> To unsubscribe, E-mail to:<CGatePro-off@mail.stalker.com>
>>>
>>> To switch to the DIGEST mode, E-mail
>>>to<CGatePro-digest@mail.stalker.com>
>>>
>>> To switch to the INDEX mode, E-mail
>>>to<CGatePro-index@mail.stalker.com>
>>
>>>
>>> Send administrative queries to<CGatePro-request@mail.stalker.com>
>
> #############################################################  This
>message is sent to you because you are subscribed to    the mailing
>list <CGatePro@mail.stalker.com>.  To unsubscribe, E-mail to:
><CGatePro-off@mail.stalker.com>  To switch to the DIGEST mode, E-mail
>to <CGatePro-digest@mail.stalker.com>  To switch to the INDEX mode,
>E-mail to <CGatePro-index@mail.stalker.com>  Send administrative
>queries to  <CGatePro-request@mail.stalker.com>

Subscribe (FEED) Subscribe (DIGEST) Subscribe (INDEX) Unsubscribe Mail to Listmaster