general spamassassin qmail scanner question

[faris]

There's a whole topic on this subject here somewhere....but I could not find it when I looked. It was basically down to creating a rule that added a big minus score when .. well, I can't remember where we ended up with it.

What's curious is why it only affects some people under certain circumstances. We never get this problem. I guess it is because the IP of the sender is blacklisted (i.e. on one of the dynamic IP lists or something)?

Faris.

[coolemail]

Hi faris,

Certainly both those IP addresses I guess are dynamic ones (one is btinternet and the other is in Kenya and I'm sure that they do not have a static IP).

breun said that it will learn as it goes along, but I'm concerned that it might be "learning" the wrong stuff and deciding that the genuine emails are in fact Spam! It would be embarassing if we were to blacklist our own authenticated outoging emails!

Strange that it is not happening to you. http://wiki.apache.org/spamassassin/ManualWhitelist is what I'm trying to use as the guide in this case.

[biggles]

There's a whole topic on this subject here somewhere....but I could not find it when I looked. It was basically down to creating a rule that added a big minus score when .. well, I can't remember where we ended up with it.

What's curious is why it only affects some people under certain circumstances. We never get this problem. I guess it is because the IP of the sender is blacklisted (i.e. on one of the dynamic IP lists or something)?

Faris.

This one might be the one you are thinking of? It helped my at least...

[faris]

Yes, that's the one.

I guess it might be necessary to remove all PTRs though, because with my local PTR lookups the hostname is always in the header no matter what.

Faris.

[breun]

I'd check which SpamAssassin tests are triggered for these messages and see if there's nothing strange there.

Maybe setting trusted_networks will do what you want? http://wiki.apache.org/spamassassin/TrustPath

[coolemail]

Bi breun,
How can we see which tests are triggered by SpamAssassin, short of using SA_ALT/SA_DEBUG which you have recommended not to do for CPU purposes? Is this is the maillog? All I can see is

spamd: result: Y 3 - AWL,BAYES_00,DYN_RDNS_SHORT_HELO_HTML,HTML_MESSAGE,RDNS_DYNAMIC,SUBJ_ALL_CAPS,TVD_RCVD_IP

but maybe that is what you meant.

I'm a bit confused about the dynamic IP as well. Because all outgoing mail is authenticated through our server which clearly has a static IP. I thought it should not matter that the sender's own PC is on a dynamic IP?

Thanks to all for this continued help.

[breun]

How can we see which tests are triggered by SpamAssassin, short of using SA_ALT/SA_DEBUG which you have recommended not to do for CPU purposes? Is this is the maillog? All I can see is

spamd: result: Y 3 - AWL,BAYES_00,DYN_RDNS_SHORT_HELO_HTML,HTML_MESSAGE,RDNS_DYNAMIC,SUBJ_ALL_CAPS,TVD_RCVD_IP

but maybe that is what you meant.

That's indead what I meant. See the SpamAssassin website for what these tests do: http://spamassassin.apache.org/tests.html

I'm a bit confused about the dynamic IP as well. Because all outgoing mail is authenticated through our server which clearly has a static IP. I thought it should not matter that the sender's own PC is on a dynamic IP?

SpamAssassin looks at the source IP address of the message, so if you use rulesets that score for dynamic IP's then it does matter.

[coolemail]

This morning I put some domains on my local.cf in an attempt not to check any emails in or out for these two domains

[plesk2.expat-email.co.uk ~]# cat /etc/mail/spamassassin/local.cf
report_safe 0
rewrite_header subject *****SPAM*****
required_score 3
# whitelist domain1
whitelist_from *@domain1.com
whitelist_to *@domain1.com
# whitelist domain2
whitelist_from *@domain2.com
whitelist_to *@domain2.com

but I am seeing some emails still being identified as Spam:

Jun 30 15:34:29 plesk2 spamd[4004]: spamd: identified spam (3.1/3.0) for qscand:10113 in 16.4 seconds, 28709 bytes.
Jun 30 15:34:29 plesk2 spamd[4004]: spamd: result: Y 3 - AWL,BAYES_00,DYN_RDNS_SHORT_HELO_HTML,HTML_MESSAGE,RDNS_DYNAMIC,SUBJ_ALL_CAPS,TVD_RCVD_IP scantime=16.4,size=28709,user=qscand,uid=10113,required_score=3.0,rhost=localhost,raddr=127.0.0.1,rport=60805,mid=<001801cb1860$e961d120$bc257360$@com>,bayes=0.000000,autolearn=no
Jun 30 15:34:29 plesk2 qmail-scanner[10549]: Clear:RC:0(41.206.41.100):SA:1(3.1/3.0): 17.39310 28664 cakich@domain1.com staffan.fritzsche@remote-domain.com RE:_SASOL_PP_OFFER_FOR_JUNE_SHIPMENT_-_PPEAK688/2010 <001801cb1860$e961d120$bc257360$@com> 1277908452.10554-0.plesk2.hostname.co.uk:4366 orig-plesk2.hostname.co.uk127790845179010549:28664 1277908452.10554-1.plesk2.hostname.co.uk:21901

And 125 emails have been identified as Spam in total to these 2 domains:

[plesk2.hostname.co.uk ~]# grep :SA:1 /usr/local/psa/var/log/maillog | grep domain1 | wc -l
124
[plesk2.hostname.co.uk ~]# grep :SA:1 /usr/local/psa/var/log/maillog | grep domain2 | wc -l
1
[plesk2.hostname.co.uk ~]#

Can someone tell me what I have done wrong please? I think it worked generally because hundreds of emails to and from these two domains have stopped. Do I need to restart something to make it work? Like:

Code: Select all

qmail-scanner-reconfigure

possibly?

Can someone who has had SpamAssassin on qmail-scanner for a longer time please share with me what their various files look like?

[biggles]

Have you tried the solution posted in the thread I suggested above? You solution isn't very effective. One of the most common ways of spamming is using the same domain as you are spamming as sender. Your rule would make these mails slip through. Using the method I suggested (lowering score for locally sent e-mails) is much more effective. Or am I missing something?

You should also check how the message is composed. The tests the mails are failing indicates they have for example ALL CAPS SUBJECT which spamassassin frowns upon... Also setting a spamlevel to 3 is very low on a newly, untuned (ie one with a bayes with not very much correct information) system...

[coolemail]

Hi Biggles,
Thanks Biggles,
I agree that my temporary fix is not ideal (and is not working anyway!) for all your reasons. I think you meant http://atomicorp.com/forums/viewtopic.php?f=4&t=3811 as the thread. I had a good look at it but have not yet had a chance to go down all my domains and confirm that none of them has a PTR record less the hostname domain.

I was also confused with

# custom rules
header LOCAL_AUTH_RCVD Received =~ /by yourhostname\.yourdomain\.com with \(.* encrypted\) SMTP/
score LOCAL_AUTH_RCVD -10.0

My hostname is plesk2.main-domain.com. So do I have to put in a line for every hosted domain to the effect of:

header LOCAL_AUTH_RCVD Received =~ /by plesk2.main-domain.com\.hosted-domain1\.com with \(.* encrypted\) SMTP/

or a single entry of

header LOCAL_AUTH_RCVD Received =~ /by plesk2\.main-domain\.com with \(.* encrypted\) SMTP/

which I THINK is probably the answer. Now in the maillog itself, emails being sent do not show a LOCAL_AUTH_RCVD entry at all when I grep for it. Is this simply something in the email headers which I cannot grep in maillog?

Thank you for the quick response.

[biggles]

My rows are:

Code: Select all

header LOCAL_AUTH_RCVD Received =~ /by server7\.domain\.se with \(.* encrypted\) SMTP/
score LOCAL_AUTH_RCVD -10.0

It makes all e-mails sent with secure SMTP score -10 (port 465 default). It works for all domains and I only have one entry for my main server domain.

Check your e-mail source to see that the header matches your regex. Do this by sending yourself an e-mail and view the header rows in your e-mail program. Mine is:

Code: Select all

by server7.domain.se with (DHE-RSA-AES256-SHA encrypted) SMTP; 30 Jun 2010 20:40:56 +0200

Good luck!

[coolemail]

Brilliant. I understand that a bit more now, thank you.

I'm using (and all customers as well) port 25 or 587 for outgoing mail, so mine is showing just:
by plesk2.mydomain.co.uk with SMTP; 30 Jun 2010 20:28:44 +0100

so should my entry be just

header LOCAL_AUTH_RCVD Received =~ /by plesk2\.mydomain\.com with SMTP/

[breun]

Do I need to restart something to make it work? Like:

Code: Select all

qmail-scanner-reconfigure

possibly?

Generally you need to reload or restart the service after changing its configuration. So yes, after changing the SpamAssassin configuration you'll need to run 'service spamassassin restart'.

[biggles]

Brilliant. I understand that a bit more now, thank you.

I'm using (and all customers as well) port 25 or 587 for outgoing mail, so mine is showing just:
by plesk2.mydomain.co.uk with SMTP; 30 Jun 2010 20:28:44 +0100

so should my entry be just

header LOCAL_AUTH_RCVD Received =~ /by plesk2\.mydomain\.com with SMTP/

Yes, that should work! Give it a try!

[faris]

Yes, but isn't the original problem cropping up here? i.e. all incoming email will have "plesk2.mydomain.com with SMTP" in the header, whether sent by an authenticated user or not?

(unless you fiddle with the PTRs ?)

Faris.