Page 2 of 3
Re: general spamassassin qmail scanner question
Posted: Tue Jun 29, 2010 12:49 pm
by faris
There's a whole topic on this subject here somewhere....but I could not find it when I looked. It was basically down to creating a rule that added a big minus score when .. well, I can't remember where we ended up with it.
What's curious is why it only affects some people under certain circumstances. We never get this problem. I guess it is because the IP of the sender is blacklisted (i.e. on one of the dynamic IP lists or something)?
Faris.
Re: general spamassassin qmail scanner question
Posted: Tue Jun 29, 2010 12:56 pm
by coolemail
Hi faris,
Certainly both those IP addresses I guess are dynamic ones (one is btinternet and the other is in Kenya and I'm sure that they do not have a static IP).
breun said that it will learn as it goes along, but I'm concerned that it might be "learning" the wrong stuff and deciding that the genuine emails are in fact Spam! It would be embarassing if we were to blacklist our own authenticated outoging emails!
Strange that it is not happening to you.
http://wiki.apache.org/spamassassin/ManualWhitelist is what I'm trying to use as the guide in this case.
Re: general spamassassin qmail scanner question
Posted: Tue Jun 29, 2010 1:28 pm
by biggles
faris wrote:There's a whole topic on this subject here somewhere....but I could not find it when I looked. It was basically down to creating a rule that added a big minus score when .. well, I can't remember where we ended up with it.
What's curious is why it only affects some people under certain circumstances. We never get this problem. I guess it is because the IP of the sender is blacklisted (i.e. on one of the dynamic IP lists or something)?
Faris.
This
one might be the one you are thinking of? It helped my at least...
Re: general spamassassin qmail scanner question
Posted: Tue Jun 29, 2010 4:15 pm
by faris
Yes, that's the one.
I guess it might be necessary to remove all PTRs though, because with my local PTR lookups the hostname is always in the header no matter what.
Faris.
Re: general spamassassin qmail scanner question
Posted: Tue Jun 29, 2010 7:33 pm
by breun
I'd check which SpamAssassin tests are triggered for these messages and see if there's nothing strange there.
Maybe setting trusted_networks will do what you want?
http://wiki.apache.org/spamassassin/TrustPath
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 5:15 am
by coolemail
Bi breun,
How can we see which tests are triggered by SpamAssassin, short of using SA_ALT/SA_DEBUG which you have recommended not to do for CPU purposes? Is this is the maillog? All I can see is
spamd: result: Y 3 - AWL,BAYES_00,DYN_RDNS_SHORT_HELO_HTML,HTML_MESSAGE,RDNS_DYNAMIC,SUBJ_ALL_CAPS,TVD_RCVD_IP
but maybe that is what you meant.
I'm a bit confused about the dynamic IP as well. Because all outgoing mail is authenticated through our server which clearly has a static IP. I thought it should not matter that the sender's own PC is on a dynamic IP?
Thanks to all for this continued help.
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 5:34 am
by breun
coolemail wrote:How can we see which tests are triggered by SpamAssassin, short of using SA_ALT/SA_DEBUG which you have recommended not to do for CPU purposes? Is this is the maillog? All I can see is
spamd: result: Y 3 - AWL,BAYES_00,DYN_RDNS_SHORT_HELO_HTML,HTML_MESSAGE,RDNS_DYNAMIC,SUBJ_ALL_CAPS,TVD_RCVD_IP
but maybe that is what you meant.
That's indead what I meant. See the SpamAssassin website for what these tests do:
http://spamassassin.apache.org/tests.html
I'm a bit confused about the dynamic IP as well. Because all outgoing mail is authenticated through our server which clearly has a static IP. I thought it should not matter that the sender's own PC is on a dynamic IP?
SpamAssassin looks at the source IP address of the message, so if you use rulesets that score for dynamic IP's then it does matter.
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 1:05 pm
by coolemail
This morning I put some domains on my local.cf in an attempt not to check any emails in or out for these two domains
[plesk2.expat-email.co.uk ~]# cat /etc/mail/spamassassin/local.cf
report_safe 0
rewrite_header subject *****SPAM*****
required_score 3
# whitelist domain1
whitelist_from *@domain1.com
whitelist_to *@domain1.com
# whitelist domain2
whitelist_from *@domain2.com
whitelist_to *@domain2.com
but I am seeing some emails still being identified as Spam:
Jun 30 15:34:29 plesk2 spamd[4004]: spamd: identified spam (3.1/3.0) for qscand:10113 in 16.4 seconds, 28709 bytes.
Jun 30 15:34:29 plesk2 spamd[4004]: spamd: result: Y 3 - AWL,BAYES_00,DYN_RDNS_SHORT_HELO_HTML,HTML_MESSAGE,RDNS_DYNAMIC,SUBJ_ALL_CAPS,TVD_RCVD_IP scantime=16.4,size=28709,user=qscand,uid=10113,required_score=3.0,rhost=localhost,raddr=127.0.0.1,rport=60805,mid=<001801cb1860$e961d120$bc257360$@com>,bayes=0.000000,autolearn=no
Jun 30 15:34:29 plesk2 qmail-scanner[10549]: Clear:RC:0(41.206.41.100):SA:1(3.1/3.0): 17.39310 28664
cakich@domain1.com staffan.fritzsche@remote-domain.com RE:_SASOL_PP_OFFER_FOR_JUNE_SHIPMENT_-_PPEAK688/2010 <001801cb1860$e961d120$bc257360$@com> 1277908452.10554-0.plesk2.hostname.co.uk:4366 orig-plesk2.hostname.co.uk127790845179010549:28664 1277908452.10554-1.plesk2.hostname.co.uk:21901
And 125 emails have been identified as Spam in total to these 2 domains:
[plesk2.hostname.co.uk ~]# grep :SA:1 /usr/local/psa/var/log/maillog | grep domain1 | wc -l
124
[plesk2.hostname.co.uk ~]# grep :SA:1 /usr/local/psa/var/log/maillog | grep domain2 | wc -l
1
[plesk2.hostname.co.uk ~]#
Can someone tell me what I have done wrong please? I think it worked generally because hundreds of emails to and from these two domains have stopped. Do I need to restart something to make it work? Like:
possibly?
Can someone who has had SpamAssassin on qmail-scanner for a longer time please share with me what their various files look like?
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 1:13 pm
by biggles
Have you tried the solution posted in the thread I suggested above? You solution isn't very effective. One of the most common ways of spamming is using the same domain as you are spamming as sender. Your rule would make these mails slip through. Using the method I suggested (lowering score for locally sent e-mails) is much more effective. Or am I missing something?
You should also check how the message is composed. The tests the mails are failing indicates they have for example ALL CAPS SUBJECT which spamassassin frowns upon...
Also setting a spamlevel to 3 is very low on a newly, untuned (ie one with a bayes with not very much correct information) system...
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 1:34 pm
by coolemail
Hi Biggles,
Thanks Biggles,
I agree that my temporary fix is not ideal (and is not working anyway!) for all your reasons. I think you meant
http://atomicorp.com/forums/viewtopic.php?f=4&t=3811 as the thread. I had a good look at it but have not yet had a chance to go down all my domains and confirm that none of them has a PTR record less the hostname domain.
I was also confused with
# custom rules
header LOCAL_AUTH_RCVD Received =~ /by yourhostname\.yourdomain\.com with \(.* encrypted\) SMTP/
score LOCAL_AUTH_RCVD -10.0
My hostname is plesk2.main-domain.com. So do I have to put in a line for every hosted domain to the effect of:
header LOCAL_AUTH_RCVD Received =~ /by plesk2.main-domain.com\.hosted-domain1\.com with \(.* encrypted\) SMTP/
or a single entry of
header LOCAL_AUTH_RCVD Received =~ /by plesk2\.main-domain\.com with \(.* encrypted\) SMTP/
which I THINK is probably the answer. Now in the maillog itself, emails being sent do not show a LOCAL_AUTH_RCVD entry at all when I grep for it. Is this simply something in the email headers which I cannot grep in maillog?
Thank you for the quick response.
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 2:43 pm
by biggles
My rows are:
Code: Select all
header LOCAL_AUTH_RCVD Received =~ /by server7\.domain\.se with \(.* encrypted\) SMTP/
score LOCAL_AUTH_RCVD -10.0
It makes all e-mails sent with secure SMTP score -10 (port 465 default). It works for all domains and I only have one entry for my main server domain.
Check your e-mail source to see that the header matches your regex. Do this by sending yourself an e-mail and view the header rows in your e-mail program. Mine is:
Code: Select all
by server7.domain.se with (DHE-RSA-AES256-SHA encrypted) SMTP; 30 Jun 2010 20:40:56 +0200
Good luck!
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 3:55 pm
by coolemail
Brilliant. I understand that a bit more now, thank you.
I'm using (and all customers as well) port 25 or 587 for outgoing mail, so mine is showing just:
by plesk2.mydomain.co.uk with SMTP; 30 Jun 2010 20:28:44 +0100
so should my entry be just
header LOCAL_AUTH_RCVD Received =~ /by plesk2\.mydomain\.com with SMTP/
Re: general spamassassin qmail scanner question
Posted: Wed Jun 30, 2010 6:31 pm
by breun
coolemail wrote:Do I need to restart something to make it work? Like:
possibly?
Generally you need to reload or restart the service after changing its configuration. So yes, after changing the SpamAssassin configuration you'll need to run 'service spamassassin restart'.
Re: general spamassassin qmail scanner question
Posted: Thu Jul 01, 2010 12:59 am
by biggles
coolemail wrote:Brilliant. I understand that a bit more now, thank you.
I'm using (and all customers as well) port 25 or 587 for outgoing mail, so mine is showing just:
by plesk2.mydomain.co.uk with SMTP; 30 Jun 2010 20:28:44 +0100
so should my entry be just
header LOCAL_AUTH_RCVD Received =~ /by plesk2\.mydomain\.com with SMTP/
Yes, that should work! Give it a try!
Re: general spamassassin qmail scanner question
Posted: Fri Jul 02, 2010 8:40 am
by faris
Yes, but isn't the original problem cropping up here? i.e. all incoming email will have "plesk2.mydomain.com with SMTP" in the header, whether sent by an authenticated user or not?
(unless you fiddle with the PTRs ?)
Faris.