qmail-scanner and Plesk Spamassassin settings

[hotgazpacho]

Well, it looks like using qmail-scanner together with using Plesk to configure Spamassassin does not work. Here are some thing I have noticed that lead me to this conslusion:


- If I set an email account to delete spam above a certain score, such email does not get deleted when qmail-scanner is installed.
- Setting server-wide whitelists in Plesk has no effect when Spamassassin is run from qmail-scanner
- looking at /usr/local/psa/var/log/maillog, for a known user configured in Plesk, I see the following message (assume known_user@example.com is a valid email address):

Code: Select all

Feb 11 09:32:01 u15151464 spamd[32654]: handle_user: unable to find user 'known_user@example.com'!

Is there some configuration option that I am missing in qmail-scanner that will allow Spamassassin to be configured using Plesk?

[hotgazpacho]

OK, looks like I have more information now.

service psa-spamassassin was not running on my server. I started it, and whitelisting works now (according to the spamassassin headers). HOWEVER, I'm watching my maillogs at this pint, and notice that Spamassassin is attempting to scan each message TWICE, once from qmail-scanner, another AFTER qmail-scanner has accepted the message. I get messages like the following:

Code: Select all

Feb 11 10:00:25 u15151464 spamc[1720]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#1 of 3): Connection refused
Feb 11 10:00:26 u15151464 spamc[1720]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#2 of 3): Connection refused
Feb 11 10:00:27 u15151464 spamc[1720]: connect(AF_INET) to spamd at 127.0.0.1 failed, retrying (#3 of 3): Connection refused
Feb 11 10:00:28 u15151464 spamc[1720]: connection attempt to spamd aborted after 3 retries

However, I also get entires subsequent to that, like:

Code: Select all

Feb 11 10:00:28 u15151464 qmail-scanner[1712]: Clear:RC:0(82.236.29.245):SA:0(?/?): 3.523614 2145 vuby34uytrg@got.net known_user@example.com _TUSSI0NEEX,_AMB11EN,_XANAAX,_\/11AGRRA,_C1AL1IS,_\/AL11UM,_AD1PEX_FROM_$65__ <%CUSTOM_A0[20-22]vuby34uytrg@got.net> 1108134025.1714-0.u15151464.onlinehome-server.com:12 1108134025.1714-1.u15151464.onlinehome-server.com:1320 orig-u15151464.onlinehome-server.com11081340246941712:2145 
Feb 11 10:00:28 u15151464 spamd[1426]: got connection over /tmp/spamd_full.sock 
Feb 11 10:00:28 u15151464 spamd[1731]: Using default config for known_user@example.com: /var/qmail/mailnames/example.com/known_user/user_prefs 
Feb 11 10:00:28 u15151464 spamd[1731]: processing message <%CUSTOM_A0[20-22]vuby34uytrg@got.net> for known_user@example.com:110. 
Feb 11 10:00:28 u15151464 spamd[1731]: identified spam (7.1/6.0) for known_user@example.com:110 in 0.4 seconds, 2552 bytes. 

I want to make sure that spamassassin is run only once per message, and I want qmail-scanner to run it (since it can reject email at the SMTP conversatioin phase), and I want users to be able to configure their spam settings and white/blacklists from plesk. I'll continue to investigate. In the mean time, if anyone (scott?) has any ideas, I'd very much appreciate them!

[scott]

THe first bit of logging there says that spamd isnt running. psa-spamassassin is only compatible with older versions of SA. It wont work with 3.0. If you're using q-s and PSA's spam stuff, all your messages will be scanned twice. You'll have disable one or the other to keep it from being scanned twice.

[hotgazpacho]

Scott,

Spamassassin is most defintely running. However, it appears that qmail-scanner was attempting to connect over TCP/IP and not the socket.
I got qmail-scanner to use user config files as configured in plesk by changing:

Code: Select all

my $spamd_socket='';

to

Code: Select all

my $spamd_socket='/tmp/spamd_full.sock';

on about line 341.

HOWEVER, Spamassassin is still being invoked twice per message. Are you sure there is no way to have users configure SA using Plesk, but have the message only scanned once, by qmail-scanner?

[scott]

You're very much using a hybrid environment there. For example, setting the spamd_socket manually like that, PSA has 2 instances of spamd running. One is the "global" (full), and the other is the per-user. With PSA that matters for doing per-user spam settings. With q-s its going to be global either way. My way of doing per-user settings is to store them in mysql, theirs is to store them in files in the users mailbox dir. So psa-spamassassin and the psa code itself wont be able to interface with that (squirrelmail on the other hand can).

That doesn't mean you cant ultimately link all this together, you'll just have to write some perl modules.

[hotgazpacho]

I guess I can live with the double scanning of email. BTW, thanks for the info on psa-spamassassin. I set qmail-scanner to use /tmp/spamd_light.sock so that it would pick up the users SA settings (I did notice a difference in the logs when set to light). This way, I can reject spam during the SMTP conversation while using the user's SA config to do it. Non-spam gets scanned twice, but since I don't have a huge user base, I don't think this will be a problem.

Eventually, I might deploy a Gamera server and offload the spam filtering to that instead.

[scott]

Changing the socket in relation to q-s wont make any difference as to whose settings it will use. As it runs in the mail queue, it has no context on the user its loading settings on (unless you use mysql). This isnt exactly a bad thing, provided spamd is running as qmailq it will be able to create a global AWL and Bayes DB. If it cant, then SA (at least as it relates to q-s) isnt going to be as effective as SA as executed via the users .qmail files (the way PSA does it).

But, since you're running both, I doubt its going to be detrimental to the effectiveness overall.

[hotgazpacho]

Um, according to my log files, it DOES make a difference which socket you connect on. Both times, it uses the user's pref files. Check it out:

Code: Select all

Feb 11 16:14:46 u15151464 spamd[27050]: got connection over /tmp/spamd_light.sock 
Feb 11 16:14:46 u15151464 spamd[7213]: Using default config for known_user@example.com: /var/qmail/mailnames/example.com/known_user/user_prefs 
Feb 11 16:14:46 u15151464 spamd[7213]: checking message <077BD107E21$15C1RPH03$hm305nac9b@acrylatehv54muo9> for known_user@example.com:110. 
Feb 11 16:14:47 u15151464 spamd[7213]: clean message (4.2/5.0) for known_user@example.com:110 in 1.5 seconds, 2105 bytes. 
Feb 11 16:14:47 u15151464 qmail: 1108156487.878895 new msg 4186315
Feb 11 16:14:47 u15151464 qmail: 1108156487.878981 info msg 4186315: bytes 2504 from <hfcrtqtrfn@otakumail.com> qp 7216 uid 2520
Feb 11 16:14:47 u15151464 qmail: 1108156487.884545 starting delivery 165: msg 4186315 to local 10-known_user@example.com
Feb 11 16:14:47 u15151464 qmail: 1108156487.884642 status: local 1/10 remote 0/20
Feb 11 16:14:47 u15151464 qmail-scanner[7205]: Clear:RC:0(217.160.230.41):SA:0(4.2/5.0): 1.546824 2063 hfcrtqtrfn@otakumail.com known_user@example.com Any_pharmaceuticals_at_0.64$_per_portion_dart <077BD107E21$15C1RPH03$hm305nac9b@acrylatehv54muo9> 1108156486.7207-0.u15151464.onlinehome-server.com:841 orig-u15151464.onlinehome-server.com11081564866947205:2063 
Feb 11 16:14:47 u15151464 spamd[27038]: got connection over /tmp/spamd_full.sock 
Feb 11 16:14:47 u15151464 spamd[7223]: Using default config for known_user@example.com: /var/qmail/mailnames/example.com/known_user/user_prefs 
Feb 11 16:14:47 u15151464 spamd[7223]: processing message <077BD107E21$15C1RPH03$hm305nac9b@acrylatehv54muo9> for known_user@example.com:110.
Feb 11 16:14:49 u15151464 spamd[7223]: clean message (4.2/6.0) for known_user@example.com:110 in 1.4 seconds, 2504 bytes. 
Feb 11 16:14:49 u15151464 qmail: 1108156489.370358 delivery 165: success: 1108156497.M980056P7222.u15151464.onlinehome-server.com/did_0+0+2/
Feb 11 16:14:49 u15151464 qmail: 1108156489.370455 status: local 0/10 remote 0/20
Feb 11 16:14:49 u15151464 qmail: 1108156489.370468 end msg 4186315

As you can see, the same message (<077BD107E21$15C1RPH03$hm305nac9b@acrylatehv54muo9>) gets scanned twice, once by qmail-scanner, and a second time by psa-spamassassin. Both times, the user's preferences are loaded from /var/qmail/mailnames/example.com/known_user/user_prefs. So, it would seem that qmail-scanner DOES have a context to load user settings from. However, notice that q-s is connecting on spamd_light, whereas SA is connecting on spamd_full. Also notice that each scan has a different upper threshold for spam. Which is actually kind of odd, because the system default is 6 (not 5), and this particular user's threshold happens to be 7 (not 6).

Interresting...

[scott]

this has to do with the settings used to run spamd. One group of settings says "use the user_prefs for required_hits" the other is saying "use the global local.cf required_hits". Check the man page on spamd for what they do.

[perceptibility]

I was able to get qmail-scanner and psa-spamassassin to work together on PSA 7.5.2 (linux) with the Plesk spamassassin front-end (spamassassin-2.63-0.2).

You can stop the double spamd scanning on a message by editing the qmail-scanner-queue.pl file in /var/qmail/bin/

remove the 'spamassassin' item from the @scanners_installed array on line 255, this seems to stop the spamd process preceeding the qmail-scanner action in the maillog.

don't know if this is exactly what you're looking for, but thought it might help.

incidentally, I ran across a bug with the Plesk 7.5 spamassassin interface when user-level settings are set to delete a message when there are server-wide directives set as well (message is gets a 0.0 score).

anyone have any more info on this one:
SWSoft forum thread

oh, and scott, thanks for the super legit work!

[phatPhrog]

That problem

Code: Select all

@scanners_installed array on line 255

must be a 7.5 version problem, as I am running 7.0.4 and tested it to find spamassassin nor qmail-scanner checked mail anymore.

After reconfiguring qmail-scanner everything returned to normal and mail is being scanned properly.