Gamera installed but SA doesn't seem to be working at all

[xw00t]

I have followed the Project Gamera installation instructions exactly... including the fine tuning tips and all. I have not had a single error and mail is routing through the box just fine though SpamAssassin is not scanning or rewriting the subjects. The PG install is on a fully patched CentOS 4.3 installation.

In order to test this I am forwarding mail from one of my PSA 8.0.1 boxes running SA as well. The test domain has only one MX record which is the PG system. Both systems are configured to use the Rule Du Jour scripts (which is also seemingly working on the PG system as well) and the logs show SA starting and stopping properly.

Aug 20 01:08:20 nikon spamd[2869]: spamd: server killed by SIGTERM, shutting down
Aug 20 01:08:23 nikon spamd[2914]: logger: removing stderr method
Aug 20 01:08:32 nikon spamd[2916]: spamd: server started on port 783/tcp (running version 3.1.3)
Aug 20 01:08:32 nikon spamd[2916]: spamd: server pid: 2916
Aug 20 01:08:32 nikon spamd[2916]: spamd: server successfully spawned child process, pid 2920
Aug 20 01:08:32 nikon spamd[2916]: spamd: server successfully spawned child process, pid 2921
Aug 20 01:08:32 nikon spamd[2916]: prefork: child states: II

That does seem correct right?

my /etc/tcpcontrol/smtp.rules looks like this:
:allow,RELAYCLIENT="",RBLSMTPD="sbl-xbl.spamhaus.org",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"

though i have tried this as well and I get the same results:
:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"

After getting this to work properly I would like to try and implement some for of greylisting as well... any thoughts on this or has it been done with PG already?

Thanks in advance

[xw00t]

After doing some more research I found that I am not the only one who is/was having this problem with SpamAssassin on Project Gamera. I found this article posted by Travisbell: http://atomicrocketturtle.com/forum/viewtopic.php?t=944 ... but he went unanswered. I have PM'd him just incase he may had found an answer but I have not heard back yet.

NightStorm has found out how to put a stop to the "prefork: child states: II" message in the /var/log/maillog.

Once I applied:
SPAMDOPTIONS="-d -u qmailq -c --round-robin -H /var/qmail"
to my /etc/sysconfig/spamassassin file I began to get the following:

Aug 20 02:38:00 nikon spamd[3749]: logger: removing stderr method
Aug 20 02:38:05 nikon spamd[3751]: spamd: server started on port 783/tcp (running version 3.1.3)
Aug 20 02:38:05 nikon spamd[3751]: spamd: server pid: 3751
Aug 20 02:38:05 nikon spamd[3751]: spamd: server successfully spawned child process, pid 3755
Aug 20 02:38:05 nikon spamd[3751]: spamd: server successfully spawned child process, pid 3756
Aug 20 02:38:05 nikon spamd[3751]: spamd: server successfully spawned child process, pid 3757
Aug 20 02:38:05 nikon spamd[3751]: spamd: server successfully spawned child process, pid 3758
Aug 20 02:38:05 nikon spamd[3751]: spamd: server successfully spawned child process, pid 3759

Which looks a lot better...

... but SA still refuses to actually scan anything and so I am still looking for some sort of assistance with this issue.

Also it seems that the RBLSMTPD="sbl-xbl.spamhaus.org" statement in my /etc/tcpcontrol/smtp.rules isn't working as the SBL test fails each time.

Any thoughts on this could help.

Thanks again.

[scott]

There are a bunch of questions in here, so I'll try to hit on the solutions quick:
1) greylisting: yum install qgreylist (works on PSA too)
2) make sure spamassassin and clamav are running, then run qmail-scanner-reconfigure. This will detect and add SA into the the q-s config. You can verify this by looking at the scanners array in /var/qmail/bin/qmail-scanner-queue.pl. If thats still not being detected, remove the >/dev/null line from qmail-scanner-reconfigure, and run it again. This time you'll see the output of whats failing.
3) rbls can be added in /var/qmail/control/rbldomains, adding them to the tcpserver configs works too, just keep in mind youve got to rebuild the cdb file each time you do it. Regardless of weither you add those in to the smtp layer, spamassassin will run RBL tests on its own. So either way, you're still getting the benefit.
4) Logging, I dont turn those off personally, since they help me on tuning rules for performance. Might want to ask about that one on the spamassassin list

[travisbell]

2) make sure spamassassin and clamav are running, then run qmail-scanner-reconfigure. This will detect and add SA into the the q-s config. You can verify this by looking at the scanners array in /var/qmail/bin/qmail-scanner-queue.pl. If thats still not being detected, remove the >/dev/null line from qmail-scanner-reconfigure, and run it again. This time you'll see the output of whats failing.

Took two months but I can finally resume my attempts to get SA working. Sweet. I'll let you guys know how my process goes...

[xw00t]

After completing this step:

2) make sure spamassassin and clamav are running, then run qmail-scanner-reconfigure. This will detect and add SA into the the q-s config. You can verify this by looking at the scanners array in /var/qmail/bin/qmail-scanner-queue.pl. If thats still not being detected, remove the >/dev/null line from qmail-scanner-reconfigure, and run it again. This time you'll see the output of whats failing.

SpamAssassin began to actually run and filter mail correctly. The "reconfigure" though replaced my /etc/sysconfig/spamassassin from

SPAMDOPTIONS="-d -u qmailq -c --round-robin -H /var/qmail"

to

SPAMDOPTIONS="-d -c -m5 -H"

The reason I had made that change is becasue I was getting that Prefork error as I stated before. After the reconfigure I no longer get those errors.

NightStorm: You should give this a shot as you may be missing something.

I went a head and removed the RBLSMTPD="sbl-xbl.spamhaus.org" statement from my /etc/tcpcontrol/smtp.rules and created the rblhosts file under /var/qmail/control as well and put sbl-xbl.spamhaus.org in there. (SPAMHAUS Rocks BTW)

The only issue I seem to be getting now is the fact that AWL and Bayes are not working proplerly. I have done some more research along those lines and have found that I am not the only one with these errors even though they are not considered to be "a critical error"... here is the ART forum post on the issue:
http://atomicrocketturtle.com/forum/viewtopic.php?t=496

I ended up creating a .spamassassin directory and chown it to qscand and created files: bayes, auto-whitelist & user_prefs. These files are seeminly not an issue until spamd tries to lock them. Here is the error:

Aug 20 16:53:02 nikon spamd[3759]: locker: safe_lock: cannot create tmp lockfile /var/spool/qscan/.spamassassin/auto-whitelist.lock.nikon.[domain removed].3759 for /var/spool/qscan/.spamassassin/auto-whitelist.lock: Permission denied

this leads me to believe that this directory and associated files are unaccessible to which ever user is being used to scan and lock these files. They are unable to write to the /var/spool/qscan/ dir and below.

Any thoughts?

[xw00t]

So it seems that after I looked into this SA issue a bit further the problem was indeed with user permissions. Here is how to fix the:

spamd[3759]: auto-whitelist: open of auto-whitelist file failed: locker: safe_lock: cannot create tmp lockfile /var/spool/qscan/.spamassassin/auto-whitelist.lock.[domain.tld].3759 for /var/spool/qscan/.spamassassin/auto-whitelist.lock: Permission denied

&

spamd[3759]: bayes: locker: safe_lock: cannot create tmp lockfile /var/spool/qscan/.spamassassin/bayes.lock.[domain.tld].3759 for /var/spool/qscan/.spamassassin/bayes.lock: Permission denied

issues.

ps aux | grep spam

locate the user after the -u option (with Gamera its qmailq)
this user doesn't have it's own group but that doesn't matter.

go ahead and create the .spamassassin dir under /var/spool/qscan and chown -R qmailq:qscand .spamassassin/

this will take care of the perl modules permission issues. You could go ahead and create the bayes, auto-whitelist & user_prefs files and chown them the same way or let spamd do it for you once it trys to write to them the first time.

I think that about covers all of those issues ... I have yet to install the greylisting on the PG server but I'm not sure how much help it is going to be for my usage.

Again... any thoughts on any of this would be much appreciated

[scott]

Yep, I know exactly what did that. Your old spamassassin config was running the spamd daemon as qmailq, which couldnt write to the qscand dir.

[xw00t]

Just for curiosity sake ... at what point did i change the default spamassassin config to make this happen? I know there are a few others on this forum that are currently having the exact same issue and it make me wonder what they did as well.

This system was a brand new install of CentOS 4.3 and I thought I followed the install process exactly.

Thanks again Scott

[scott]

In 2.0 I finished the qmail-scanner privilege separation. To correct it, I had set up qmail-scanner-reconfigure to check the spamassassin settings, which should have fixed this for you automatically, but didn't for some reason.

[xw00t]

Scott -

It turns out that the reason I was having issues in the first place with the user permissions has something to do with how Rules Du Jour is working with SpamAssassin.

Today I have received the usual email updates with regards to the RDJ updated successfully and then noticed that the logs had gone all crazy with

Aug 28 11:20:27 nikon spamd[1977]: locker: safe_lock: cannot create tmp lockfile /var/spool/qscan/.spamassassin/auto-whitelist.lock.[domain_removed].1977 for /var/spool/qscan/.spamassassin/auto-whitelist.lock: Permission denied
Aug 28 11:20:27 nikon spamd[1977]: auto-whitelist: open of auto-whitelist file failed: locker: safe_lock: cannot create tmp lockfile /var/spool/qscan/.spamassassin/auto-whitelist.lock.[domain_removed].1977 for /var/spool/qscan/.spamassassin/auto-whitelist.lock: Permission denied
Aug 28 11:20:27 nikon spamd[1977]: bayes: locker: safe_lock: cannot create tmp lockfile /var/spool/qscan/.spamassassin/bayes.lock.[domain_removed].1977 for /var/spool/qscan/.spamassassin/bayes.lock: Permission denied

I edited the /etc/sysconfig/spamassassin file back to its original state... chown -R qmailq:qscand /var/spool/.spamassassin dir and restarted xinetd and SA to be sure.

now it works again ... i have disabled the cron event for RDJ temporarily until I can find the issue.

Have you seen anything like this before?

[scott]

no I havent, but definitely do not screw with the user SA runs as. That causes all kinds of headaches.

[humpy]

ps aux | grep spam

locate the user after the -u option (with Gamera its qmailq)
.

go ahead and create the .spamassassin dir under /var/spool/qscan and chown -R qmailq:qscand .spamassassin/

-------------------------
i did as described above, but still have the same error occuring. any tips for trying to further troubleshoot? im stumped..
Cheers W.

[ryanz]

So what should /var/qmail/control/rbldomains and /etc/tcpcontrol/smtp.rules look like when moving th erbl's to rbldomains?

[scott]

you just add each rbl one per line to /var/qmail/control/rbldomains. I highly recommend zen.spamhaus.org

[ryanz]

Hi Scott,

What should we do with /etc/tcpcontrol/smtp.rules to read the rbldomains

Ryan