Spamassassin not working

Forum for getting help with Project Gamera, Spamassassin, Clamav, qmail-scanner and other anti-spam tools.
piotrek
New Forum User
New Forum User
Posts: 4
Joined: Wed Nov 23, 2005 11:15 am

Spamassassin not working

Unread post by piotrek »

Dear Forum members,

I have new box with FC3, PSA 7.5.4 ... all upgrades from ART but i think spamassassin is not working or not working well.

When i do: ps -aux |grep spam
i get:

Code: Select all

root     19052  0.1  3.2 27332 25268 ?       Ss   21:45   0:01 /usr/bin/spamd --username=popuser --daemonize --helper-home-dir=/var/qmail --max-children 5 --create-prefs --nouser-config --virtual-config-dir=/var/qmail/mailnames/%d/%l --pidfile=/var/run/spamd/spamd_full.pid --socketpath=/tmp/spamd_full.sock
root     19053  0.0  2.9 25208 22868 ?       Ss   21:45   0:01 /usr/bin/spamd --username=popuser --daemonize --helper-home-dir=/var/qmail --max-children 5 --create-prefs --nouser-config --virtual-config-dir=/var/qmail/mailnames/%d/%l --pidfile=/var/run/spamd/spamd_light.pid --socketpath=/tmp/spamd_light.sock --siteconfigpath=/dev/null
popuser  19065  0.0  3.3 27332 25340 ?       S    21:45   0:00 spamd child
popuser  19066  0.0  3.2 27332 25268 ?       S    21:45   0:00 spamd child
popuser  19067  0.0  2.9 25208 22872 ?       S    21:45   0:00 spamd child
popuser  19068  0.0  2.9 25208 22872 ?       S    21:45   0:00 spamd child
but i receive tons of spam without *** SPAM *** mark.

Mail headers looks like this:

Code: Select all

Received: (qmail 12636 invoked by uid 2520); 27 Nov 2005 16:27:03 +0100 Received: from IP_ADRESS by HOSTNAME (envelope-from <email_adress>, uid 2020) with qmail-scanner-1.25st 
 (perlscan: 1.25st.   Clear:RC:1(IP_ADRESS):
Headers of emails from my previous (old) server was:

Code: Select all

Received: from IP_ADRESS by HOSTNAME (envelope-from <EMAIL_ADRESS>, uid 2020) with qmail-scanner-1.25st 
 (clamdscan: 0.83/825. spamassassin: 2.63. perlscan: 1.25st.  
 Clear:RC:0(IP_ADRESS):. Processed in 1.067102 secs); 
So, can you tell me what is going on? :)
I did before (from: http://www.grafxsoftware.com/util/PLESK-HOWTO.txt):
shell: yum remove drweb-qmail drweb
shell: yum install qmail-scanner clamav clamav-devel

After install is done, use

shell: /usr/bin/qmail-scanner-reconfigure

Now allshould work fine, you can check to send to you a test virus from http://www.webmail.us/testvirus
In meantime check log files to see if all work fine and no errors are there.
tail -f /var/spool/qmailscan/qmail-queue.log
tail -f /usr/local/psa/var/log/maillog
and earlier i did of course:

Code: Select all

yum update

Even if ClamAV is not listed in mail header it is working, because i see it in action while: tail -f /var/spool/qmailscan/qmail-queue.log

Can you tell me somebody, what is going on?
Where can i find qmail-scanner config?
Can you tell me what should I check or test?

Peter
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Unread post by scott »

that says its working, the next line would tell you where the mail scored, it looks like this:

Clear:RC:1(127.0.0.1):SA:0(-2.3/4.5):. Processed in 7.844983 secs); 29 Nov

SA:0(-2.3/4.5) tells you spamassassin scored that message at -2.3, with a spam threshold of 4.5.
jamesyeeoc
Forum User
Forum User
Posts: 46
Joined: Thu May 12, 2005 3:50 am
Location: Sunny California

Unread post by jamesyeeoc »

I have one RH9 box (Plesk 6.02 last un-upgraded server) also with SA 2.x (clamav, f-prot, qmail-scanner, mod_security) and have seen that many (over 35% of those I've checked) of the emails do not have the "SA:0(-2.3/4.5):. Processed in 7.844983 secs)" portion of the line while all other emails do. The header stops at: " Clear:RC:1(xx.yy.zz.aa)"

On the other servers using Plesk 7.5.x and SA 3.x (clamav, f-prot, qmail-scanner, mod_security) I have not seen this nearly as often, almost never.

It's still catching tons of the spam so I just never worried about it (much).

Here's a snippet of a header posted on this forum by GalacticZero which shows the missing "SA:0(-2.3/4.5)" part of the qmail-scanner header:
Received: from 64.12.138.17 by gz.galacticzero.net (envelope-from <>, uid 2020) with qmail-scanner-1.25st
(clamdscan: 0.87.1/1198. perlscan: 1.25st.
Clear:RC:0(64.12.138.17):.
Processed in 0.147234 secs); 01 Dec 2005 04:41:51 -0000
http://atomicrocketturtle.com/forum/viewtopic.php?t=662

Although his at least shows the "Processed in ..." portion... and has the "X-Spam-Status: No, score=0.7 required=7.0 tests=FUZZY_AMBIEN,
" which allows you to still see what it was scored.
kafka
New Forum User
New Forum User
Posts: 1
Joined: Wed Dec 07, 2005 1:09 am

bump - same here

Unread post by kafka »

I'm running into the same.. any help would be much appreciated!

Here's a header from mine:

(clamdscan: 0.87.1/1204. perlscan: 1.25st.
Clear:RC:0(65.166.51.6):.
Processed in 0.044267 secs); 07 Dec 2005 05:13:29 -0000


And here's one that DID get marked as spam, but that didn't seem to have a spamassassin header..

(clamdscan: 0.87.1/1204. perlscan: 1.25st.
Clear:RC:0(80.250.188.162):.
Processed in 0.870131 secs); 07 Dec 2005 05:08:12 -0000

to be honest I'm not really sure why the second one IS being moved to the spam mailbox.. spamassassi doesn't appear to be running on either.

Any ideas?
jamesyeeoc
Forum User
Forum User
Posts: 46
Joined: Thu May 12, 2005 3:50 am
Location: Sunny California

Unread post by jamesyeeoc »

I'm about ready to scrap the SA on this one server and start over. It just scored this forum's "Topic Reply Notification" email with 51.4, but I don't see anything about the standard topic reply that should cause it. Sheesh.... LOL maybe it's making up for not scoring other obvious Spam mails...

Another point is that the email passes through my secondary MX and was scored on that server with SA:0.2, then was passed to my domain's primary MX (and scanned again by the same AV/SA/RDJ/qmail-scanner versions) and was then scored SA:51.4 , so definitely something gone wrong with SA on that server!
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Unread post by scott »

wow, 51 huh. Id say your bayes and AWL are doing that. You might want to zap those databases and start over from scratch.
jamesyeeoc
Forum User
Forum User
Posts: 46
Joined: Thu May 12, 2005 3:50 am
Location: Sunny California

Unread post by jamesyeeoc »

I would but this server is not using bayes or AWL.... It never used to flag your forum emails, and the only change recently was that I disabled the modsecautoupdate cronjob since the server is still running modsec 1.8, other than that, the server has been 'static' for months.

Just realized, the other thing not working in SA is the tests=_TESTS_ header... strange... time to dig out my double barrel shotgun...
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Unread post by scott »

SA will automatically generate the AWL and Bayes db's over time when mail hits a certain threshold. If you're using qmail-scanner to run SA you can look at the bayes db with this:

sa-learn --dump magic -C /var/qmail/.spamassassin/

0.000 0 56297 0 non-token data: nspam
0.000 0 80945 0 non-token data: nham

Meaning that particular DB had 56K spam, and 80K not spam (ham) messages.

51 though is pretty high, bayes probably couldnt do all that. Maybe AWL could do it, or a specific blacklist.
jamesyeeoc
Forum User
Forum User
Posts: 46
Joined: Thu May 12, 2005 3:50 am
Location: Sunny California

Unread post by jamesyeeoc »

# sa-learn --dump magic -C /var/qmail/.spamassassin/
Use of uninitialized value in numeric lt (<) at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/BayesStore.pm line 1281.
0.000 0 0 0 non-token data: bayes db version
0.000 0 0 0 non-token data: nspam
0.000 0 0 0 non-token data: nham
0.000 0 0 0 non-token data: ntokens
0.000 0 0 0 non-token data: oldest atime
0.000 0 0 0 non-token data: current scan-count
0.000 0 0 0 non-token data: last expiry atime

I'm not worried about it right now, I put your domain on the whitelist and the lastest notification came through untagged. I'll eventually get around to wiping that server's install and re-doing it. Thanks.
Galactic Zero
Forum Regular
Forum Regular
Posts: 471
Joined: Mon Dec 06, 2004 10:43 pm

Unread post by Galactic Zero »

How do I correct this error?

/root/.spamassassin
/root/.spamassassin/user_prefs
/root/.spamassassin/auto-whitelist
/.spamassassin
[root@gz root]# cd /
[root@gz /]# sa-learn --dump magic -C -D /var/qmail/.spamassassin/
ERROR: Bayes dump returned an error, please re-run with -D for more information
[root@gz /]# sa-learn --dump magic -D /var/qmail/.spamassassin/
[7956] info: config: failed to parse line, skipping: user_scores_sql_table userpref
ERROR: Bayes dump returned an error, please re-run with -D for more information
[root@gz /]#


[root@gz /]# sa-learn --dump magic -D
[8190] dbg: logger: adding facilities: all
[8190] dbg: logger: logging level is DBG
[8190] dbg: generic: SpamAssassin version 3.1.0
[8190] dbg: config: score set 0 chosen.
[8190] dbg: util: running in taint mode? yes
[8190] dbg: util: taint mode: deleting unsafe environment variables, resetting PATH
[8190] dbg: util: PATH included '/usr/kerberos/sbin', keeping
[8190] dbg: util: PATH included '/usr/kerberos/bin', keeping
[8190] dbg: util: PATH included '/sbin', keeping
[8190] dbg: util: PATH included '/bin', keeping
[8190] dbg: util: PATH included '/usr/local/sbin', keeping
[8190] dbg: util: PATH included '/usr/local/bin', keeping
[8190] dbg: util: PATH included '/sbin', keeping
[8190] dbg: util: PATH included '/bin', keeping
[8190] dbg: util: PATH included '/usr/sbin', keeping
[8190] dbg: util: PATH included '/usr/bin', keeping
[8190] dbg: util: PATH included '/usr/X11R6/bin', keeping
[8190] dbg: util: PATH included '/root/bin', keeping
[8190] dbg: util: final PATH set to: /usr/kerberos/sbin:/usr/kerberos/bin:/sbin:/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/root/bin
[8190] dbg: dns: is Net::DNS::Resolver available? yes
[8190] dbg: dns: Net::DNS version: 0.48
[8190] dbg: dns: name server: 207.218.192.38, family: 2, ipv6: 0
[8190] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
[8190] dbg: config: read file /etc/mail/spamassassin/init.pre
[8190] dbg: config: read file /etc/mail/spamassassin/v310.pre
[8190] dbg: config: using "/usr/share/spamassassin" for sys rules pre files
[8190] dbg: config: using "/usr/share/spamassassin" for default rules dir
[8190] dbg: config: read file /usr/share/spamassassin/10_misc.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_advance_fee.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_anti_ratware.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_body_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_compensate.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_dnsbl_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_drugs.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_fake_helo_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_head_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_html_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_meta_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_net_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_phrases.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_porn.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_ratware.cf
[8190] dbg: config: read file /usr/share/spamassassin/20_uri_tests.cf
[8190] dbg: config: read file /usr/share/spamassassin/23_bayes.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_accessdb.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_antivirus.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_body_tests_es.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_body_tests_pl.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_dcc.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_domainkeys.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_hashcash.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_pyzor.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_razor2.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_replace.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_spf.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_textcat.cf
[8190] dbg: config: read file /usr/share/spamassassin/25_uribl.cf
[8190] dbg: config: read file /usr/share/spamassassin/30_text_de.cf
[8190] dbg: config: read file /usr/share/spamassassin/30_text_fr.cf
[8190] dbg: config: read file /usr/share/spamassassin/30_text_it.cf
[8190] dbg: config: read file /usr/share/spamassassin/30_text_nl.cf
[8190] dbg: config: read file /usr/share/spamassassin/30_text_pl.cf
[8190] dbg: config: read file /usr/share/spamassassin/30_text_pt_br.cf
[8190] dbg: config: read file /usr/share/spamassassin/50_scores.cf
[8190] dbg: config: read file /usr/share/spamassassin/60_awl.cf
[8190] dbg: config: read file /usr/share/spamassassin/60_whitelist.cf
[8190] dbg: config: read file /usr/share/spamassassin/60_whitelist_spf.cf
[8190] dbg: config: read file /usr/share/spamassassin/60_whitelist_subject.cf
[8190] dbg: config: using "/etc/mail/spamassassin" for site rules dir
[8190] dbg: config: read file /etc/mail/spamassassin/70_sare_adult.cf
[8190] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum0.cf
[8190] dbg: config: read file /etc/mail/spamassassin/70_sare_genlsubj0.cf
[8190] dbg: config: read file /etc/mail/spamassassin/70_sare_header0.cf
[8190] dbg: config: read file /etc/mail/spamassassin/70_sare_html0.cf
[8190] dbg: config: read file /etc/mail/spamassassin/70_sare_obfu0.cf
[8190] dbg: config: read file /etc/mail/spamassassin/70_sare_uri0.cf
[8190] dbg: config: read file /etc/mail/spamassassin/70_sc_top200.cf
[8190] dbg: config: read file /etc/mail/spamassassin/antidrug.cf
[8190] dbg: config: read file /etc/mail/spamassassin/local.cf
[8190] dbg: config: using "/root/.spamassassin/user_prefs" for user prefs file
[8190] dbg: config: read file /root/.spamassassin/user_prefs
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x8bc9ebc)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8b8bc00)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH(0x8c218c0)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
[8190] dbg: pyzor: network tests on, attempting Pyzor
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::Pyzor=HASH(0x8c38280)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::SpamCop from @INC
[8190] dbg: reporter: network tests on, attempting SpamCop
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::SpamCop=HASH(0x8cb9cdc)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::AWL from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::AWL=HASH(0x8cc8c74)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::AutoLearnThreshold from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0x8ce1ed0)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::WhiteListSubject from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::WhiteListSubject=HASH(0x8c4bea0)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::MIMEHeader from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::MIMEHeader=HASH(0x8c4cb0c)
[8190] dbg: plugin: loading Mail::SpamAssassin::Plugin::ReplaceTags from @INC
[8190] dbg: plugin: registered Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0x8c4db5c)
[8190] dbg: config: adding redirector regex: /^http:\/\/chkpt\.zdnet\.com\/chkpt\/\w+\/(.*)$/i
[8190] dbg: config: adding redirector regex: /^http:\/\/www(?:\d+)?\.nate\.com\/r\/\w+\/(.*)$/i
[8190] dbg: config: adding redirector regex: /^http:\/\/.+\.gov\/(?:.*\/)?externalLink\.jhtml\?.*url=(.*?)(?:&.*)?$/i
[8190] dbg: config: adding redirector regex: /^http:\/\/redir\.internet\.com\/.+?\/.+?\/(.*)$/i
[8190] dbg: config: adding redirector regex: /^http:\/\/(?:.*?\.)?adtech\.de\/.*(?:;|\|)link=(.*?)(?:;|$)/i
[8190] dbg: config: adding redirector regex: m'^http.*?/redirect\.php\?.*(?<=[?&])goto=(.*?)(?:$|[&\#])'i
[8190] dbg: config: adding redirector regex: m'^https?:/*(?:[^/]+\.)?emf\d\.com/r\.cfm.*?&r=(.*)'i
[8190] info: config: failed to parse line, skipping: user_scores_sql_table userpref
[8190] dbg: plugin: Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0x8c4db5c) implements 'finish_parsing_end'
[8190] dbg: replacetags: replacing tags
[8190] dbg: replacetags: done replacing tags
[8190] dbg: bayes: no dbs present, cannot tie DB R/O: /root/.spamassassin/bayes_toks
[8190] dbg: config: score set 1 chosen.
[8190] dbg: bayes: no dbs present, cannot tie DB R/O: /root/.spamassassin/bayes_toks
ERROR: Bayes dump returned an error, please re-run with -D for more information
[root@gz /]#
Franklyn Halamka
Still learning my way around Linux Security.
http://www.galacticzero.net
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Unread post by scott »

Get rid of the -D flag, you want to use just -C /var/qmail/.spamassassin/
Galactic Zero
Forum Regular
Forum Regular
Posts: 471
Joined: Mon Dec 06, 2004 10:43 pm

Unread post by Galactic Zero »

Did that first and it said to run it again with the -D flag.

[root@gz /]# sa-learn --dump magic -C /var/qmail/.spamassassin/
ERROR: Bayes dump returned an error, please re-run with -D for more information
[root@gz /]
Franklyn Halamka
Still learning my way around Linux Security.
http://www.galacticzero.net
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Unread post by scott »

Your bayes db is bad, you need to delete it
Galactic Zero
Forum Regular
Forum Regular
Posts: 471
Joined: Mon Dec 06, 2004 10:43 pm

Unread post by Galactic Zero »

Do I need to delete the ones in each users dir?

/var/qmail/mailnames/honeainsurance.com/lee/.spamassassin/bayes_journal
/var/qmail/mailnames/honeainsurance.com/lee/.spamassassin/bayes_toks
/var/qmail/mailnames/honeainsurance.com/lee/.spamassassin/bayes_seen
/var/qmail/mailnames/honeainsurance.com/lee/.spamassassin/bayes_seen-do_not_remove
/var/qmail/mailnames/honeainsurance.com/lee/.spamassassin/bayes_toks-do_not_remove
/var/qmail/mailnames/honeainsurance.com/lee/.spamassassin/.spamassassin/bayes_toks
/var/qmail/mailnames/honeainsurance.com/lee/.spamassassin/.spamassassin/bayes_seen

besides the ones in /var/qmail/.spamassassin
??

Thanks
Franklyn Halamka
Still learning my way around Linux Security.
http://www.galacticzero.net
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Unread post by scott »

No you just need to zap the /var/qmail one
Post Reply