[atomic-testing] clapf 0.4.3, a mail scanner for postfix

[Kalimari]

Installing/tweaking Postfix+Spamasaassin+ClamAV+clampf

FOLLOW AT YOUR OWN RISK (I was forced into this by a badly behaving QMail)

# As the mail queue is reconfigured while switching MTA, all messages that are still in queue are lost.
# Stop SMTP service first and clear queue

Code: Select all

/usr/local/psa/admin/sbin/mailmng --stop-smtpd
ps -ef | grep qmail-send
kill -ALRM `pidof qmail-send`

# install postfix

Code: Select all

/usr/local/psa/admin/sbin/autoinstaller --select-release-current --install-component postfix

# install clapf

Code: Select all

yum --enablerepo=atomic-testing install clapf
/etc/init.d/clapf start

# TRANSFER SPAMASSASSIN BAYES DB FROM QSCAND -> CLAPF (optional, but useful if you have a good bayes)

# su to user qscand with bash feature, check current sa-learn settings, backup and exit

Code: Select all

su -s /bin/bash qscand;
sa-learn --dump magic
sa-learn --backup > ~/.spamassassin/bayes.txt;
exit;

# as root, move bayes.txt and change ownership

Code: Select all

mv /var/spool/qscan/.spamassassin/bayes.txt /var/spool/clapf/.spamassassin/bayes.txt;
chown clapf:clapf /var/spool/clapf/.spamassassin/bayes.txt;

# su to user clapf, check sa-learn, restore bayes.txt, check sa-learn (should match qscand) remove bayes.txt and exit

Code: Select all

su -s /bin/bash clapf;
sa-learn --dump magic;
sa-learn --restore ~/.spamassassin/bayes.txt;
sa-learn --dump magic;
rm -f ~/.spamassassin/bayes.txt;
exit;

# Never managed to get the cron reporting/stats working

Code: Select all

remove clapf entries from /etc/cron.d /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly, /etc/cron.monthly

# CONFIGURE CLAPF

Code: Select all

vi /etc/clapf.conf

#comment out lines containing: avg_addr, avg_port, spaminess_of_too_much_spam_in_top15 and if not using avast!, Kaspersky or Dr.Web, comment out those lines also

# check the following paths:
pidfile=/var/spool/clapf/clapf.pid
workdir=/var/spool/clapf/
quarantine_dir=/var/spool/clapf/quarantine/

# ensure ownership for these files/dirs is clapf.clapf

# SPAMASSASSIN BAYES is already tuned (from clamav) so the results can be trusted
spam_overall_limit=0.9
max_ham_spamicity=0.45
spaminess_oblivion_limit=0.99

# mysql
update_tokens=0
mysqlsocket=/var/lib/mysql/mysql.sock
mysqluser=clapf
mysqlpwd=yourpassword
mysqldb=clapf
# comment out all other logging methods/lines

# USING phpMyadmin or root CREATE Database
create database clapf
grant all privileges on clapf.* to clapf@localhost identified by 'yourpassword' etc
Download matching version http://clapf.acts.hu/download/clapf-0.4.3-rc2.tar.gz and locate db-mysql.sql for db structure and populate db.
# nothing ever gets logged BTW, but seems to be required for full clapf functionality

# CONFIGURE POSTFIX

Code: Select all

# Reduce backscatter & bounce messages in Plesk, activate smtp-auth & uncheck the dnsbl-feature in mailserver-settings, save then edit main.cf
vi /etc/postfix/main.cf

# fairly strict RBL; all accounts use submission port for SMTP, adjust to your taste/requirements
smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_rbl_client b.barracudacentral.org, reject_rbl_client zen.spamhaus.org, reject_rbl_client bogons.cymru.com

smtpd_sender_restrictions = check_sender_access hash:/var/spool/postfix/plesk/blacklists, reject_non_fqdn_sender, reject_unauthenticated_sender_login_mismatch, reject_unknown_sender_domain, permit_sasl_authenticated, check_client_access pcre:/var/spool/postfix/plesk/non_auth.re

smtpd_recipient_restrictions = permit_mynetworks, check_client_access pcre:/var/spool/postfix/plesk/no_relay.re, permit_sasl_authenticated, reject_unauth_destination, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unlisted_recipient, reject_unverified_recipient

# Leave the settings for smtp-auth and dnsbl in Plesk->mailserver-settings alone and these changes will stick. Max. message size, webmail-frontend etc have no effect for smtpd_client_restrictions in main.cf

# Next, Tighten anti-spam measures - add the following:
# stop techniques used to harvest email addresses
disable_vrfy_command = yes

# allow Postfix to log recipient address information when rejecting an address or sender address, to monitor which mail is being rejected
smtpd_delay_reject = yes

# Require that remote SMTP client send EHLO/HELO at the beginning of SMTP session
smtpd_helo_required = yes

# Set helo restrictions, warn_if_reject logs a warning instead of rejecting request (grep "reject_warning" in maillog), non fqdn gave too many errors for our users
smtpd_helo_restrictions = permit_mynetworks, warn_if_reject reject_non_fqdn_hostname, warn_if_reject reject_invalid_helo_hostname

# Block clients that speak too early
smtpd_data_restrictions = reject_unauth_pipelining

#tarpit bots/spammers
smtpd_error_sleep_time = 2s
smtpd_soft_error_limit = 5
smtpd_hard_error_limit = 10

# limit junk commands (NOOP, VRFY, ETRN, RSET) that a remote SMTP client can send before incrementing error counter (default 100)
smtpd_junk_command_limit = 20

# restart clapf/postfix and watch the log

Code: Select all

/etc/init.d/clapf restart
/etc/init.d/postfix restart

tail -f /usr/local/psa/var/log/maillog /var/log/clamav/clamd.log /var/log/clamav/freshclam.log

If all works OK and you feel so inclined yum remove qmail-scanner (spamdyke <- if you were using it).

This is probably not 100% exhaustive and your mileage may vary, but have it working reliably on a busy RHEL5 server. Thought it was about time a working postfix solution was documented. If you spot anything stupid/obvious/otherwise, reply...

[scott]

Great followup, I'll definitely be including a lot of this in the next release. If you've got any more improvements, no matter how small please post them here

[Geoff Riley]

I've made a little more progress... the clapf files in the various cron.* directories are all trying to pick up and place into /var/lib/... changing those to /var/spool/... gets us a little closer.

The biggest improvement so far has been in changing /var/log/maillog to /usr/local/psa/var/log/maillog in /etc/cron.d/clapf.

So with both changes, /etc/cron.d/clapf comes out as:

Code: Select all

59 * * * * root /usr/share/doc/clapf-0.4.3.1/stat/process_syslog.pl `date +\%b\ \%e\ \%H` < /usr/local/psa/var/log/maillog >> /var/spool/clapf/stat/clapf.stat
1 * * * * root /usr/share/doc/clapf-0.4.3.1/stat/clapf-rrd-update.sh /var/spool/clapf/stat/clapf-ham-spam.rrd `tail -1 /var/spool/clapf/stat/clapf.stat | awk '{ print $1 ":" $5 ":" $8 }'`

It's still not right, I'm getting complaints about the .rrd file: if I don't have it present I get a file missing error, if I touch a blank file there I get a "ERROR: reading the cookie off /var/spool/clapf/stat/clapf-ham-spam.rrd faild" message.

...but it's a step in the right direction again. :)

Edit... quote the % in the date command within in the crontab... works wonders. [slaps head]

[Geoff Riley]

The missing link... creating the rrd files....

Code: Select all

/usr/share/doc/clapf-0.4.3.1/stat/clapf-rrd-create.sh /var/spool/clapf/stat/clapf-ham-spam.rrd `date +%s`
/usr/share/doc/clapf-0.4.3.1/stat/clapf-rrd-create.sh /var/spool/clapf/stat/clapf-spam-ratio.rrd `date +%s`
/usr/share/doc/clapf-0.4.3.1/stat/clapf-rrd-create.sh /var/spool/clapf/stat/clapf-spam-size.rrd `date +%s`

...found it on the clapf wiki eventually...

Paths modified appropriately, I've now got three rrd files, just short of 3M each, waiting for data. Let's see what happens over the next few days.

[Waylanderl]

clapf 4.4 is out 6 weeks now. are you think this will aviable in near future?

sorry for my bad english

way

[Geoff Riley]

No idea about 4.4, but I can report that I am now getting graphics being produced after going through the above procedures and creating the rdd files.