Problem using xbl.spamhaus.org

Community support forums for the free/delayed modsecurity rules feed. There is no such thing as a bad question here as long as it pertains to using the delayed modsecurity rules feed. Newbies feel free to get help getting started or asking questions that may be obvious.
londoh
Forum User
Forum User
Posts: 28
Joined: Sat Oct 16, 2010 12:05 pm

Problem using xbl.spamhaus.org

Unread post by londoh »

Hi

I've seen some other posts about issues with spamhaus, but not this aspect.
I've been using the delayed rules for a while and had some concerns about the number of blocks from xbl.spamhaus.org

In fact I dont think its at all correct to use the XBL for a web server blocklist.

I quote from the page at http://www.spamhaus.org/xbl/
XBL is also part of a combined DNSBL comprising SBL, XBL and PBL
so the XBL is made of of various other rbl's, some external but including spamhaus's own PBL.
And here's a quote from the PBL page:
The Spamhaus PBL is a DNSBL database of end-user IP address ranges which should not be delivering unauthenticated SMTP email to any Internet mail server except those provided for specifically by an ISP for that customer's use
So basically the PBL lists some/most dynamic IP's - whether or not they've ever done a bad thing. Simply because they are not supposed to be sending out un-auth'ed email.

Personally I'm gonna turn it off for now while I read a bit more.
Any other opinion?
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Problem using xbl.spamhaus.org

Unread post by faris »

Yes, definitely turn it off. There's a long thread about this elsewhere.

It is an experimental feature and is not on by default. And one reason it is off is because it would cause a lot of false positives just as you have noted.

[Edit: Ah. I see you use the delayed rules. I don't know how these are packaged and configured -- but in the non-delayed rules the rbl rules are commented out and have been since the start ]
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
londoh
Forum User
Forum User
Posts: 28
Joined: Sat Oct 16, 2010 12:05 pm

Re: Problem using xbl.spamhaus.org

Unread post by londoh »

thats from a fresh download of the delayed rules so (if I understand the rule correctly) it looks like its turned on out of the can...

from 00_asl_rbl.conf

Code: Select all

#Global RBL rules
SecRule REMOTE_ADDR "!@pmFromFile /etc/asl/whitelist" \
"chain,deny, log, id:350000,rev:2,msg:'Global RBL Match: IP is on the xbl.spamhaus.org Blacklist',severity:'3'" 
SecRule REMOTE_ADDR "@rbl xbl.spamhaus.org" 
I know there are plenty of disclaimers, beware false positives, your server... your rules, whadaya expect for free! etc etc
but that really ought to be off by default
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Problem using xbl.spamhaus.org

Unread post by scott »

We don't enable this by default in ASL for this reason.
londoh
Forum User
Forum User
Posts: 28
Joined: Sat Oct 16, 2010 12:05 pm

Re: Problem using xbl.spamhaus.org

Unread post by londoh »

scott wrote:We don't enable this by default in ASL for this reason.
maybe not. But you do enable it by default in the delayed rules. Perhaps it simply didnt occur to anyone to turn it off?

Altho in fact the delayed rules are billed as
30 day delay of the ASL modsecurity rules
but clearly by what you say they aren't.
(well also thats apart from the fact that the last release was 04/2010 but anyway...)

and there's a recent thread here https://www.atomicorp.com/forum/viewtop ... =15&t=4402

where confused posters have pasted log clips clearly showing xbl.spamhaus returns throwing 403's
and mike says:
mikeshinn wrote:As an aside, this isn't a rule issue. The RBL engine is very simple: If your DNS setup returns a match, mod_Sec will fire, if not it won't - theres literally no way for the rule to get the answer wrong
well not unless its config'ed with the wrong rbl that is

Obviously the delayed ver. is a free sampler, comes with disclaimers and its up to you guys what you put in it.
I tried it - it gives erroneous results. Thats all.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Problem using xbl.spamhaus.org

Unread post by scott »

ASL manages the rules, so what you're seeing is a ruleset in an unmanaged state.
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Problem using xbl.spamhaus.org

Unread post by faris »

I fear I may have accidentally caused some confusion.

I've just hunted down the *non-delayed* 01_asl_rbl.conf file and found that contrary to what I said, the xbl rule is uncommented (though the rest are commented). Maybe it has been this way from the start and I just mis-remembered things due to the other rules all being commented. The point is that the free 30-day delayed rules really are exactly the same rulset and always has been - just delayed.

However, if you have an ASL subscription and get the non-delayed rules you also get what is effectively a rule manager which, to a certain extent, allows you to disable certain rulesets. By default the rbl set is disabled in this config file. Certain other things are disabled by default too (e.g. the whitelist).

Again sorry for any confusion.

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
londoh
Forum User
Forum User
Posts: 28
Joined: Sat Oct 16, 2010 12:05 pm

Re: Problem using xbl.spamhaus.org

Unread post by londoh »

Faris - I dont think it was you that caused the confusion at all - in fact you said turn xbl off which was helpful.

whats confusing is that the delayed_rules are presented in a state that is arguably highly likely to cause confusion.
they arent 30 day delayed - I spent ages clicking around the sites looking for a download that was approximately 30 days old. gave up and tried what there was
and when you load them up there are loads of false positives
and when you read the forums to figure out why atomic staff havent pointed out (on several occasions) that using the xbl is on by default but its a no-no

thats all caused me loads of confusion

Honestly... I was gonna sign up but now I dunno how much more confusion there is in there.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Problem using xbl.spamhaus.org

Unread post by scott »

Like I said, its because ASL manages the rules based on the environment. Its not just a big stack of stuff like an AV scanner, in ASL rules get organized and configured based on other settings. We tried to make everything available in the delayed feed, so you can get exposed to all the different things you can do with it (for better or worse).
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: Problem using xbl.spamhaus.org

Unread post by mikeshinn »

Thanks for the feedback. This RBL has been changed in the real time rules and was released today. Real time rules are released daily. We'll also make the update in the free rules when the next release is published. Free rule releases are made when our schedule allows. The next free release is scheduled for November.

Also, as Scott mentioned, the RBL have been completed disabled in ASL by default for years - its an experimental feature and you have to turn it on. ASL also manages the rules, so it doesnt matter whats not commented out in the rule files - ASL will enable/disable rules for you. You do not have to comment anything out (so it also doesnt matter if its in a rule file or not).

If you are not using ASL, then yes you need to manually configure the rules to meet your needs. This process is documented here:

https://www.atomicorp.com/wiki/index.php/Mod_security

As to the rules, we publish our free rules as a courtesy and appreciate any feedback. As you may know, we were the first people to publish mod_security rules. No one has been publishing rules longer than we have, and we've always made our feed available for free. Thank you for the feedback, and we hope you are enjoying the use of our rules for free.

Just to clarify, we publish two versions of our rules:

RealTime Rules: The latest and greatest version of the rules, with all the performance enhancements, new security features and bug fixes released by us on a daily basis. These rules are fully supported and are recommended for production use.

If you use Atomic Secured Linux, the rules are managed by the system and you dont have to manually configure the rule files or anything.

Free/Delayed Rules: These are a subset of the realtime rules (because they don't have all the updates of the real time rules, features go into the real time rules first, so they will be missing new features in the real time rules). They are also based on older versions of the rules and are released several times a year. These rules are not supported and are only recommended for those sites with the expertise to manage and tune them for their systems. If you need production quality supported rules, use the Real Time rules. The website should not have said they are delayed 30 days, we've updated that now and thank you for bringing that to our attention. The free rules are released several times a year on a non-standard schedule.
Post Reply