Atomicorp

Hi,

The asl firewall creates problems on our web sites and every time we disable it, it repops.
How do we disable it completely?

Thanks

ASL uses the Linux standard for managing services:

To stop:

service asl-firewall stop

To remove the service:

chkconfig --del asl-firewall

ASL upgrades will re-enable the asl-firewall. Shunning will also not work correctly without the ASL firewall enabled.

The ASL firewall does not block anything by default, so you may want to check your configuration if its blocking anything. It will only block what you have configured it to block, by default it doesnt block anything. Are you sure its the firewall and not something else, like modsecurity and a modsecurity rule thats being triggered? If its the later, please report the event as a false positive and we'd be happy to resolve the issue for the same day its reported.

Hi,

It's definatly the firewall. When we counter measure it (using minute cron to service asl-firewall stop) everything went well.

We have a site that uses google to load the pages faster. This involves many connection to about a class c of google addresses. The firewall just automaticaly adds rules with a many /32 addresses from this class c and blocks the connection. This causes us to get a connection error to google site instead of the page loading. The firewall just keeps blocking after we disabled it with service asl-firewall stop. We had to disable the asl automatic event handler (not sure if this is how it's called) in order for the firewall to stop creating blocking rules.

So basically without the event handler we're fine. However we want to continue using it.

Thanks.

Perhaps I'm not understand your issue. The firewall doesnt automatically block anything. It doesnt block anything by default. It only blocks what you tell it to block. So I dont think the firewall is your issue. If ASL blocks anything, it will log what it did and why it did it, as explained in this FAQ:

https://www.atomicorp.com/wiki/index.ph ... g_an_IP.3F

Can you tell us exactly what ASL is doing, in terms of the rule IDs that are being triggered for these events? Each rule ID is documented in the wiki, and you can search for the rule ID or click on it from the ASL web console which will explain why the event occurred, and what you can do to configure ASL for your needs.

Also, have you enabled the Crawler protector?

https://www.atomicorp.com/wiki/index.ph ... _Protector

That would prevent a search engine from ever being blocked.

looking at the faq we found out it is blocked by waf 309925
here's some examples:
Tue Nov 26 20:57:43 IST 2013 /var/ossec/active-response/bin/asl-shun.pl add - 74.125.16.21 1385492263.11732 309925
Tue Nov 26 20:57:43 IST 2013 /var/ossec/active-response/bin/host-deny.sh add - 74.125.16.21 1385492263.11732 309925
Tue Nov 26 20:57:43 IST 2013 /var/ossec/active-response/bin/asl-shun.pl add - 74.125.181.3 1385492263.12647 309925
Tue Nov 26 20:57:43 IST 2013 /var/ossec/active-response/bin/host-deny.sh add - 74.125.181.3 1385492263.12647 309925
Tue Nov 26 20:57:43 IST 2013 /var/ossec/active-response/bin/asl-shun.pl add - 74.125.181.13 1385492263.13562 309925
Tue Nov 26 20:57:43 IST 2013 /var/ossec/active-response/bin/host-deny.sh add - 74.125.181.13 1385492263.13562 309925

This are all google addresses and there are many more in the 74.125.181 class.

Those may be part of a range which is registered to google, but those arent used by googlebot, all of googlebots addresses have PTR records and that IP does not:

https://support.google.com/webmasters/a ... 0553?hl=en

You can verify that a bot accessing your server really is Googlebot (or another Google user-agent) by using a reverse DNS lookup, verifying that the name is in the googlebot.com domain, and then doing a forward DNS lookup using that googlebot name. This is useful if you're concerned that spammers or other troublemakers are accessing your site while claiming to be Googlebot.

For example:

> host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer
crawl-66-249-66-1.googlebot.com.

> host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1

[root@asl-modsec-test ~]# host 74.125.16.21
Host 21.16.125.74.in-addr.arpa. not found: 3(NXDOMAIN)

Nothing. So according to google, thats not googlebot. Nevertheless, you can whitelist a range if you want to let google do whatever they want to your system. Just follow this FAQ:

https://www.atomicorp.com/wiki/index.ph ... _in_ASL.3F

The rule 309925 from your logs is documented at the URL below:

https://www.atomicorp.com/wiki/index.php/WAF_309925

This rule detects suspicious user agents. What does the event data show the IP was trying to do?

where do I see the event data to see what the ip was trying to do?

In the apache error log I see this more of google IPs:
[Sat Nov 30 17:20:12 2013] [error] [client 74.125.181.15] ModSecurity: [file "/usr/local/apache/modsecur
ity.d/20_asl_useragents.conf"] [line "320"] [id "309925"] [rev "6"] [msg "Atomicorp.com WAF Rules: Suspic
ious User-Agent, parenthesis closed with a semicolon Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; S
V1;)"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Match of "rx (Qualidator\\\\.com|Exal
eadCloudView|^Mozilla/4\\\\.0 \\\\(compatible;\\\\)$|UTVDriveBot|Add Catalog|^Appcelerator)" against "REQ
UEST_HEADERS:User-Agent" required. [hostname "www.host.com"] [uri "/"] [unique_id "UpoCLD7bCxAAADLyCU0
AAAAS"]

for this we've disabled rule id 309925.

Can you post the actual audit_log entry, the apache error_logs really arent useful.

The useragent however is fake:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)

The real MSIE6 UA looks like this:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ;

So that was not a valid request to your server. We highly recommend you investigate the request, as opposed to disabling this rule which has correctly detected a system that is behaving suspiciously. This useragent is commonly used with attacks and comment spammers, and trends very high in our honeypots.

Can you post the audit_log entry for this event?

Here's a short example from the audit_log of the relevent IPs:

[modsecurity] [client 74.125.18.207] [domain http://www.host.com] [403] [/20131126/20131126-2120/20131126-212
035-UpT0fj7bCxAAAAiKJrMAAAAZ] [file "/usr/local/apache/modsecurity.d/20_asl_useragents.conf"] [line "320
"] [id "309925"] [rev "6"] [msg "Atomicorp.com WAF Rules: Suspicious User-Agent, parenthesis closed with
a semicolon Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)"] [severity "CRITICAL"] Access denie
d with code 403 (phase 2). Match of "rx (Qualidator\\.com|ExaleadCloudView|^Mozilla/4\\.0 \\(compatible;\
\)$|UTVDriveBot|Add Catalog|^Appcelerator)" against "REQUEST_HEADERS:User-Agent" required.
[modsecurity] [client 74.125.181.4] [domain http://www.host.com] [403] [/20131126/20131126-2120/20131126-2120
35-UpT0fz7bCxAAAAi6OVAAAAAw] [file "/usr/local/apache/modsecurity.d/20_asl_useragents.conf"] [line "320"
] [id "309925"] [rev "6"] [msg "Atomicorp.com WAF Rules: Suspicious User-Agent, parenthesis closed with a
semicolon Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)"] [severity "CRITICAL"] Access denied
with code 403 (phase 2). Match of "rx (Qualidator\\.com|ExaleadCloudView|^Mozilla/4\\.0 \\(compatible;\\
)$|UTVDriveBot|Add Catalog|^Appcelerator)" against "REQUEST_HEADERS:User-Agent" required.
[modsecurity] [client 74.125.181.15] [domain http://www.host.com] [403] [/20131126/20131126-2120/20131126-212035-UpT0gD7bCxAAAAjCOqIAAAA0] [file "/usr/local/apache/modsecurity.d/20_asl_useragents.conf"] [line "320"] [id "309925"] [rev "6"] [msg "Atomicorp.com WAF Rules: Suspicious User-Agent, parenthesis closed with a semicolon Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Match of "rx (Qualidator\\.com|ExaleadCloudView|^Mozilla/4\\.0 \\(compatible;\\)$|UTVDriveBot|Add Catalog|^Appcelerator)" against "REQUEST_HEADERS:User-Agent" required.
[modsecurity] [client 74.125.181.25] [domain http://www.host.com] [403] [/20131126/20131126-2120/20131126-212035-UpT0fj7bCxAAAAiFGpMAAAAU] [file "/usr/local/apache/modsecurity.d/20_asl_useragents.conf"] [line "320"] [id "309925"] [rev "6"] [msg "Atomicorp.com WAF Rules: Suspicious User-Agent, parenthesis closed with a semicolon Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Match of "rx (Qualidator\\.com|ExaleadCloudView|^Mozilla/4\\.0 \\(compatible;\\)$|UTVDriveBot|Add Catalog|^Appcelerator)" against "REQUEST_HEADERS:User-Agent" required.
[modsecurity] [client 74.125.181.9] [domain http://www.host.com] [403] [/20131126/20131126-2120/20131126-212035-UpT0gD7bCxAAAAkPgPgAAABU] [file "/usr/local/apache/modsecurity.d/20_asl_useragents.conf"] [line "320"] [id "309925"] [rev "6"] [msg "Atomicorp.com WAF Rules: Suspicious User-Agent, parenthesis closed with a semicolon Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Match of "rx (Qualidator\\.com|ExaleadCloudView|^Mozilla/4\\.0 \\(compatible;\\)$|UTVDriveBot|Add Catalog|^Appcelerator)" against "REQUEST_HEADERS:User-Agent" required.

and here's an entry from modsec_audit.log:
--77001952-A--
[26/Nov/2013:20:01:04 +0200] UpTh4D7bCxAAAApHwCYAAAAg 74.125.181.11 64104 xxx.xxx.xxx.xxx 80
--77001952-B--
GET / HTTP/1.1
Host: www.host.com
Accept-Language: zh-cn
Accept: */*
Connection: Keep-alive
X-Forwarded-For: 117.25.15.138
X-Pss-Loop: pagespeed_proxy
Accept-Encoding: gzip,deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)

--77001952-F--
HTTP/1.1 403 Forbidden
X-Powered-By: PHP/5.3.21
X-Pingback: http://www.host.com/xmlrpc.php
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

--77001952-H--
Message: Access denied with code 403 (phase 2). Match of "rx (Qualidator\\.com|ExaleadCloudView|^Mozilla/4\\.0 \\(compatible;\\)$|UTVDriveBot)" against "REQUEST_HEADERS:User-Agent" required. [file "/usr/local/apache/conf/modsec_rules/20_asl_useragents.conf"] [line "265"] [id "309925"] [rev "4"] [msg "Atomicorp.com UNSUPPORTED DELAYED Rules: Suspicious User-Agent, parenthesis closed with a semicolon Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)"]
Action: Intercepted (phase 2)
Stopwatch: 1385488864218072 422491 (- - -)
Stopwatch2: 1385488864218072 422491; combined=1147, p1=68, p2=1061, p3=0, p4=0, p5=18, sr=0, sw=0, l=0, gc=0
Producer: ModSecurity for Apache/2.7.0 (http://www.modsecurity.org/).
Server: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips mod_bwlimited/1.4
Engine-Mode: "ENABLED"

--77001952-Z--

again, this are all google IPs. I highly doubt someone from google is trying to break into our system from multiple google ips. Further more we see errors in our google webmaster site with lots of server errors.

OK, so yes, that looks google is running a proxy, and some forum spammer was trying to get your server thru googles proxy servers. Lets look at the headers:

Accept-Language: zh-cn

Chinese source.

X-Forwarded-For: 117.25.15.138

Known forum spammer:
http://stopforumspam.com/ipcheck/117.25.15.138

User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)

Fake useragent. That is not MSIE6. MSIE6 never sends a user agent field like that. Thats a common error used by forum spammers pretending to be running MSIE6. We see that exact fake user-agent all the time in our honey pots, and its always a comment spammer, always.

So yes, attacks do come from google sources, especially if its a proxy server which is sending traffic from someone else and not from google. ASL will never block googles actual search bots, but this is not one of googles search bots. Its a proxy server, run by google, that someone else, that is not google is using to get to your server.

But dont take my word for it:

http://blog.sucuri.net/2013/11/google-b ... tacks.html

so basically, we have our hands tied up. We can't allow this attacks, but we can't block the offending IPs as this are google IPs? Unfortunatly I think we will have to allow this attacks.

I would highly recommend you contact google. This is not one of these search bots, their search bots do not use those ranges. This is a proxy server googles running, and they are clearly letting bad people use it.

Now with that said, what legitimate traffic are you getting from these proxy servers? It may be that these blocks have no adverse impact on your server.

But if you dont want to shun this range, but just want to block the attacks, just add the ranges to your whitelist and make you you do not have the whitelist enabled for the WAF. That will prevent the address from being shunned, so any future connections will still come thru, but will block that specific attack and only that specific attack with no impact on anything else from that IP address.

The default in ASL is to not apply the whitelist to the WAF. So if you havent changed that, just add this range to your whitelist and contact google. Why they let people use their proxies for malicious behavior is a mystery.