Search engines crawlers being blocked
Posted: Wed Jun 11, 2014 5:42 am
in the recent events screen the search engine crawlers like google and msn bots are being blocked on all our ASL servers.
some of the IP addresses being blocked:
66.249.65.139
66.249.67.133
66.249.67.95
from the config file /etc/asl/config
MODSEC_00_WHITELIST="no"
MODSEC_00_SEARCHENGINE="yes"
MODSEC_00_AUTOWHITELIST_SEARCHENGINE="yes"
MODSEC_00_ANTIEVASION="yes"
MODSEC_00_STRICT="no"
MODSEC_00_RBL="yes"
from /etc/resolv.conf
# Generated by NetworkManager
search dutch-net.com
nameserver 127.0.0.1
nameserver 80.84.230.153
nameserver 80.84.230.151
options single-request
some of the IP addresses being blocked:
66.249.65.139
66.249.67.133
66.249.67.95
from the config file /etc/asl/config
MODSEC_00_WHITELIST="no"
MODSEC_00_SEARCHENGINE="yes"
MODSEC_00_AUTOWHITELIST_SEARCHENGINE="yes"
MODSEC_00_ANTIEVASION="yes"
MODSEC_00_STRICT="no"
MODSEC_00_RBL="yes"
from /etc/resolv.conf
# Generated by NetworkManager
search dutch-net.com
nameserver 127.0.0.1
nameserver 80.84.230.153
nameserver 80.84.230.151
options single-request