Blocking by rDNS' third level domain
Blocking by rDNS' third level domain
Got a hungry bot on our server this morning. It is spread across a wide range of networks, so blocking it by IP would be at least impractical and at most ineffective.
Assuming the people who run it will keep using the same third level domain for all rDNS addresses where this bot comes from, is there a way to block any request whose rDNS is 007AC9.net?
PS: Although the link above shows this bot using an unique UA, it hit my server using a generic UA ("Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/10.0.649.0 Safari/534.17").
Assuming the people who run it will keep using the same third level domain for all rDNS addresses where this bot comes from, is there a way to block any request whose rDNS is 007AC9.net?
PS: Although the link above shows this bot using an unique UA, it hit my server using a generic UA ("Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/10.0.649.0 Safari/534.17").
CentOS 6.9
ASL 4.0.19-37
ASL 4.0.19-37
Re: Blocking by rDNS' third level domain
Thanks Mike. I placed "007ac9.net" in the file. Will it satisfy the filter to effectively block, for example, crawl07.lp.007ac9.net (91.121.79.180)? AFAIU it should work:mikeshinn wrote:Yes, you can do this with this ruleset:
https://www.atomicorp.com/wiki/index.ph ... AIN_BLOCKS
CentOS 6.9
ASL 4.0.19-37
ASL 4.0.19-37
- mikeshinn
- Atomicorp Staff - Site Admin
- Posts: 4149
- Joined: Thu Feb 07, 2008 7:49 pm
- Location: Chantilly, VA
Re: Blocking by rDNS' third level domain
Yes, that will block everything from that domain.
Michael Shinn
Atomicorp - Security For Everyone
Atomicorp - Security For Everyone
Re: Blocking by rDNS' third level domain
Just got a very draining bot coming from bzq-82-80-249-168.dcenter.bezeqint.net.
I added dcenter.bezeqint.net to the MODSEC_01_DOMAIN_BLOCKS list (I was able to get rid of 007AC9.net this way). But dcenter.bezeqint.net didnt work for bzq-82-80-249-168.dcenter.bezeqint.net. Why?
I can't block the entire bezeqint.net netblock because they are also an ISP for legit customers.
Thanks
I added dcenter.bezeqint.net to the MODSEC_01_DOMAIN_BLOCKS list (I was able to get rid of 007AC9.net this way). But dcenter.bezeqint.net didnt work for bzq-82-80-249-168.dcenter.bezeqint.net. Why?
I can't block the entire bezeqint.net netblock because they are also an ISP for legit customers.
Thanks
CentOS 6.9
ASL 4.0.19-37
ASL 4.0.19-37
Re: Blocking by rDNS' third level domain
Also, I couldnt block the evil (1, 2, 3)
coming from dozens of different IPs to scrape a magento site, which threw it for an endless loop.
The offending netblocks were
172.255.0.0/16 NOBIS-TECHNOLOGY-GROUP-15
23.80.0.0/14 NOBIS-TECHNOLOGY-GROUP-17
23.104.0.0/13 NOBIS-TECHNOLOGY-GROUP-18
and an example IP 23.81.239.84.
Adding "as15003.net" to the custom-domain-blocks file didn't do it.
Am I missing something or is was not supposed to work for these IPs?
coming from dozens of different IPs to scrape a magento site, which threw it for an endless loop.
The offending netblocks were
172.255.0.0/16 NOBIS-TECHNOLOGY-GROUP-15
23.80.0.0/14 NOBIS-TECHNOLOGY-GROUP-17
23.104.0.0/13 NOBIS-TECHNOLOGY-GROUP-18
and an example IP 23.81.239.84.
Adding "as15003.net" to the custom-domain-blocks file didn't do it.
Am I missing something or is was not supposed to work for these IPs?
CentOS 6.9
ASL 4.0.19-37
ASL 4.0.19-37
- mikeshinn
- Atomicorp Staff - Site Admin
- Posts: 4149
- Joined: Thu Feb 07, 2008 7:49 pm
- Location: Chantilly, VA
Re: Blocking by rDNS' third level domain
The forward and reverse records have to match for the domain blocking rules to work, and in this case they dont:
[mshinn@kungfu ~]$ nslookup 23.81.239.84.rdns.as15003.net
;; Got SERVFAIL reply from 8.8.8.8, trying next server
;; Got SERVFAIL reply from 8.8.8.8, trying next server
Server: 8.8.4.4
Address: 8.8.4.4#53
** server can't find 23.81.239.84.rdns.as15003.net: SERVFAIL
[mshinn@kungfu ~]$ nslookup 23.81.239.84
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
84.239.81.23.in-addr.arpa name = 23.81.239.84.rdns.as15003.net.
We could add in a capability to make the lookups non-verified (the PTR doesnt have to match the A) - or both (you decide how verified it needs to be). This would only work on ASL systems so if thats something you'd like we can add it into the FRs and see about rolling it out next week.
If those network ranges are something you really want to block I supposed you could blacklist them, but I gather you may not want to do that. But let me know what approach you prefer. We may be able to create an RBL for this kind of thing as well, which would make it more dynamic. We'd have to brainstorm a little about how to do that intelligently for this use case.
[mshinn@kungfu ~]$ nslookup 23.81.239.84.rdns.as15003.net
;; Got SERVFAIL reply from 8.8.8.8, trying next server
;; Got SERVFAIL reply from 8.8.8.8, trying next server
Server: 8.8.4.4
Address: 8.8.4.4#53
** server can't find 23.81.239.84.rdns.as15003.net: SERVFAIL
[mshinn@kungfu ~]$ nslookup 23.81.239.84
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
84.239.81.23.in-addr.arpa name = 23.81.239.84.rdns.as15003.net.
We could add in a capability to make the lookups non-verified (the PTR doesnt have to match the A) - or both (you decide how verified it needs to be). This would only work on ASL systems so if thats something you'd like we can add it into the FRs and see about rolling it out next week.
If those network ranges are something you really want to block I supposed you could blacklist them, but I gather you may not want to do that. But let me know what approach you prefer. We may be able to create an RBL for this kind of thing as well, which would make it more dynamic. We'd have to brainstorm a little about how to do that intelligently for this use case.
Michael Shinn
Atomicorp - Security For Everyone
Atomicorp - Security For Everyone
Re: Blocking by rDNS' third level domain
That would be great. Please add it to the FR list.mikeshinn wrote:We could add in a capability to make the lookups non-verified (the PTR doesnt have to match the A) - or both (you decide how verified it needs to be). This would only work on ASL systems so if thats something you'd like we can add it into the FRs and see about rolling it out next week.
Correct. Plus I'd have to stay on top of it every time they add a new netblock.mikeshinn wrote: If those network ranges are something you really want to block I supposed you could blacklist them, but I gather you may not want to do that.
I'd be glad to assist with placing these bad agents in an RBL.mikeshinn wrote:We may be able to create an RBL for this kind of thing as well, which would make it more dynamic. We'd have to brainstorm a little about how to do that intelligently for this use case.
Thanks
CentOS 6.9
ASL 4.0.19-37
ASL 4.0.19-37
- mikeshinn
- Atomicorp Staff - Site Admin
- Posts: 4149
- Joined: Thu Feb 07, 2008 7:49 pm
- Location: Chantilly, VA
Re: Blocking by rDNS' third level domain
Can you share your access logs with me, I'll see what we might be able to do on the RBL side as well. Since we can create as many RBLs as you can imagine, I'm thinking we might create some RBLs for things like "impolite bots" similar to the spammer RBLs and others we already have.
Michael Shinn
Atomicorp - Security For Everyone
Atomicorp - Security For Everyone
Re: Blocking by rDNS' third level domain
PM me an email address I'll give it access to papertrail so you can browse/filter/search the logs.mikeshinn wrote:Can you share your access logs with me, I'll see what we might be able to do on the RBL side as well. Since we can create as many RBLs as you can imagine, I'm thinking we might create some RBLs for things like "impolite bots" similar to the spammer RBLs and others we already have.
CentOS 6.9
ASL 4.0.19-37
ASL 4.0.19-37