Rate Limiting to Escape Attacks

Community support forums for Virtualization products, such as Xen(TM), VMWare(TM), Virtuozzo(TM), KVM, Qemu, lguest, openvz and others! There is no such thing as a bad question here as long as it pertains to a virtualization technology or product. Newbies feel free to get help getting started or asking questions that may be obvious.
KrazyBob
Forum Regular
Forum Regular
Posts: 310
Joined: Mon Mar 19, 2007 3:47 pm

Rate Limiting to Escape Attacks

Unread post by KrazyBob »

I MUST rate limit my Virtuozzo containers. Too many exploits at the moment running up my bill. I came across this but I am 1. afraid to try it and 2. Is it put into a script for rc.local perhaps? PLEASE help me understand how to rate limit to 5mbit. Please?

Code: Select all

    Limiting outgoing bandwidth

    We can limit container outgoing bandwidth by setting the tc filter on eth0.

    DEV=eth0
    tc qdisc del dev $DEV root
    tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
    tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
    tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip src X.X.X.X flowid 1:1
    tc qdisc add dev $DEV parent 1:1 sfq perturb 10

    X.X.X.X is an IP address of container.
    Limiting incoming bandwidth

    This can be done by setting the tc filter on venet0:

    DEV=venet0
    tc qdisc del dev $DEV root
    tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
    tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
    tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip dst X.X.X.X flowid 1:1
    tc qdisc add dev $DEV parent 1:1 sfq perturb 10

    Note that X.X.X.X is an IP address of container.

I really like to learn this. I want to rate limit at 5Mb/s. Would this be a start-up script in rc.local?
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Rate Limiting to Escape Attacks

Unread post by faris »

This is outside my area of knowledge.

How about turning the problem on its head? My Co-lo company offer two different types of bandwidth. Metered on a 100Mbit/s port, and unmetered on a 10Mbit/sec port. Might your co-lo company/connectivity supplier do the same thing?

Virtuozzo 4.7 has some sort of built-in bandwidth limiting. Unfortunately the documentation seems to contradict itself. In one place it seems to say that you cannot limit the rate to be under a certain level. But in another, it says 'This bandwidth is not the limit for a Container (unless the RATEBOUND parameter is set to "yes" in the Container configuration file) – the Container is able to take the needed bandwidth from the TOTALRATE bandwidth pool if it is not used by other Containers.'

The above is from http://download.swsoft.com/virtuozzo/vi ... UG/300.htm

Going back to the script you posted, this is from here https://openvz.org/Traffic_shaping_with_tc

This appears to have to go on the hardware node, not in the container itself. So why not type it in, line by line? That way, if it screws things up, a reboot will resolve it.

It is outbound that's worrying you, so just the first half would need doing.

Note that further down the page, there's a simple "two liner" that may be all you need, which rate limits the traffic FROM the container.

Code: Select all

Limiting packets per second rate from container[edit]

To prevent dos atacks from the container you can limit packets per second rate using iptables. 


DEV=eth0
iptables -I FORWARD 1 -o $DEV -s X.X.X.X -m limit --limit 200/sec -j ACCEPT
iptables -I FORWARD 2 -o $DEV -s X.X.X.X -j DROP

Here X.X.X.X is an IP address of container. 
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
KrazyBob
Forum Regular
Forum Regular
Posts: 310
Joined: Mon Mar 19, 2007 3:47 pm

Re: Rate Limiting to Escape Attacks

Unread post by KrazyBob »

Hi Faris,

Thank you for weighing in. I was hopeful. But you are correct that the 4.7 docs contradict themselves. I've got to rate limit. Changing the NIC speed didn't work. I copied from the page you referenced on Traffic Shaping. But I don't know how the syntax needs to be to rate limit 2 containers.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Rate Limiting to Escape Attacks

Unread post by scott »

This is something we'll be adding into Rev Limiter pretty soon. I'm not sure how virtuozzo has evolved in light of cgroups, so this may not be applicable in virtozzo, however to effectively do this on a container (in modern terms, these are now called "Namespaces") requires a kernel that supports control handles, and/or a firewall that can handle mark support. Using control handles you can associate these tc type rules to a user, group, process, or namespace.

https://access.redhat.com/documentation ... t_cls.html

something to consider with tc and control handles, the handles are assigned in hex (ie: 0x10012 = 1:12) but tc assumes that handle is in ascii, ie:

tc class add dev eth0 parent 1:1 classid 1:12 hfsc sc rate 1.0mbps ul rate 256kbs
KrazyBob
Forum Regular
Forum Regular
Posts: 310
Joined: Mon Mar 19, 2007 3:47 pm

Re: Rate Limiting to Escape Attacks

Unread post by KrazyBob »

Umm... what did --==>> he say? LOL!
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Rate Limiting to Escape Attacks

Unread post by faris »

I'm totally confused about the status of cgroups in VZ. People seem to have been talking about them in general ways back as far as 2008 but then it all goes quiet after 2012. But I'm sure I read something about them again in relation to the upcoming 3.x-based VZ kernels though.

Anyway, I would count them out within the container.


Going back to VZ's built-in network limiting support, I have re-read the docs and rate limiting is supported in VZ4.7. With the GUI this is easy to set up - you just tick a box. It is ever so slightly more complex at the command line, but I think I've worked it out.

If you refer back to http://download.swsoft.com/virtuozzo/vi ... UG/300.htm you'll see that it actually uses "tc" to do this under the hood :-)

And if I've read this correctly then:
1) /etc/vz/vz.conf should have

Code: Select all

## Network traffic parameters
TRAFFIC_SHAPING=yes
BANDWIDTH="eth0:102400"
TOTALRATE="eth0:1:4096"
RATE="eth0:1:8"
The first line enables support for it.

The second line defines the speed of the ethernet adapter

The third line confuses me but won't matter as it will shortly be ignored. I *think* this is the upper limit speed for traffic going to all IPs in the default network class (1) which is everywhere. But forget this for now. It isn't what we are interested in.

The fourth line is the default minimum guaranteed rate, 8Kbit/sec.

Then you activate it:

Code: Select all

# /etc/init.d/vz shaperon
Starting Virtuozzo shaping: Ok
Set shaping on running Container :
vz WARNING: Can't get tc class for Container(101).
vz WARNING: Can't access file /var/run/vz_tc_classes. \
Creating new one.
vz WARNING: Can't get tc class for Container(1).
(your error messages may differ)

Use /etc/init.d/vz shaperoff to disable it if things go wrong

Next we need to add to or edit /etc/vz/conf/[CTID].conf

Code: Select all

RATE="eth0:1:512"
RATEBOUND="yes"
And here is where it gets interesting.
In line 1, we override the default 8Kbit/sec default minimum and make it 512Kbit/sec.
Then in line 2, we turn this into an upper limit, not a minimum.

By doing this, we limit the container to 512kbit/sec and hopefully your problem is solved.

You don't need to read any further. But if you are bored, lets go back to TOTALRATE="eth0:1:4096" in vz.conf
If we had RATEBOUND="no" in [ctid].conf, then all containers that have traffic shaping enabled would share a virtual network pipe limited to 4096kbit/sec when communicating to IPs in network class 1 (defined as everywhere).

Why the default is 4096 is completely beyond me and led to a great deal of confusion on my part. Surely a more logical default would be 102400? Anyway, as an alternative way to get your containers to behave, you might consider setting it to 512 or 1024 in vz.conf and setting RATEBOUND="no" in [ctid].conf

And finally....what's this network class 1 (everywhere) I keep going on about?
VZ can have up to 15 network address classes, allowing you to set different rates/limits on different IP address ranges. By default, class 1 is 0.0.0.0 = all IP addresses. I have not figured out which configuration file this is defined in, I'm afraid, but in this situation it doesn't matter as we want the shaping to apply to all external IP addresses. EDIT: found it: /etc/vz/conf/networks_classes

As usual, I may have got all of this wrong or some small but crucial thing wrong, but I'm pretty certain I'm right. Nevertheless, keep in mind that you might encounter unexpected problems if you follow my advice - every change can be dangerous.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
Post Reply