Server Backups rsync vs plesk

[douglaseggleton]

We have two servers both running plesk, centos 5 and ASL. Server A is our live server, however we would like to use server B as our backup server, so that we can get straight off the ground when server A gives in.

We were previously warned for high usage due to our backups. (Done daily via Plesk Backup Manager to remote FTP server, so had to limit the frequency of backups) - I'm quite new to this server, so all help would be much appreciated. I've done some searching - Would rsync from the backup server be the most effective way of doing it? I'm not worried about it automatically switching to the backup server at the moment, we just want to get a reliable backup system working.

[douglaseggleton]

Just a little more information I have found out...

Our old server is running an older version of plesk - Which files/directory will I need to backup using rsync and can I just copy them to the same directory on the old server?

[faris]

I don't think this will work for different versions of plesk.

There's all sorts of stuff that will be different in the Plesk configuration databases, plus locations of files and so on and so forth.

In addition, rsync doesn't/can't handle backing up your MySQL databases fully even if there wasn't an issue with the different versions of plesk. You'd need to do a mysql dump (with locked tables) of all your databases and rsync those rather than the database files themselves. And of course that makes restoration that little bit more difficult (alternatively I suppose you could stop mysql before rsynch, then restart, but that might cause problems in itself as it won't be instant).

[douglaseggleton]

Hmmm ok. So a failover option is probably not going to be an option without a second plesk license. So if I wanted to backup say just the www directory and mysql (as you described)... I could rsync these to a directory on the old server, this would sync files of the new server and old (therefore keeping bandwidth down). Then I could perform a backup on the old plesk server? from the rsync directory. This should make it a lot quicker and keep the bandwidth lower - whilst keeping the load of the main server?

Another question, on the old server, would it be possible to put the files into the vhosts and potentially run them without plesk? Would this be in anyway viable?

Thanks for all your help, much appreciated! This is just something I need to get sorted.

[faris]

You could, of course, use the same version of Plesk *and the same key* on the backup server. As long as Plesk wasn't actually running in two places at once, it should be OK.

Basically you'd make an exact mirror on server B, but with services not running. If A goes down, you fire up B.

I can think of some complications regarding IP addresses, but they should be easy enough to get around with a bit of thought.

I don't think this is the best way to go, however.

I realise you have a limited budget (don't we all?) and you want to acheive the best you can within this budget, and rsync between two servers seems to be the cheapest option. But is it?

How about getting a better product for server A, where it doesn't matter how much load you put on it? Thinking about your original post, is this a VPS? I can't see why load would be an issue if it was not?

So how about this: Find a VPS provider that allows you to back up the VPS as a whole, and who has an infrastructure that is not likely to fail.

For example, all our host nodes for our VPSes have mirrored drives, thus protecting against the most likely point of failure (IMHO). Some form of RAID array is quite common, and for all I know universal, for VPS providers.

On top of that, our VPS customers can backup their entire VPS at will. This is a standard feature of Virtuozzo and is therefore a common feature to find for a VPS. There would be no question of load problems with this sort of setup. It doesn't come into the equation. Yes, we'd get annoyed if you decided to backup every 10 minutes, but you only really need to backup daily at the very most, and weekely would be enough to protect against 99% of likely disasters, with a maximum of 7 days data loss and most likely less.

If your customers expect more reliability than this then they need to pay a LOT more and you can then afford to invest in even more protection.

Alternatively, how about some sort of continuous data protection product if you want this level of reliability (see R1Soft's CDP with Mysql add-on), maybe backing up to server B? Then if A dies, you just get it replaced quickly and restore. Downtime would probably be in the region of 4 hours depending on how long it took to get a replacement for A plugged in and the CDP agent installed in order to do the restore, and the speed of the connection between the two servers, and how much data was involved. CDP is load-load (once the first backup is made). And the R1Soft product is quite cost effective. I've not used it personally, but it seems to be used by a lot of companies.

[douglaseggleton]

Thanks for the advice! Sorry... my wording wasn't correct. We had high bandwidth usage when doing backups. They are both two dedicated servers. Is there a way of maybe using virtualmin (or free panel) just to get the sites up and running on a backup or is this completely mad. Thanks in advance.

[faris]

In theory you wouldn't need virtualmin or anything else. just apache and qmail and spamassassin and..... but there are too manu other "ands" that I don't think it would work. Does anybody else have an opinion on this?

Here are two other solutions:

Have server a and b in the same data centre, with a dedicated network cable between them. No more bandwidth problems.

OR, limit the backup transfer speed*. This should solve the problem with the provider? No huge burst of traffic? Or is it the sheer amount of data that it taking you over a set monthly data transfer limit?

*I have no idea how you might do this, but it should be possible via some kind of netfilters thingie.

[douglaseggleton]

Thanks once agian! The servers I believe are networked in the datacentre, but previously backups were being done elsewhere. Ideally what we wanted was to rsync everything to the old server (without plesk) and run that if the main fails. I can get the backup working - but was just pondering the thought of a failover without plesk (if it wasn't too much configuration issues).

[faris]

Yeah....I don't know...I'm not feeling love for this idea basically. I'm not saying it definitely wouldn't work, but I can imagine all sorts of problems, that's all.

[douglaseggleton]

Ok not a problem Thanks for your help!

We've just acquired another license to give a backup a go... basically we want to mirror server a to server b. We currently have r sync dumping everything to server b - which is breaking that plesk panel and showing up with the wrong server names... we realise dumping the whole database is not best practice - so was after any advice on getting this set up? Thanks

[douglaseggleton]

Ok new idea...

We have a mucked up backup plesk, however we are still determined to do something useful with it. We have been backing up with rsync - however, we would like to use the plesk backup system.

So here's the idea - we make the backup's on plesk A using the control panel and FTP these to our plesk B server (into /backups for example, as a compressed file). That way, if our plesk A goes down, we have backups to restore as soon as plesk A goes up again.

If the scenario was that plesk A went down, and couldn't come back up, could we then use plesk B with the backups?

If anyone has any suggestions regarding a clever solution here. Thanks

*The ftp should't increase outgoing bandwidth from if the two servers are networked together, if anyone can confirm this, would be much appreciated.

[faris]

Yes, this should work.

There are two or three issues to consider, however.

1) Do a test restore to see how things go. Then do it again to see what happens when you restore over data that's already there.

2) It can take a very long time indeed to restore. Hours and hours and hours.

3) rsync would be better to transfer the files from one machine to another, especially if you are using incremental backups, but only if the backup solution you are using doesn't have built-in incremental backups with FTP integration.

As to whether it will count towards your data usage, it depends if the data actually passes over the direct link between your servers or not. Are we talking about a dedicated cable connection between the two? If so then you'll need to get the networking right, and set up routing to match.

If there's no direct cable connection then it really depends on your co-lo company, and how they charge for bandwidth.

For example we have both a direct cable connection and the ability to transfer as much data as we want between any of our machines as long as all our machines are on the same subnet. If we outgrow our subnet and decide not to request a bigger one (which would mean having to re-address all the machines - no thanks), we'd have to request a second subnet and would be charged for data transfer between the two subnets if it doesn't go via our own switch for whatever reason. That's just the way things are with our co-lo company. Each one has its own policies.

[douglaseggleton]

Thanks Faris, a great help!

So I would take it that both of the plesk panels need to be the same version for this to happen?
I think I might stick with the rsync option if it's quicker and uses less bandwidth. At the moment, we are rsyncing the following:

Vhosts – everything in /var/www/vhosts/ vhosts.exp
Mysql – everything in /var/lib/mysql/ EXCEPT the mysql and tortix DBs mysql.exp
Mail – everything in /var/qmail/mailnames/ mail.exp
Spamassassin config in /etc/mail/spamassassin/local.cf
Spamdyke config in /etc/spamdyke.conf
Mailman lists in /var/lib/mailman/lists/ maillists.exp
DNS /var/named/run-root/var/ to /root/syncs/named dns.exp
Mysql other db stuff – everything in /var/lib/mysql/mysql and /var/lib/mysql/tortix mysqltortix.exp

Would this be everything we need, at the moment they are being copied directly on to the corresponding files on server b, (running plesk 10.3) - The database breaks the admin panel of plesk b. Is this the best way of doing a backup? I've taken over the server from someone else, so just trying to establish where everything is going.

Or should I just, copy these all over to seperate directory, zip them all up, then give them a backup date as a filename? If plesk/server A goes down, just copy this back accross and restore?

[faris]

In theory, the backup made by Plesk should be restorable in a higher version, but this would be risky. So yes, same version ideally.

Another risk with Plesk's own backup is the danger of it not restoring due to some tiny error, and the huge length of time it takes to restore.

Thinking about it, you'd be so much better off using 4psa's TotalBackup. http://www.4psa.com/products-4psatbackup.html

This backs up (tar and zip plus other options) what you need to back up, dumps mysql databases, can do incremental, can do ssh and ftp transfers to remote server etc etc and costs very little. You'd store the backup on the other server. It isn't doing anything massively special, but it takes care of all the details for you so you don't have to worry about them. We use it to backup locally, then copy the backup to Amazon S3 (which is hard and requires doing silly things to achieve).

If server A dies, you restore server A to working, then restore data from your backup ontop of this to get back to where you were.

Alternatively, you can restore your backup on Server B as long as Server B has identical OS and probably Plesk to Server B.

There may be problems with this that I've not thought of. You should consult 4PSA sales about it all in case anything I've said isn't actually correct.

If you do contact them please tell them I sent you so I can get some brownie points

www.4psa.com

Faris.

[douglaseggleton]

Thanks for your help faris!

I think we are going to stick with rsync, I just have a query regarding Mysql, we are using an expects script early in the morning to copy the /var/lib/mysql/ directory. Should I be shutting MySQL down first?