Cluster, Cloud or something

[laughingbuddha]

Hi,

I'm developing a blogging platform of sorts, and prior to moving the beta version to a dedicated server, I'm forward thinking about a scalable hosting architecture.

My thought is that as the service becomes more and more popular, I would like to chuck more servers at it, so as to handle expansion and also redundancy in case of server failure.

I'm looking at clustering currently, although to be honest this is new ground to me, but I want to avoid using a third party provider like AWS EC2, and instead own the boxes myself. More control over the service, content, and also the cost.

So I heard about clustering, and wondered if that is the way to go, and also if Plesk can operate on a cluster environment.

[scott]

there are some services like memcached or redis(my preference) that would let you create an object cache. Those are a great low-complexity way to accelerate service (and cluster easily) and reduce db overhead.

[laughingbuddha]

Oh right, I'll look into that.

I'm wondering though if a server cluster is also a good option. Thinking is that the project I developed will server content in much the same way as Tumblr (well in the simplest terms). Even though I've built cache functions into the application to reduce some db requests, I'm wondering if all the PHP work load would put strain on the CPU(s) of the server.

My thinking is if I plan the infrastructure now, and begin to implement the basics, when/if the demand grows, I chuck servers at the cluster to handle the load. Rather than having to revisit the infrastructure issue down the line again, which might then require a moving the whole application/service to another setup/server, thus causing down time during the move.

To combat some down time, I recently switched my domains NS and DNS from my own personal name servers, to Amazons Route 53. I knew I would have to at some point any way, so I preempted myself and made the change prior to starting the Alpha/early beta testing phase.

[scott]

The config we use for this mobile gaming platform (100 million+ users) is a 3 tier design:

Tier 1: Load balancer, this uses nginx and varnish. Nginx handles SSL traffic, Varnish handles http. We ended up using varnish because nginx wasn't doing that well with balancing traffic to layer 2. This may have been resolved by newer versions of nginx. However during testing we found varnish to be just a better all around caching platform. Load is non-existent on these systems

The down side, both of these have issues with invalidating the cache cleanly. Varnish is much better than nginx (at the time, again this could have changed). Varnish uses memory, a lot of it.

Tier 2: Processing layer, Apache + PHP. APC is used to optimize the code, memcached is used to manage sessions across nodes. Sync'ing code is handled with saltstack. Redis is being added into this layer to act as a PHP object cache, to reduce load on layer 3 (database).

The down side: Logging, if that matters, is kind of pain. We dont look at that often. Sync'ing code is much easier with saltstack, but we're still early in working through all the ways to break it. Its promising though.

Tier 3: Database layer, mariadb using galera cluster. Tier 2 clients talk to this through haproxy. Otherwise not much fancy stuff here.

The down side: Not much here, early on we ran into some sync issues with galera because the port speeds were locked at 100mbit on the switch. Once that was squared away everything was fine. Load on the DB's averages at 1-2, with redis it drops to 0.

[laughingbuddha]

Interesting, thanks Scott. A lot there for me to take in and research.

My current software design is a PHP 5.x MVC framework, using MySQL on the db end. Caching is basic, just writing files to the server with expire datetime. Media is currently handled through an S3 subdomain bucket, with Cloudfront on the front end. Even though Plesk 11 has nginx, I switch to Apache as nginx had all sorts of permission issues and code compatibility problems. The subdomains and domains are handled with wildcard dns and modifications to the vhost(s), plus a bucket load of validation and security to filter out bad requests.

I'm due to install a new box at some point soon, but I need to think through the best way to do this. Maybe even hire a consultant if I get over my head. Had considered building out an environment in my home office, to test and experiment. But the over all aim is to keep costs as low as possible as I'm bootstrapping this thing at the moment.