Default vs Compiled apache benchmarks

General Discussion of atomic repo and development projects.

Ask for help here with anything else not covered by other forums.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8330
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Default vs Compiled apache benchmarks

Unread post by scott »

Given the vast amounts of information about optimizing webservers, I recently had the opportunity to do some tests to see what the results of those methods were. One of the most often mentioned methods is in reference to re-compiling httpd on your native system, as using your own compiler will result in faster & more optimized code than the httpd package shipped by redhat and centos.

First let me state that this is a work in progress, and I'm looking for 3rd party verification of these results and improvements to the testing methods described. Benchmarks are an inconsistent way of measuring performance, and this is by no means definitive. Lastly while it may seem like I'm singling out cpanel in this test, its only because that is a more common environment to measure against.

In these test scenarios, rather than test to see how much performance can be squeezed out of a daemon, the intent is to determine the delta of performance difference between each implementation.

The test environment:
* centos 5.7 with all updates applied, x86_64 in a 1G KVM virtual machine
* mod_security is NOT installed
* This was then cloned into 2 separate VM's, with the default 2.2.3 httpd daemon from centos on one, and the latest httpd build using easyapache on the later.
* No two VM's were running on the KVM server at any time
* The testing machine is a separate system, and testing was performed over an isolated network.
* the systems are otherwise identical in terms of compilers, kernels, libraries, etc. Except where otherwise modified by easyapache & cpanel. No manual changes are made to the default or cpanel configs.
* Tests are performed in groups of 5, 5 times. The best score for each group is used in the final tally. This is to eliminate any disk caching
* Break between each test is 60 seconds.
* Apache benchmark (ab -n 50000 -c 100 <url>) is used to perform all tests.

Notes:
The centos httpd daemon consumed considerably more memory than its easyapache counterpart.
While the easyapache configuration minimized the use of apache modules, the default build does not. Nearly every possible module is enabled on the httpd package by default
All PHP modules were installed on the default build. These are (cli, common, gd, imap, mbstring, mysql, pdo, xml, bcmath, devel, ldap, mcrypt, mhash, mssql, ncurses, odbc, pear, pgsql, readline, snmp, soap, tidy, and xmlrpc).


Test 1: Static HTML test. This is to determine a baseline for the maximum number of requests per second the system could handle

Default httpd 2.2.3 - php 5.1 - 2118.72 2084.90 2079.82 2101.53 2038.12
Compiled httpd 2.2.21 - php 5.3.8 - 1852.82 1841.36 1822.64 1823.00 1802.68

Conclusion: Default apache performed better in all 5 tests, with a peak request rate of 2118.72/sec, performing 13.39% better than its counterpart.


Test 2: Basic php "Hello world" script
Default httpd 2.2.3 - php 5.1 - 1981.16 1946.88 2067.97 1940.40 2011.86
Compiled httpd 2.2.21 - php 5.3.8 - 1127.08 1150.86 1155.22 1142.74 1123.78
Compiled2 httpd 2.2.21 - php 5.3.8 - 1378.77 1361.93 1335.77 1371.42 1344.23

Conclusion: Default apache again performs better in this test, against the first configuration by a staggering 56.63%. At this point I thought something was wrong so I made a note to go into the easyapache and disable everything I could to get more out of it after performing test #3. It worked, albeit not considerably, the second test resulted in a 17% improvement. Final tally, default build 37% faster.


Test 3: Basic phpinfo test
Default httpd 2.2.3 - php 5.1 - 1856.41 1907.56 1902.89 1861.44 1892.12
Compiled httpd 2.2.21 - php 5.3.8 - 374.67 266.297 376.67 371.01 364.76
Compiled2 httpd 2.2.21 - php 5.3.8 - 403.52 400.20 399.61 401.98 401.71

Conclusion: Again very consistent results across all 3 tests for the default build, but a surprising drop for the compiled builds. The default build showing up 4.7 times faster than the compiled builds.

Final conclusion: This is hardly a complete or conclusive test. Far more scenarios need to be investigated and verified by others before this is anything more than a basic exercise. (For example, how do the atomic php & apache builds compare, how about a real world application like phpBB, joomla, or magneto?) But I can state that given these preliminary results the most accessible implementations of apache to the general public (native httpd, and one built via easyapache), it does *appear* that the apache design from redhat performs very well, and that users of this daemon are well served in terms of overall capacity and performance. It does not appear that these daemons under-perform against their locally compiled counterparts, and that in fact the locally compiled versions may be slower than the native builds.
breun
Long Time Forum Regular
Long Time Forum Regular
Posts: 2813
Joined: Sat Aug 20, 2005 9:30 am
Location: The Netherlands

Re: Default vs Compiled apache benchmarks

Unread post by breun »

Does Red Hat have some kind of magic secret turbo compiler? Or is the difference just be something in their SPEC file, but not something (or things) that has been identified yet?
Lemonbit Internet Dedicated Server Management
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8330
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Default vs Compiled apache benchmarks

Unread post by scott »

Great question, I don't know what it is exactly. My guess is its a combination of all those things. My next test will be to do this same series using the updated atomic httpd & php rpms so we can do a straight version to version compare. If that maintains the same result, then we could remove the version(s) of the packages themselves as being the source.

Also we havent ruled out my test methods artificially influencing the result either.

I have more testing along these lines in relation to mysql, and opcode compilers like apc ad eaccelerator. When the results are in I think I'm going to put this up on the website.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8330
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Default vs Compiled apache benchmarks

Unread post by scott »

Follow up, I have now completed the atomic-testing httpd 2.2.21 and atomic php 5.3.8 results:

Test #2: php hello world
atomic httpd 2.2.21 - php 5.3.8 - 1777.20 1758.54 1738.90
compiled2 httpd 2.2.21 - php 5.3.8 - 1378.77 1361.93 1335.77

Test #3: phpinfo()
atomic httpd 2.2.21 - php 5.3.8 - 1801.26 1577.38 1710.29
compiled2 httpd 2.2.21 - php 5.3.8 - 403.52 400.20 399.61

Conclusion: A direct version-to-version test returned more or less the same result. We can rule out that the newer version of apache & php being the culprit. The next tests will be to look into the build optimizations between the two.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8330
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Default vs Compiled apache benchmarks

Unread post by scott »

Minor update with some cgi, fastcgi, and suphp benchmarks.

Test 5: 50000 / 100 / php hello world
default - httpd 2.2.21 - fcgi 5.3.9 - 663.11 697.48 661.67
default - httpd 2.2.21 - cgi 5.3.9 - 15.21 14.67 14.11
default - httpd 2.2.21 - suphp 5.3.9 - 12.23 12.99 12.42

Notes:In testing the cgi & suphp tests cannot run the full 100 connection test, and had to be stopped about 50% the way through (taking over an hour to get to that point.) Fastcgi does appear to be the optimal method if you are using the cgi approach to invoking php. I will try and re-test these with older versions of PHP to rule out performance bottlenecks in php 5.3.9 at a later time.
Post Reply