Page 1 of 1

High cpu issue (need your advice, please)

Posted: Thu Feb 02, 2017 7:29 am
by copernic2006
Hello,
I am faced with a situation that I can not resolve and I ask for your help.
This situation has started since the ASL version 5 update. No changes were made to the site before the update(I had no high serverload value).
On a VPS with only one account, I have up to 500 incoming concurrent requests that target a page that does not exist on the account (RSS Feed), which causes a value up to 30 (serverload) and crash notifications d 'Apache.
The site is made under vivvo CMS (I know, it is no longer maintained, but the client wants to keep vivvo CMS), I naturally disabled the RSS feed from the admin, so the queries go on a Page 404.
I tried to mitigate the situation by configuring mod_fcgid and refining apache (but still MPM Prefork) and installing Nginx (reverse caching proxy) with an additional micro-cache layer.
Prohibiting access to the page (via htacess) does not solve the problem, the load (serverload) remains high, this problem occurs every 2 hours and apart from these picks, the site does not consume an enormous amount of resources.
Of course, during picks, I get the error: "502 Bad Gateway" and sometimes error 503.

Any suggestions to remedy this is welcome.


For information:
1. VPS config:
Ram : 8 Go
Processeur : 6 vCPU
Apache : 2.4.25
Mysql : 5.5.52-cll
WHM (cPAnel) : 60.0.36
ASL : 5-0-165
Nginx used : https://github.com/engintron/engintron

2. Apache Config
StartServers 5
MinSpareServers 5
MaxSpareServers 10
ServerLimit 500
MaxRequestWorkers 500
MaxRequestsPerChild 4000
Timeout 50 (I deliberately lowered this value to reduce CPU consumption)

3. Fastcgi configuration for PHP5
FcgidBusyTimeout 300
FcgidConnectTimeout 300
FcgidIOTimeout 300
FcgidIdleScanInterval 120
FcgidIdleTimeout 300
FcgidMaxRequestLen 268435456
FcgidMaxRequestsPerProcess 5000
FcgidMinProcessesPerClass 0
FcgidMaxProcessesPerClass 8
FcgidMaxProcesses 150
FcgidProcessLifeTime 120


4. Apache status:
Current Time: Thursday, 02-Feb-2017 12:00:48 CET
Restart Time: Wednesday, 01-Feb-2017 20:43:27 CET
Parent Server Config. Generation: 27
Parent Server MPM Generation: 26
Server uptime: 15 hours 17 minutes 20 seconds
Server load: 16.39 10.04 5.54
Total accesses: 111935 - Total Traffic: 2.1 GB
CPU Usage: u31.97 s83.33 cu0 cs0 - .209% CPU load
2.03 requests/sec - 39.9 kB/second - 19.6 kB/request
312 requests currently being processed, 5 idle workers
LLWLLLLRLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
RLLLLLLLLLLLLWLLLLLWLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLWLLLLLLLLLLLLLLLLLLLLLLLL
LLLLLLLLLLLLLLLLLLLLLWLWLLLLLLLLLLWLLLWLLLLLLWWWWWCWWWWW_____...
................................................................
................................................................
....................................................

Re: High cpu issue (need your advice, please)

Posted: Sun Feb 26, 2017 4:19 pm
by faris
In a different scenario, we were seeing something that resulted in a 404 cause a high load. I forget what script it was.

The problem was that whatever the script was, it was trying to search for a page that was related to whatever was being requested. This was causing a high php and MySQL load for some reason, and as a result a high server load.

From the sounds of things, however, your 404 is a real 404 as opposed to a 404 generated by the CMS.

So how about this:

Code: Select all

#prevent DoS
<IfModule mod_rewrite.c>
RewriteRule ^rssfeed\.php$ "http\:\/\/0\.0\.0\.0\/" [R=301,L]
</IfModule>
#end prevent DoS
Obviously you need to change rssfeed to whatever the filename really is.

We use something similar to block xmlrpc attacks in WP in non-ASL systems. Those can cause high loads themselves. And adding this kills the attack and the load.

I have no idea if it will help in your situation, but I suppose it might be worth a go?

EDIT: Those 500 requests - are they from the same IP? If so, mod_evasive should kill them ideally. That might be worth looking into.

Re: High cpu issue (need your advice, please)

Posted: Thu Mar 02, 2017 5:01 am
by prupert
A relatively high amount of Apache threads are busy in "Logging" state. Check your disk load, this may be the bottleneck.

Re: High cpu issue (need your advice, please)

Posted: Thu Mar 09, 2017 4:52 am
by copernic2006
sorry for the late response
@faris, thanks for the tip, I will include it immediately in the .htacess.
@ Prupert, yes, I also thought of a disk performance concern but I have seen nothing abnormal.

The problem did not arise anymore and this for 10 days, all I did was uninstall ASL, reboot the server and reinstall ASL again.
I think that thing go wrong during the update of ASL (from version 4 to 5)

Thank you, both for your help.