Atomicorp

Starting sshd: /etc/init.d/sshd: line 128: 3318 Killed $SSHD $OPTIONS
[FAILED]


I have emails every minute where SSHD is trying to restart. I had a number of entries in the event logs for

Rules
60038 Process Monitor: Failed to spawn service
61027 Denied a RWX mprotect event. An application just attmpted to use the mprotect function to bypass memory protection functions in the kernel.
61028 Denied an untrusted non system library binary from hooking an application.

I disabled all these rules and still can't get it to start

Joe

did you read those articles? they explain what to do

If SSHD is triggering this rule:

https://www.atomicorp.com/wiki/index.php/HIDS_61027

Then its either been replaced by a backdoored version, or someone has horribly misconfigured it so that its trying to do something very dangerous on your system. Either way, SSH never ever does this otherwise, and does not need to do this. This will only happen if your system has been either compromised, or someone has done something very very wrong to sshd. Either way, its bad.

Whats the exact event log message on your system for 60127, for example you'll see something like this:

May 5 09:24:02 host kernel: grsec: From 1.2.3.4: denied RWX mprotect of /lib64/ld-2.12.so by /usr/sbin/sshd[sshd:3705] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/sshd[sshd:3642] uid/euid:0/0 gid/egid:0/0

Please post the log message so we can see whats happening on your system. Also, thankfully disabling that rule will not disable that protection, it just tells ASL to not alert you that your system is trying to be compromised and ASL is preventing the compromise of your system.

This is what I see in the event log

srv01 kernel: grsec: denied RWX mprotect of /usr/sbin/sshd by /usr/sbin/sshd[sshd:19653] uid/euid:0/0 gid/egid:0/0, parent /etc/rc.d/init.d/sshd[sshd:19639] uid/euid:0/0 gid/egid:0/0

This was the log for 61028

srv01 kernel: grsec: denied exec of usermode helper binary /usr/libexec/abrt-hook-ccpp located outside of /sbin and system library paths

SSHD shouldnt be doing that and definitely doesnt need to do that. ASL is definitely protecting you from something bad. Either someone has replaced sshd with a backdoored version or someone has seriously misconfigured sshd on your system. Either way, do not allow this. Your system is either compromised, or is about to be compromised.

The first thing I would do is check the file integrity watches in ASL to see when that file was changed. If this just started to happen, then you know it was very recent.

If the files integrity is valid, that is its not been replaced, then someone modified the execstack settings on sshd to allow it to this this dangerous operation it doesnt need to do. You'll need to remove that, however you really need to confirm that sshd hasnt been replaced before you do that. I definitely have seen backdoored versions of sshd do this.

hmm ok so not sure of an appropriate approach to fix this at this point

I haven't changed SSHD in a couple years the only thing I did was change the port and this problem just started this past week so something is wrong.

Joe

Step 1: confirm it hasnt been replaced by someone else

Step 2: if someone set RWX mprotect on sshd this can happen as well, but dont assume it was that. Thats a weird thing for someone to do, but people do it all the time on other things like PHP thinking they need to. So its not impossible, but very strange for someone to do that. Definitely start with step 1, I've seen this happen with backdoored versions of SSH (probably because the bad guys thought they needed to do this as well)

I am the only person that monitors and updates this server and I haven't changed anything recently. Looks like I will have to make a trip to the DataCenter

Looks like you have KVM access, so before you do that check the file integrity reports from ASL to see if it reported any changes to SSHD. Also, check to see who else has logged into the system, perhaps the bad guys stole credentials to the system and logged in as root. The real time file integrity checks will have a record of any changes to /usr/sbin/sshd provided the defaults were left in place for the file integrity system.

NO KVM access and I can't SSH into it. I just checked the file integrity and don't see anything pertaining to SSHD. I am logged into my WHM interface

What do you see in the ASL file integrity reports inside the ASL web console?

I was having a different problem with ssh and uninstalled and reinstalled via yum and it solved my issues

https://www.atomicorp.com/forum/viewtop ... f=3&t=7915

So I removed SSH and reinstalled it via Cpanel and still having the same issue which does not make sense?

sshd is still trying to smash your stack? If so, then thats not the real sshd, someones modified it or replaced. The real sshd doesnt do that.