Ossec Agent stays in Never connected state

Support/Development for OSSEC
User avatar
pteros
Forum User
Forum User
Posts: 8
Joined: Thu May 20, 2021 10:15 am

Ossec Agent stays in Never connected state

Unread post by pteros »

Hello folks,

I installed an ossec-hids agent on a freebsd PPC 12.2 using the binary package, via the pkg install ossec-hids-agent. Now I have a ossec-hids-agent-3.6.0_1 up and running, but it is not seen by the server which is seeing the other agents perfectly well (they are a bunch of freebsd intel based of different versions).

The server claims to be, via ossec-analysisd -V:

OSSEC HIDS v3.6.0 - OSSEC Foundation

If I run tcpdump, I see:

Code: Select all

tcpdump -X -i bge0 src host saguarone
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bge0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:53:22.716459 IP saguarone.23261 > EX.www.inpe.it.1514: UDP, length 73
	0x0000:  4500 0065 dd1a 0000 4011 d475 d42d 907b  E..e....@..u.-.{
	0x0010:  d42d 9021 5add 05ea 0051 6d3e 3a07 5383  .-.!Z....Qm>:.S.
	0x0020:  c89f c05b ad0f bf9e 5cfa b5ef e297 e23c  ...[....\......<
	0x0030:  af52 dec8 030b 9556 9723 090b 52af 3bbd  .R.....V.#..R.;.
	0x0040:  2f44 e315 6a84 04ac 299a 193c a4ec dfc5  /D..j...)..<....
	0x0050:  89c3 7e6c 95fc 62b5 311a 9d5a 9156 cc60  ..~l..b.1..Z.V.`
	0x0060:  ec3c 1be4 3a                             .<..:
18:53:24.325022 IP saguarone > EX.www.inpe.it: ICMP echo reply, id 60308, seq 0, length 64
	0x0000:  4500 0054 3b23 0000 4001 768e d42d 907b  E..T;#..@.v..-.{
	0x0010:  d42d 9021 0000 1112 eb94 0000 0082 c1ea  .-.!............
	0x0020:  0058 5591 0809 0a0b 0c0d 0e0f 1011 1213  .XU.............
	0x0030:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
	0x0040:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
	0x0050:  3435 3637                                4567
18:53:25.324786 IP saguarone > EX.www.inpe.it: ICMP echo reply, id 60308, seq 1, length 64
	0x0000:  4500 0054 3b25 0000 4001 768c d42d 907b  E..T;%..@.v..-.{
	0x0010:  d42d 9021 0000 0fad eb94 0001 0082 c1eb  .-.!............
	0x0020:  006d 56df 0809 0a0b 0c0d 0e0f 1011 1213  .mV.............
	0x0030:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
	0x0040:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
	0x0050:  3435 3637                                4567
18:53:26.332956 IP saguarone > EX.www.inpe.it: ICMP echo reply, id 60308, seq 2, length 64
	0x0000:  4500 0054 3b27 0000 4001 768a d42d 907b  E..T;'..@.v..-.{
	0x0010:  d42d 9021 0000 a0df eb94 0002 0082 c1ec  .-.!............
	0x0020:  007c c59b 0809 0a0b 0c0d 0e0f 1011 1213  .|..............
	0x0030:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
	0x0040:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
	0x0050:  3435 3637                                4567
18:53:27.302640 IP saguarone > EX.www.inpe.it: ICMP echo reply, id 60308, seq 3, length 64
	0x0000:  4500 0054 3b29 0000 4001 7688 d42d 907b  E..T;)..@.v..-.{
	0x0010:  d42d 9021 0000 4b64 eb94 0003 0082 c1ed  .-.!..Kd........
	0x0020:  008c 1b05 0809 0a0b 0c0d 0e0f 1011 1213  ................
	0x0030:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
	0x0040:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
	0x0050:  3435 3637                                4567
18:53:28.303678 IP saguarone > EX.www.inpe.it: ICMP echo reply, id 60308, seq 4, length 64
	0x0000:  4500 0054 3b2b 0000 4001 7686 d42d 907b  E..T;+..@.v..-.{
	0x0010:  d42d 9021 0000 ed73 eb94 0004 0082 c1ee  .-.!...s........
	0x0020:  0099 78e6 0809 0a0b 0c0d 0e0f 1011 1213  ..x.............
	0x0030:  1415 1617 1819 1a1b 1c1d 1e1f 2021 2223  .............!"#
	0x0040:  2425 2627 2829 2a2b 2c2d 2e2f 3031 3233  $%&'()*+,-./0123
	0x0050:  3435 3637                                4567
18:53:28.849327 IP saguarone.23261 > EX.www.inpe.it.1514: UDP, length 73
	0x0000:  4500 0065 dd1b 0000 4011 d474 d42d 907b  E..e....@..t.-.{
	0x0010:  d42d 9021 5add 05ea 0051 69f8 3a20 4cec  .-.!Z....Qi.:.L.
	0x0020:  f64d 9cdd ec72 a534 abff dc44 463d 175b  .M...r.4...DF=.[
	0x0030:  5c73 99fc c350 2937 48ca 8942 dd7a 1b77  \s...P)7H..B.z.w
	0x0040:  b96b 00b2 093e f874 14b1 4932 9da5 08e2  .k...>.t..I2....
	0x0050:  9a14 52fa 23cf 4672 f1d9 e0c4 a5e9 a90c  ..R.#.Fr........
	0x0060:  d39c 4c22 d3                             ..L".
18:53:32.893173 IP saguarone.23261 > EX.www.inpe.it.1514: UDP, length 73
	0x0000:  4500 0065 dd1c 0000 4011 d473 d42d 907b  E..e....@..s.-.{
	0x0010:  d42d 9021 5add 05ea 0051 5820 3a2d 8e5c  .-.!Z....QX.:-.\
	0x0020:  4c7d 0f1c a54c 3dd0 6571 1ed9 fd46 5e61  L}...L=.eq...F^a
	0x0030:  acda 7dc4 da32 4167 2486 c2fd 7224 3de2  ..}..2Ag$...r$=.
	0x0040:  fb2c c874 f9a2 79f5 8bfd 880b 46c8 4fd8  .,.t..y.....F.O.
	0x0050:  56e4 390c 02ca ea46 08d2 da2f 2376 8c10  V.9....F.../#v..
	0x0060:  ce47 eebf 0a                             .G...
and nothing in the logs. Is there a way to debug this?

Thanks in advance,

Luciano.
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: Ossec Agent stays in Never connected state

Unread post by mikeshinn »

easiest way is to start remoted from the command line and start it with -d which puts into debug mode.
User avatar
pteros
Forum User
Forum User
Posts: 8
Joined: Thu May 20, 2021 10:15 am

Re: Ossec Agent stays in Never connected state

Unread post by pteros »

ok, I started it with -d -d -d -f and got:

2021/05/31 14:06:06 ossec-remoted(1403): ERROR: Incorrectly formatted message from '212.45.144.123'.
2021/05/31 14:06:12 ossec-remoted(1403): ERROR: Incorrectly formatted message from '212.45.144.123'.
2021/05/31 14:06:16 ossec-remoted(2202): ERROR: Error uncompressing string.
2021/05/31 14:06:21 ossec-remoted(1403): ERROR: Incorrectly formatted message from '212.45.144.123'.
2021/05/31 14:06:27 ossec-remoted(1403): ERROR: Incorrectly formatted message from '212.45.144.123'.
2021/05/31 14:06:29 ossec-remoted(1403): ERROR: Incorrectly formatted message from '212.45.144.123'.
2021/05/31 14:06:35 ossec-remoted(1403): ERROR: Incorrectly formatted message from '212.45.144.123'.
....

where could I look further on?

Many thanks,

Luciano.
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: Ossec Agent stays in Never connected state

Unread post by mikeshinn »

So this error means whatevers trying to connect isnt using the right protocol (which could be anything, nmap, telnet, etc.). If thats what you were doing, thats what that means. If not, what agent and version is running on the endpoint, and was this something trying to send events to the hub for syslog as well? (port 514 as opposed to port 1514)

Code: Select all

 2021/05/31 14:06:06 ossec-remoted(1403): ERROR: Incorrectly formatted message from '212.45.144.123'.
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: Ossec Agent stays in Never connected state

Unread post by mikeshinn »

Code: Select all

 2021/05/31 14:06:16 ossec-remoted(2202): ERROR: Error uncompressing string.
That means something tried to send a message of one size, and it was actually of another. Was this a device sending messages to the syslog listener on 514, or an agent on 1514, and if the later, which version and platform?
mmudabbir
New Forum User
New Forum User
Posts: 2
Joined: Fri May 28, 2021 6:55 am

Re: Ossec Agent stays in Never connected state

Unread post by mmudabbir »

Hello,

This error usually occurs when you have mistakes in config file, or if you have not given the proper key.

Follow this,
1) go to rids folder in your ossec agent folder and delete the file with agent id you have just added eg. "001". If dont have any other agents simply delete all.
2) delete you agent from the server. restart the HIDS service.
3) Add the agent again on your server and copy the key.
4) On your end-host paste your key, give your server IP, and save. Then restart the agent.
5) On your server restart the HIDS service again.

Note: Make sure port UPD/TCP 1514 is open. Also if there is a firewall you have to write outbound/inbound rules for statefull connection.
User avatar
pteros
Forum User
Forum User
Posts: 8
Joined: Thu May 20, 2021 10:15 am

Re: Ossec Agent stays in Never connected state

Unread post by pteros »

The latter. It is trying to speak through port 1415.
This is the version:

Code: Select all

./ossec-agentd -V
 
OSSEC HIDS v3.6.0 - OSSEC Foundation
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: Ossec Agent stays in Never connected state

Unread post by mikeshinn »

Should be port 1514, is it trying 1415 on your system?
User avatar
pteros
Forum User
Forum User
Posts: 8
Joined: Thu May 20, 2021 10:15 am

Re: Ossec Agent stays in Never connected state

Unread post by pteros »

mmudabbir wrote: Wed Jun 02, 2021 3:24 am Hello,

This error usually occurs when you have mistakes in config file, or if you have not given the proper key.

Follow this,
1) go to rids folder in your ossec agent folder and delete the file with agent id you have just added eg. "001". If dont have any other agents simply delete all.
2) delete you agent from the server. restart the HIDS service.
3) Add the agent again on your server and copy the key.
4) On your end-host paste your key, give your server IP, and save. Then restart the agent.
5) On your server restart the HIDS service again.
Tried this 3 times.
No effect.
User avatar
pteros
Forum User
Forum User
Posts: 8
Joined: Thu May 20, 2021 10:15 am

Re: Ossec Agent stays in Never connected state

Unread post by pteros »

mikeshinn wrote: Thu Jun 03, 2021 3:28 pm Should be port 1514, is it trying 1415 on your system?
1514, eventually. My typo, sorry.
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: Ossec Agent stays in Never connected state

Unread post by mikeshinn »

How did you provision the key for the agent?
User avatar
pteros
Forum User
Forum User
Posts: 8
Joined: Thu May 20, 2021 10:15 am

Re: Ossec Agent stays in Never connected state

Unread post by pteros »

mikeshinn wrote: Thu Jun 10, 2021 10:18 am How did you provision the key for the agent?
The very same way I did for the other agents (that are working): added the agent on the server via manage_agents, extracted the key, copied in my clipboard, restarted ossec on the server, run manage_agents on the agent and pasted the key from my clipboard and restarted the agent. I tied few times deleting the agent first.

Thanks again,

Luciano.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Ossec Agent stays in Never connected state

Unread post by scott »

OK so at this point (correct me if any of these arent validated)

1. Key: Probably OK, unless theres a transcribing error.
2. remoted is listening on port UDP 1514
3. agent traffic is confirmed to be reaching the server on UDP 1514

Never connected is a state you'd get only if initial session packet doesnt get accepted. Things that could do that are either the packet is not getting to the daemon (issue 2, or 3), or the key its using isnt accepted by it (1).

So other things to confirm, if you change the key, restart the agent, is remoted getting any traffic/processing anything at all. There are some internal_options.conf settings you can enable to increase debugging here, but my gut feeling is its either the key is changed/service not restarted (would be logged as an invalid key in ossec.log) or the traffic is never getting past a filter/to the right port for remoted to process it.

4. This is very fringe, but Ive seen filters block by *size* of packet. So some get through, some dont. This typically manifests as connected though, since you'd get that initial heartbeat that increments the counter, it just doesnt work otherwise.
User avatar
pteros
Forum User
Forum User
Posts: 8
Joined: Thu May 20, 2021 10:15 am

Re: Ossec Agent stays in Never connected state

Unread post by pteros »

scott wrote: Thu Jun 10, 2021 4:29 pm OK so at this point (correct me if any of these arent validated)

1. Key: Probably OK, unless theres a transcribing error.
2. remoted is listening on port UDP 1514
3. agent traffic is confirmed to be reaching the server on UDP 1514
Yes I can confirm all the 3 points above.
Transcribing error is something I consirered, especially due to the fact that the agent runs on a Powerpc under FreeBSD, which is big endian on this platform. So I installed the agent (by compiling the source) on a linux machine that is little endian on PPC64 and got the very same results. The Intel (amd64) machine do connect to the same server whithout any problem.
scott wrote: Thu Jun 10, 2021 4:29 pm Never connected is a state you'd get only if initial session packet doesnt get accepted. Things that could do that are either the packet is not getting to the daemon (issue 2, or 3), or the key its using isnt accepted by it (1).

So other things to confirm, if you change the key, restart the agent, is remoted getting any traffic/processing anything at all. There are some internal_options.conf settings you can enable to increase debugging here, but my gut feeling is its either the key is changed/service not restarted (would be logged as an invalid key in ossec.log) or the traffic is never getting past a filter/to the right port for remoted to process it.
You mean I should delete the agen on the server, create it again so it issues a new key, restart the server, change the key on the agent and restart it?

Thanks again,

Luciano.
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Ossec Agent stays in Never connected state

Unread post by scott »

Nothing like that, just that when you change the key you have to restart the agent, otherwise its using the older key still.

So next take a look in /var/ossec/etc/internal_options.conf and enable the debug settings for remoted. You can do the same on the agent side for agentd, and then see if the output there is helpful at all
Post Reply