Page 3 of 5
Re: nf_conntrack: table full, dropping packet
Posted: Tue Nov 30, 2010 1:16 pm
by chrismcb
Honest answer - I don't know!
These are supposed to be the default rules from the Plesk Firewall module with a port opened for ASL and SSH changed from 22 to 3112 for private access.
By your question, I presume they shouldn't be there?
Re: nf_conntrack: table full, dropping packet
Posted: Tue Nov 30, 2010 3:37 pm
by biggles
I am also running the Plesk firewall on one server and it has the same forwarding rules present...
Re: nf_conntrack: table full, dropping packet
Posted: Tue Nov 30, 2010 3:57 pm
by scott
Unless you were running as a gateway for something, no those arent needed. Im not sure thats the root of your problem though, unless something crazy were going on (running a proxy for a ddos or something)
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 7:55 am
by chrismcb
OK, finally got round to fixing this - rather than just restarting apache.
I turned apache debugging on and got some (many!) core dumps.
The highlights of them all are:
Code: Select all
Core was generated by `/usr/sbin/httpd'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007a3af3961be1 in __libc_res_nquery () from /lib64/libresolv.so.2
Looking at the contents of /proc/net/ip_conntrack, I see thousands of references to my DNS servers.
I've now found the cause, but can anyone offer any advice on a solution?
Thanks
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 10:20 am
by scott
Did you ever remove those forwarding rules?
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 11:48 am
by chrismcb
Hi Scott,
No, I didnt remove them (until now) as biggles also had them.
I've removed all in the "FORWARD" chain and will keep an eye on the system again.
Thanks
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 12:33 pm
by scott
The other thing I'd do after this is to remove those nf_conntrack modules. Test it out with the purged FORWARD rules first
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 12:51 pm
by chrismcb
Thanks - is the nf_conntrack module not needed?
Looking at the IPTABLES list, it shows states in the rules - surely those wouldn't work if the module wasn't loaded?
Code: Select all
ACCEPT tcp -- anywhere anywhere tcp dpt:11444 state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:pcsync-https state NEW
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:http state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:https state NEW
ACCEPT udp -- anywhere anywhere udp dpt:domain state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:domain state NEW
ACCEPT udp -- anywhere anywhere udp dpt:tftp state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:tftp state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:smtp state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:pop3 state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:imap state NEW
ACCEPT udp -- anywhere anywhere udp dpt:ntp state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:ftp-data state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:ftp state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:mysql state NEW
ACCEPT udp -- anywhere anywhere udp dpt:mysql state NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:cddbp-alt state NEW
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 1:17 pm
by mneese77
I have those default forwarding rules also when viewed via ssh
, but when looking at my plesk firewall interface, i see no obvious way to remove these forwarding rules....
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 1:23 pm
by chrismcb
Same with me - I manually removed them from the command line with:
Then restarted iptables.
Not saved it to the system yet, but will do if it shows an improvement.
I still don't believe that this is the cause of the problem, but every little helps!
EDIT: iptables switch is -F for flush rather than -X which I had before.
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 1:28 pm
by mneese77
If you removed them with that simple command, then re-started the table, doesn't that save them to the system? Or is there another step to this process?
Re: nf_conntrack: table full, dropping packet
Posted: Tue Dec 07, 2010 1:37 pm
by chrismcb
I believe it will save for the current session, but if you want to save permenantly, you must use:
If you don't, your previously saved rules will be reapplied on restart.
Re: nf_conntrack: table full, dropping packet
Posted: Thu Dec 09, 2010 8:05 am
by chrismcb
Just had the same issue again - with the forward rules removed.
Anyone able to offer advice as to the error message from the apache crash dump?
Code: Select all
Core was generated by `/usr/sbin/httpd'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007a3af3961be1 in __libc_res_nquery () from /lib64/libresolv.so.2
The file is provided by glibc and the currently installed version (latest) is:
Re: nf_conntrack: table full, dropping packet
Posted: Fri Dec 10, 2010 12:17 pm
by chrismcb
Anyone able to help?
I'm now finding myself having to use my mobile to SSH into the server when this happens and restart apache to clear the connections - while hoping none of my clients catch on that the CPU is being thrashed and websites/emails are temporarily down.
Re: nf_conntrack: table full, dropping packet
Posted: Fri Dec 10, 2010 1:12 pm
by chrismcb
I've tried to go further with the apache dumps that I gathered and hoping to get down to the function call which caused the problem.
I'm using gdb and have attached the dump and trace.
Following sources online (specifically
http://www.omh.cc/blog/2008/mar/6/fixin ... aused-php/), I have been trying to use the following command:
Code: Select all
dump_bt executor_globals.current_execute_data
This should return a nicer backtrace - however, I'm getting:
Code: Select all
Attempt to extract a component of a value that is not a structure.
Any help is appreciated...