nf_conntrack: table full, dropping packet

General Discussion of atomic repo and development projects.

Ask for help here with anything else not covered by other forums.
BruceLee
Forum Regular
Forum Regular
Posts: 879
Joined: Sat Mar 28, 2009 6:58 pm
Location: Germany

Re: nf_conntrack: table full, dropping packet

Unread post by BruceLee »

sqlite was discussed here somewhere.
chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

Yep - got it.

To do with Sitebuilder (which i don't have).

Once that was removed, everything updated fine.


Thanks!
chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

OK, last hurdle - i think (hope!).

When i first installed ASL, I stupidly gave it an admin username which I hadn't yet created.

As a result, it said it wasnt going to configure SSH - which i thought was fine.


I then created the user and reran asl -s -f, but this error is still appearing.

Code: Select all

 Checking Admin users
    Checking [user] directory /home/[user]: found      [OK]
    Checking [user] authorized_keys: not found           [FAILED]
    Valid Admin users detected: no                         [HIGH]
    WARNING: SSH will not be reconfigured at this time.
Can you advise how to "reconfigure" SSH?
I've tried everything I can think of - bar re-installing ASL!
BruceLee
Forum Regular
Forum Regular
Posts: 879
Joined: Sat Mar 28, 2009 6:58 pm
Location: Germany

Re: nf_conntrack: table full, dropping packet

Unread post by BruceLee »

chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

Thanks, I've already done that and I am looging in with a key instead of password.

It just seems as if ASL doesnt recognise that because the user was created after the install.


Is there any harm in trying a re-install of ASL through the sh script?
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: nf_conntrack: table full, dropping packet

Unread post by scott »

You just need to define the user in /etc/asl/config under "ADMIN_USERS". Thats a safety check to keep you from locking yourself out if you dont have valid keys.
chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

Thanks, but the username is already in there!

Code: Select all

NOTIFY="yes"
EMAIL="xxxx"
HOSTNAME="xxxx"
ADMIN_USERS="xxxx"
IP_WHITELIST="/etc/asl/whitelist"
SYSTEM_TYPE="webserver"
AUTOMATIC_UPDATES="daily"
UPDATE_TYPE="all"
RESTART_APACHE="yes"
APACHE_RESTART_COMMAND="/etc/init.d/httpd restart"
ASL_USER="tortix"
I have removed it from above, but I can confirm it is spelled correctly etc and the user is valid - it's the one i log in with before SU-ing to root!
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: nf_conntrack: table full, dropping packet

Unread post by scott »

It couldnt find your key then
chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

The key is located in:

Code: Select all

/home/xxx/.ssh/
It has the filename:

Code: Select all

authorized_keys2
And is owned by that user with CHMOD set to 600.


I believe that to be correct?
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: nf_conntrack: table full, dropping packet

Unread post by scott »

No its not, that is deprecated. The file is :

~/.ssh/authorized_keys
chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

This seemed to have cleared itself up after a couple of reboots!

Back up and running now on the new server (a late night/early morning for me last night) and just going through some final tweaking now.



Thanks everyone
chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

OK, so it looks like this original problem was not related to the previous server attack - it's happening again.
Nov 30 10:50:09 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:50:13 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:50:52 xxxxx last message repeated 3 times
Nov 30 10:51:04 xxxxx last message repeated 3 times
Nov 30 10:51:16 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:51:19 xxxxx last message repeated 3 times
Nov 30 10:51:25 xxxxx clamd[19785]: SelfCheck: Database status OK.
Nov 30 10:51:29 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:51:43 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:51:47 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:51:54 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:52:00 xxxxx ntpd[3586]: kernel time sync enabled 4001
Nov 30 10:52:04 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:52:18 xxxxx last message repeated 2 times
Nov 30 10:52:26 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:52:38 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:52:58 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:53:23 xxxxx last message repeated 5 times
Nov 30 10:53:30 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:54:06 xxxxx last message repeated 2 times
Nov 30 10:54:28 xxxxx last message repeated 4 times
Nov 30 10:55:24 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:55:25 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:55:44 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:56:12 xxxxx kernel: nf_conntrack: table full, dropping packet.
Nov 30 10:56:46 xxxxx last message repeated 3 times
Nov 30 10:56:55 xxxxx last message repeated 2 times
Nov 30 10:57:28 xxxxx kernel: nf_conntrack: table full, dropping packet.
I've checked online for other solutions - specifically increasing the /proc/sys/net/ipv4/netfilter/ip_conntrack_max
value. It is, however, set to 65536 - its max.

Can anyone offer any advice on where to look to find out what is filling the connection table?


Thanks
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: nf_conntrack: table full, dropping packet

Unread post by mikeshinn »

Its a safety feature in all Linux kernels, you are tracking too many connections, which can be caused by either misconfigured iptable rules, or just WAY too much traffic. And by way too much, I mean insanely way too much. Its more likely the former, and the later could be caused by a DDOS bounce attack, or something like that (like DNS bounce attacks for example).

Check your firewall rules, and fire up a sniffer and see what kind of traffic is going on. If memory serves, you can set nf_conntrack_max really high, I believe its a 32 bit int, so billions of connections should be possible if you have the RAM for it, but 65K is pretty big, so if you are over that limit check your rules and traffic first - somethings going on there that shouldnt be.
chrismcb
Forum Regular
Forum Regular
Posts: 293
Joined: Tue Nov 23, 2010 7:30 am
Location: Glasgow, UK

Re: nf_conntrack: table full, dropping packet

Unread post by chrismcb »

Thanks, I'll look at a packet sniffer - in the mean time, here are the IPTables settings:

Code: Select all

 iptables -L
Chain INPUT (policy DROP)
target     prot opt source               destination
DROP       all  --  62.119.28.251        anywhere
DROP       all  --  user-3c2h5u6.cable.mindspring.com  anywhere
DROP       all  --  dsl88-250-50624.ttnet.net.tr  anywhere
DROP       all  --  51-130-178-94.pool.ukrtel.net  anywhere
DROP       all  --  122-36-135-95.pool.ukrtel.net  anywhere
DROP       all  --  88.103.158.23        anywhere
DROP       all  --  host6-133-dynamic.25-79-r.retail.telecomitalia.it  anywhere
DROP       all  --  Static-115.191.96.14.tataidc.co.in  anywhere
DROP       all  --  86.99.114.254        anywhere
DROP       all  --  91.75.74.12          anywhere
DROP       all  --  localhost            anywhere
DROP       all  --  95-174-214-190.nts.su  anywhere
DROP       all  --  121-72-232-248.cable.telstraclear.net  anywhere
DROP       all  --  ABTS-KK-Dynamic-077.141.167.122.airtelbroadband.in  anywhere
DROP       all  --  41.209.75.103        anywhere
DROP       all  --  ep--pc77.static.otenet.gr  anywhere
DROP       all  --  189105032004.user.veloxzone.com.br  anywhere
DROP       all  --  home-pool-164-2.com2com.ru  anywhere
DROP       all  --  195.135.239.5        anywhere
DROP       all  --  109.70.71.60         anywhere
DROP       all  --  144.28.broadband6.iol.cz  anywhere
DROP       all  --  95.67.176.171        anywhere
DROP       all  --  ppp-94-64-145-78.home.otenet.gr  anywhere
DROP       all  --  178.187.137-121.xdsl.ab.ru  anywhere
DROP       all  --  86.35.21.209         anywhere
DROP       all  --  net77.186.188-253.tmn.ertelecom.ru  anywhere
DROP       all  --  250-111-124-91.pool.ukrtel.net  anywhere
DROP       all  --  213.234.13.130       anywhere
DROP       all  --  g43252.upc-g.chello.nl  anywhere
DROP       all  --  41.64.240.72         anywhere
DROP       all  --  adsl190-25105081.dyn.etb.net.co  anywhere
DROP       all  --  sge91-5-88-160-227-197.fbx.proxad.net  anywhere
DROP       all  --  71-33-114-134.spkn.qwest.net  anywhere
DROP       all  --  173-120-215-50.pools.spcsdns.net  anywhere
DROP       all  --  165046.yiuwa.com     anywhere
DROP       all  --  bb171804.virtua.com.br  anywhere
DROP       all  --  ppp95-165-13-236.pppoe.spdop.ru  anywhere
DROP       all  --  186.143.190.167      anywhere
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
REJECT     tcp  --  anywhere             anywhere            tcp flags:!FIN,SYN,RST,ACK/SYN reject-with tcp-reset
DROP       all  --  anywhere             anywhere            state INVALID
ACCEPT     all  --  anywhere             anywhere
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ksysguard
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:30000
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:pcsync-https
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:cddbp-alt
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ftp
DROP       tcp  --  anywhere             anywhere            tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:submission
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:smtp
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:smtps
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:pop3
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:pop3s
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:imap
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:imaps
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:poppassd
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:mysql
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:postgres
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:9008
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:glrpc
ACCEPT     udp  --  anywhere             anywhere            udp dpt:netbios-ns
ACCEPT     udp  --  anywhere             anywhere            udp dpt:netbios-dgm
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:netbios-ssn
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:microsoft-ds
ACCEPT     udp  --  anywhere             anywhere            udp dpt:openvpn
ACCEPT     udp  --  anywhere             anywhere            udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:domain
ACCEPT     icmp --  anywhere             anywhere            icmp type 8 code 0
ACCEPT     all  --  anywhere             anywhere

Chain FORWARD (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
REJECT     tcp  --  anywhere             anywhere            tcp flags:!FIN,SYN,RST,ACK/SYN reject-with tcp-reset
DROP       all  --  anywhere             anywhere            state INVALID
ACCEPT     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere

Chain OUTPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
REJECT     tcp  --  anywhere             anywhere            tcp flags:!FIN,SYN,RST,ACK/SYN reject-with tcp-reset
DROP       all  --  anywhere             anywhere            state INVALID
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
Not an expert on this - can anyone have a look over this for me please?
User avatar
mikeshinn
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 4149
Joined: Thu Feb 07, 2008 7:49 pm
Location: Chantilly, VA

Re: nf_conntrack: table full, dropping packet

Unread post by mikeshinn »

What are you using your FORWARD rules for?
Post Reply