mysql.sock missing in action.

[Galactic Zero]

I just upgraded mysql last night and now it won't start. says can't connect to port, so after some research I've found that the mysql.sock file and its symlink are missing, have tried to recreate the file yet every time I do it disappears. Not sure what to do or where to look.

have the line: socket=/var/lib/mysql/mysql.sock
have the user: user=mysql
permissions on /var/lib/mysql are mysql:root 0755
permissions on /var/lib/mysql/* mysql:mysql 0660

Tail of the mysqld.log:

100619 12:58:51 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
100619 13:18:11 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
100619 13:18:11 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
100619 13:18:11 [Note] Plugin 'ndbcluster' is disabled.
/usr/libexec/mysqld: Can't find file: './mysql/plugin.frm' (errno: 13)
100619 13:18:11 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
100619 13:18:11 InnoDB: Started; log sequence number 1 3221251837
100619 13:18:11 [ERROR] /usr/libexec/mysqld: unknown option '--skip-bdb'
100619 13:18:11 [ERROR] Aborting

100619 13:18:11 InnoDB: Starting shutdown...
100619 13:18:16 InnoDB: Shutdown completed; log sequence number 1 3221251837
100619 13:18:16 [Note] /usr/libexec/mysqld: Shutdown complete

[faris]

I think the problem is more to do with your my.conf.

/usr/libexec/mysqld: unknown option '--skip-bdb'

Remove that from my.conf

run mysql_upgrade, assuming it runs after this change.

Faris.

[Galactic Zero]

nope... not it...

100619 15:37:37 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
100619 17:35:07 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
100619 17:35:07 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
100619 17:35:07 [Note] Plugin 'ndbcluster' is disabled.
/usr/libexec/mysqld: Can't find file: './mysql/plugin.frm' (errno: 13)
100619 17:35:07 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
100619 17:35:07 InnoDB: Started; log sequence number 1 3221251837
100619 17:35:07 [ERROR] /usr/libexec/mysqld: Can't find file: './mysql/host.frm' (errno: 13)
100619 17:35:07 [ERROR] Fatal error: Can't open and lock privilege tables: Can't find file: './mysql/host.frm' (errno: 13)
100619 17:35:07 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

[faris]

OK, I think it is still about the config in my.cnf, specifically plugins.

Have you tried running mysql_upgrade, in case that sorts it out somehow (I don't think it will, as it is sort of chicken and egg isn't it - I don't think mysql_upgrade will run if mysqld isn't running, unless it is very smart and knows how to handle situations like this)?

If not, maybe try commenting out any plugins mentioned in my.cnf, then running mysql_upgrade, then uncommenting them again.

I'm almost certain there's nothing wrong with the PID or anything like that, and it is just a config issue.

Please backup your databases first. This is "dangerous" stuff (on my scale).

Faris.

[Galactic Zero]

I'll try that, had The Planet looking at this also, and one message they see is permissions on host.frm. File exists and all taht jazz.

Here is what they had to say:
Basically, from what was found, as a higher level technician had to assist me with this, the core issue is that MySQL is trying to open the host.frm, but its giving permission denied. From what it looks like, it seems to be trying to open one of these files, apparently, which was created today:

-rw-r----- 1 root root 9416 Jun 19 12:09 /var/lib/mysql.old/mysql/host.frm
-rw-r----- 1 root root 8692 Jun 19 12:13 /var/lib/mysql.old/sitebuilder3/host.frm
-rw-r----- 1 root root 8628 Jun 19 12:13 /var/lib/mysql.old/sitebuilder3/plan_host.frm


It will not open it due to "permission denied" errors, so you would need to makes sure that the permissions and ownership are correct(the system is starting MySQL with the mysql user and those files are owned by root, not mysql, so that could be the cause of the issue), so that MySQL can read it and then start properly.

At first we tried to check how MySQL was starting, and from a strace, the following was found, in the output, was this error:

[pid 21825] open("./mysql/host.frm", O_RDONLY|O_LARGEFILE) = -1 EACCES (Permission denied)

[Galactic Zero]

Here is the latest tail after I tried to start mysqld with service mysqld start.

100620 12:14:10 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
100620 12:14:10 [Note] Plugin 'ndbcluster' is disabled.
/usr/libexec/mysqld: Can't find file: './mysql/plugin.frm' (errno: 13)
100620 12:14:10 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
100620 12:14:10 InnoDB: Started; log sequence number 1 3221251837
100620 12:14:10 [ERROR] /usr/libexec/mysqld: Can't find file: './mysql/host.frm' (errno: 13)
100620 12:14:10 [ERROR] Fatal error: Can't open and lock privilege tables: Can't find file: './mysql/host.frm' (errno: 13)
100620 12:14:10 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

[Galactic Zero]

As I'm thinking about it, here is the output of my.cnf, anyone see anything wrong? This was working until I upgraded.

[root@gz lib]# more /etc/my.cnf
[mysqld]
# set-variable=local-infile=0
# variable-local-infile=0
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
# wait_timeout=
# max_user_connections=500
# max_connections=500
query_cache_type=1
query_cache_limit=1M
query_cache_size=64M
# log_slow_queries=/var/log/mysqld.slow.log
long_query_time=5
thread_cache_size=6
thread_concurrency = 4
# skip-bdb
# join_buffer_size=1M
table_cache=96
tmp_table_size=64M

[mysql.server]
user=mysql
basedir=/var/lib
innodb_buffer_pool_size >= 8M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[BruceLee]

maybe check you permissions of /var/lib/mysql

[Galactic Zero]

they should be mysql:root and 0755 for the dir and mysql:mysql 0655 for the contents, yes?

[faris]

You have definitely made progress, I think, but obviously there are still issues.

From what I've seen for searching for that Can't find file: './mysql/plugin.frm' error, it does seem to indicate that something isn't where mysql thinks it should be, or does not have the right permissions.

I'm afraid I'm lost at this point and don't know what else to suggest Sorry.

Faris.

[BruceLee]

I got
mysql:mysql 0755 for the /var/lib/mysql dir
mysql:mysql 0700 for the subdirs in /var/lib/mysql
mysql:mysql 0777 for mysql.sock
mysql:mysql 0660 for the files in subdirs

I still use mysql 5.0.90 NOT 5.1.x

[Galactic Zero]

ok, my fatal error is it can't find host.frm.

100621 19:29:00 [ERROR] Fatal error: Can't open and lock privilege tables: Can't find file: './mysq
l/host.frm' (errno: 13)

-rw-rw---- 1 mysql mysql 9416 May 30 17:25 host.frm

located in ./var/lib/mysql/mysql

[Galactic Zero]

Alright, everything I've researched says it permissions, but I'm stumped as to which files are not setup correctly.

Anyone have some fresh eyes to look at this please?

[BruceLee]

I bet it's because of the /var/lib/mysql/mysql folder.
I dont know why this happened but I think the files should be in /var/lib/mysql

[Kalimari]

mysql:mysql 0755 for the /var/lib/mysql dir
mysql:mysql 0700 for the subdirs in /var/lib/mysql
mysql:mysql 0777 for mysql.sock
mysql:mysql 0660 for the files in subdirs

Using 5.1.x and these are the same permissions on our system.

Did you get a /etc/my.conf.rpmnew during install? Try using that to see if it allows mysql to start up. If not here's a copy:

Code: Select all

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

# To allow mysqld to connect to a MySQL Cluster management daemon, uncomment
# these lines and adjust the connectstring as needed.
#ndbcluster
#ndb-connectstring="nodeid=4;host=localhost:1186"

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[ndbd]
# If you are running a MySQL Cluster storage daemon (ndbd) on this machine,
# adjust its connection to the management daemon here.
# Note: ndbd init script requires this to include nodeid!
connect-string="nodeid=2;host=localhost:1186"

[ndb_mgm]
# connection string for MySQL Cluster management tool
connect-string="host=localhost:1186"

Good luck