Ran YUM upgrade. site now down & plesk wont start. donation

[jayvin]

help, ran a 'yum upgrade' via ssh, which ive done many times before without issue (last 'yum upgrade' was about a week ago), im running on plesk 9.5.2 and it wont start (& site is down) yum reported it updated successfully. Psa wont restart.

I cant even connect via ssh. Please help, I need my vps. Willing to donate for help.

Fri Nov 19 06:25:23 2010 Operation start with the VPS(s) VEIDXXXXX service "psa" is started. Completed
Fri Nov 19 06:25:23 2010 Starting xinetd service... done Completed
Fri Nov 19 06:25:23 2010 Starting named service... done Completed
Fri Nov 19 06:25:23 2010 Starting mysqld service... done Completed
Fri Nov 19 06:25:23 2010 Starting postgresql service... not installed Completed
Fri Nov 19 06:25:23 2010 Starting psa-spamassassin service... not installed Completed
Fri Nov 19 06:25:25 2010 Plesk: Starting Mail Server... failed Completed
Fri Nov 19 06:25:25 2010 Starting mail handlers tmpfs storage Completed
Fri Nov 19 06:25:26 2010 Starting Plesk... failed Completed
Fri Nov 19 06:25:27 2010 Starting drwebd service... not installed Completed
Fri Nov 19 06:25:31 2010 Operation start with the VPS(s) VEIDXXXXX service "psa" is finished with errors: #703 Can not start/stop/restart service: service "psa" status is not "running" after command "start". Failed

Some info from the error logs

[Fri Nov 19 04:36:21 2010] [notice] Apache/2.2.3 (CentOS) configured -- resuming normal operations
pipelog: Unable to connect to Plesk Database: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
System error 2: No such file or directory
pipelog: Unable to connect to the mysql database

System error 25: Inappropriate ioctl for device
[Fri Nov 19 04:52:54 2010] [notice] caught SIGTERM, shutting down
[Fri Nov 19 04:53:11 2010] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Nov 19 04:53:11 2010] [warn] RSA server certificate CommonName (CN) `plesk' does NOT match server name!?
[Fri Nov 19 04:53:11 2010] [warn] RSA server certificate CommonName (CN) `plesk' does NOT match server name!?

[scott]

Is mysql running?

[jayvin]

Is mysql running?

No. I tried restarting the various services from within the virtuozzo panel, to no avail. (apologies im not great at these things)

here is the error log, from right after I ran the 'yum upgrade', unable to connect to mysql db:

Code: Select all

[Fri Nov 19 04:35:50 2010] [notice] caught SIGTERM, shutting down
[Fri Nov 19 04:35:50 2010] [notice] mod_fcgid: process /var/www/vhosts/domain1.com/httpdocs/test/fcgi/test.fcgi(17680) exit(shutting down), get stop signal 15
[Fri Nov 19 04:36:21 2010] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Nov 19 04:36:21 2010] [warn] RSA server certificate CommonName (CN) `plesk' does NOT match server name!?
[Fri Nov 19 04:36:21 2010] [warn] RSA server certificate CommonName (CN) `plesk' does NOT match server name!?
[Fri Nov 19 04:36:21 2010] [warn] Init: SSL server IP/port conflict: domain1.com:443 (/var/www/vhosts/domain1.com/conf/httpd.include:12) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)
[Fri Nov 19 04:36:21 2010] [warn] Init: SSL server IP/port conflict: domain2.com:443 (/var/www/vhosts/domain2.com/conf/httpd.include:12) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)
[Fri Nov 19 04:36:21 2010] [warn] Init: SSL server IP/port conflict: default-87-106-100-128:443 (/etc/httpd/conf.d/zz010_psa_httpd.conf:79) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)
[Fri Nov 19 04:36:21 2010] [warn] Init: You should not use name-based virtual hosts in conjunction with SSL!!
[Fri Nov 19 04:36:21 2010] [warn] module ssl_module is already loaded, skipping
[Fri Nov 19 04:36:21 2010] [notice] Digest: generating secret for digest authentication ...
[Fri Nov 19 04:36:21 2010] [notice] Digest: done
[Fri Nov 19 04:36:21 2010] [warn] RSA server certificate CommonName (CN) `plesk' does NOT match server name!?
[Fri Nov 19 04:36:21 2010] [warn] RSA server certificate CommonName (CN) `plesk' does NOT match server name!?
[Fri Nov 19 04:36:21 2010] [warn] Init: SSL server IP/port conflict: domain1.com:443 (/var/www/vhosts/domain1.com/conf/httpd.include:12) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)
[Fri Nov 19 04:36:21 2010] [warn] Init: SSL server IP/port conflict: domain2.com:443 (/var/www/vhosts/domain2.com/conf/httpd.include:12) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)
[Fri Nov 19 04:36:21 2010] [warn] Init: SSL server IP/port conflict: default-87-106-100-128:443 (/etc/httpd/conf.d/zz010_psa_httpd.conf:79) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)
[Fri Nov 19 04:36:21 2010] [warn] Init: You should not use name-based virtual hosts in conjunction with SSL!!
[Fri Nov 19 04:36:21 2010] [notice] Apache/2.2.3 (CentOS) configured -- resuming normal operations
pipelog: Unable to connect to Plesk Database: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
System error 2: No such file or directory
pipelog: Unable to connect to the mysql database

Your help is greatly appreciated.

I dont know if this will help but here is the info from /var/log/sw-cp-server/error_log

Code: Select all

2010-11-19 04:36:00: (log.c.135) server stopped 
2010-11-19 04:36:19: (log.c.75) server started 
2010-11-19 04:52:54: (log.c.135) server stopped 
2010-11-19 04:53:09: (log.c.75) server started 
2010-11-19 04:53:09: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:53:09: (log.c.75) server started 
2010-11-19 04:53:09: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:53:17: (log.c.75) server started 
2010-11-19 04:53:17: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:53:17: (log.c.75) server started 
2010-11-19 04:53:17: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:57:28: (log.c.75) server started 
2010-11-19 04:57:28: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:57:28: (log.c.75) server started 
2010-11-19 04:57:28: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:57:35: (log.c.75) server started 
2010-11-19 04:57:35: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:57:35: (log.c.75) server started 
2010-11-19 04:57:35: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:59:04: (log.c.75) server started 
2010-11-19 04:59:04: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 
2010-11-19 04:59:04: (log.c.75) server started 
2010-11-19 04:59:04: (network.c.300) can't bind to port: 127.0.0.1 10001 Cannot assign requested address 

[scott]

I'll bet you just need to do this:
http://www.atomicorp.com/wiki/index.php/Mysql

[jayvin]

I'll bet you just need to do this:
http://www.atomicorp.com/wiki/index.php/Mysql

isnt everything inc php and mysql upgraded when you run 'yum upgrade'?

I cannot login to ssh to run that command, Its not asking for username & keeps timing out.

I dont know if any of the error details above helps. Is there a way to roll back what was changed in the latest yum upgrade. I ran a yum upgrade a wk ago with no problems.

Really desperate to get this sorted.

Heres the mysql log file info:

Code: Select all

101119  4:36:02 [Note] /usr/libexec/mysqld: Normal shutdown

101119  4:36:02 [Note] Event Scheduler: Purging the queue. 0 events
101119  4:36:02  InnoDB: Starting shutdown...
101119  4:36:05  InnoDB: Shutdown completed; log sequence number 0 17965459
101119  4:36:05 [Note] /usr/libexec/mysqld: Shutdown complete

101119 04:36:05 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
101119 04:36:21 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
101119  4:36:21 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] '--log_slow_queries' is deprecated and will be removed in a future release. Please use ''--slow_query_log'/'--slow_query_log_file'' instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] option 'innodb-additional-mem-pool-size': signed value 512000 adjusted to 524288
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Warning] /usr/libexec/mysqld: Option '--set-variable' is deprecated. Use --variable-name=value instead.
101119  4:36:22 [Note] Plugin 'ndbcluster' is disabled.
101119  4:36:22  InnoDB: Started; log sequence number 0 17965459
101119  4:36:22 [Note] Event Scheduler: Loaded 0 events
101119  4:36:22 [Note] /usr/libexec/mysqld: ready for connections.

[scott]

Unfortunately there isn't a way to automate that completely. You'll need to get on the box somehow to complete the update

[jayvin]

Unfortunately there isn't a way to automate that completely. You'll need to get on the box somehow to complete the update

hrm any ideas how I do that? my normal ssh login via putty isnt working since the 'yum upgrade'.

I can get a sort of ssh access via virtuozzo repair mode but Im not sure if that works, as when i login via ssh and type i.e. 'yum upgrade' says its not found. This is the instructions viruozzo give but im not sure what to do next.

Really appreciate any help.

Code: Select all

The repair mode is needed when your VPS cannot be started. This mode means that a new VPS is temporarily created from scratch with the same network and other parameters as the broken VPS, and the root directory of the broken VPS is mounted as /repair into the newly-created VPS.

In the repair mode, you should connect to the new VPS via ssh using the network and login parameters of the broken VPS, go to the /repair directory, and perform one of the following actions:

    * Find the personal data that need to be saved and copy it to a safe location elsewhere. Do not copy your personal data directly into your new VPS, as it will be destroyed once you exit the repair mode.
    * Identify the source of the problem and manually correct it.

After closing your ssh session, press the Repair Mode link on the VZPP menu once again and press the Finish Repair button to exit the repair mode.

[jayvin]

Please somebody, im willing to donate for a solution.

[jayvin]

Please im willing to pay for assistance.

I cant login via normal SSH but if I put virtuozzo into repair mode I can login to the root of the server via SSH (virtuozzo puts data into /repair directory).

I'm self employed and need it back online

[BruceLee]

sorry I can't help you. But you can write to support@atomicorp.com for a quote.
Or you can ask breun if he can help you: http://atomicorp.com/forums/memberlist. ... file&u=278

[faris]

Remember that the Virtuozzo Power Panel has a built-in javascript-based SSH client. Have you tried that? It might let you in if normal ssh doesn't. You never know.

[mikeshinn]

I'm not 100% sure what the repair mode is doing, but if its mounting the actual filesystem of your VPS you could try to invoke it by chrooting the file system and trying to run "yum upgrade" then, so something like this:

chroot /whatever/the/repair/data/path/is

Then try running "yum upgrade".

[jayvin]

I'm not 100% sure what the repair mode is doing, but if its mounting the actual filesystem of your VPS you could try to invoke it by chrooting the file system and trying to run "yum upgrade" then, so something like this:

chroot /whatever/the/repair/data/path/is

Then try running "yum upgrade".

Ive tried, virtuozzo mounts the filesystem to /repair. got in there via virtuozzos repair ssh, tried 'yum upgrade', reported that yum wasnt available. So tried to download new yum (wget -q -O - http://www.atomicorp.com/installers/atomic.sh | sh) and run it, but didnt work. It said something about python issue. Not sure why yum wont run when virtuozzo is in repair mode anyway, its definitely installed so dont know why it wont run via the repair ssh.

I really need this site back up and i dont have money to pay atomic quotes. I can give $50 to someone to help me.

below this is the log of the yum upgrade that failed everything. i ran 'yum upgrade' it twice in succession and it updated stuff twice. also tried 'yum upgrade mysql' but said no packages are marked for update.

the mysql Client API version is 5.0.83 according to phpinfo file. Seems like mysql isnt starting for some reason.

Im a wreck and have been on this 24hrs solid.

Code: Select all

wget -q -O - http://www.atomicorp.com/installers/atomic | sh

Atomic Archive installer, version 1.2
Configuring the [atomic] yum archive for this system

Installing the Atomic GPG key:
OK
Downloading atomic-release-1.0-12.el5.art.noarch.rpm: OK



The Atomic Rocket Turtle archive has now been installed and configured for your system
The following channels are available:
  atomic          - [ACTIVATED] - contains the stable tree of ART packages
  atomic-testing  - [DISABLED]  - contains the testing tree of ART packages
  atomic-bleeding - [DISABLED]  - contains the development tree of ART packages

Code: Select all

yum upgrade
Loaded plugins: allowdowngrade, fastestmirror
Loading mirror speeds from cached hostfile
 * atomic: www6.atomicorp.com
addons                                                   |  951 B     00:00
atomic                                                   | 1.9 kB     00:00
base                                                     | 2.1 kB     00:00
extras                                                   | 2.1 kB     00:00
mod-pagespeed                                            |  951 B     00:00
updates                                                  | 1.9 kB     00:00
Setting up Upgrade Process
Resolving Dependencies
--> Running transaction check
---> Package e2fsprogs.x86_64 0:1.39-23.el5_5.1 set to be updated
---> Package e2fsprogs-libs.i386 0:1.39-23.el5_5.1 set to be updated
---> Package e2fsprogs-libs.x86_64 0:1.39-23.el5_5.1 set to be updated
---> Package freetype.x86_64 0:2.2.1-28.el5_5.1 set to be updated
---> Package initscripts.x86_64 0:8.45.30-3.el5.centos set to be updated
---> Package python.x86_64 0:2.4.3-27.el5_5.3 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package             Arch        Version                     Repository    Size
================================================================================
Updating:
 e2fsprogs           x86_64      1.39-23.el5_5.1             updates      995 k
 e2fsprogs-libs      i386        1.39-23.el5_5.1             updates      118 k
 e2fsprogs-libs      x86_64      1.39-23.el5_5.1             updates      118 k
 freetype            x86_64      2.2.1-28.el5_5.1            updates      311 k
 initscripts         x86_64      8.45.30-3.el5.centos        updates      1.6 M
 python              x86_64      2.4.3-27.el5_5.3            updates      6.0 M

Transaction Summary
================================================================================
Install       0 Package(s)
Upgrade       6 Package(s)

Total download size: 9.0 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): e2fsprogs-libs-1.39-23.el5_5.1.x86_64.rpm         | 118 kB     00:00
(2/6): e2fsprogs-libs-1.39-23.el5_5.1.i386.rpm           | 118 kB     00:00
(3/6): freetype-2.2.1-28.el5_5.1.x86_64.rpm              | 311 kB     00:00
(4/6): e2fsprogs-1.39-23.el5_5.1.x86_64.rpm              | 995 kB     00:00
(5/6): initscripts-8.45.30-3.el5.centos.x86_64.rpm       | 1.6 MB     00:00
(6/6): python-2.4.3-27.el5_5.3.x86_64.rpm                | 6.0 MB     00:01
--------------------------------------------------------------------------------
Total                                           5.4 MB/s | 9.0 MB     00:01
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating       : e2fsprogs-libs                                          1/12
  Updating       : freetype                                                2/12
  Updating       : python                                                  3/12
  Updating       : e2fsprogs-libs                                          4/12
  Updating       : e2fsprogs                                               5/12
  Updating       : initscripts                                             6/12
  Cleanup        : python                                                  7/12
  Cleanup        : initscripts                                             8/12
  Cleanup        : freetype                                                9/12
  Cleanup        : e2fsprogs-libs                                         10/12
  Cleanup        : e2fsprogs-libs                                         11/12
  Cleanup        : e2fsprogs                                              12/12

Updated:
  e2fsprogs.x86_64 0:1.39-23.el5_5.1
  e2fsprogs-libs.i386 0:1.39-23.el5_5.1
  e2fsprogs-libs.x86_64 0:1.39-23.el5_5.1
  freetype.x86_64 0:2.2.1-28.el5_5.1
  initscripts.x86_64 0:8.45.30-3.el5.centos
  python.x86_64 0:2.4.3-27.el5_5.3

Complete!

Code: Select all

# yum upgrade
Loaded plugins: allowdowngrade, fastestmirror
Loading mirror speeds from cached hostfile
 * atomic: www6.atomicorp.com
addons                                                   |  951 B     00:00
atomic                                                   | 1.9 kB     00:00
atomic/primary_db                                        | 470 kB     00:01
base                                                     | 2.1 kB     00:00
extras                                                   | 2.1 kB     00:00
mod-pagespeed                                            |  951 B     00:00
updates                                                  | 1.9 kB     00:00
Setting up Upgrade Process
Resolving Dependencies
--> Running transaction check
--> Processing Dependency: psa-proftpd-start for package: psa
---> Package psa-proftpd.x86_64 0:1.3.3c-3.el5.art set to be updated
--> Running transaction check
---> Package psa-proftpd-xinetd.x86_64 0:1.3.3c-3.el5.art set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                 Arch        Version                  Repository   Size
================================================================================
Updating:
 psa-proftpd             x86_64      1.3.3c-3.el5.art         atomic      2.0 M
Installing for dependencies:
 psa-proftpd-xinetd      x86_64      1.3.3c-3.el5.art         atomic       12 k

Transaction Summary
================================================================================
Install       1 Package(s)
Upgrade       1 Package(s)

Total download size: 2.0 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): psa-proftpd-xinetd-1.3.3c-3.el5.art.x86_64.rpm    |  12 kB     00:00
(2/2): psa-proftpd-1.3.3c-3.el5.art.x86_64.rpm           | 2.0 MB     00:03
--------------------------------------------------------------------------------
Total                                           505 kB/s | 2.0 MB     00:04
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating       : psa-proftpd                                              1/3
  Installing     : psa-proftpd-xinetd                                       2/3
  Cleanup        : psa-proftpd                                              3/3

Dependency Installed:
  psa-proftpd-xinetd.x86_64 0:1.3.3c-3.el5.art

Updated:
  psa-proftpd.x86_64 0:1.3.3c-3.el5.art

Complete!

[BruceLee]

What do you get when you try to chroot the repair dir?
Only when chrooted correctly you can run yum. It doesn't make sense to do it just in the repair vps mode.

Well, the most important thing for you is to get you ssh running first.
It might be possible that your vps network is not running due to the initsrcipt update and therefore you can't connect to it.
Can you ping the server IP from external?

If not, try this and start the repair mode and the go to the dir /repair where the root of your VPS in mounted.
In that dir you will go further to open that file with vi
/etc/sysconfig/network-scripts/network-fuctions
Now edit the file like this:

Add this two lines to the top of the is_available() function !!!

Code: Select all

LC_ALL= LANG= ip -o link | grep -q $1

[ "$?" = "1"] || return 0 

Restart the VPS. Try to connect or ping. Step by step.

[faris]

In Repair Mode, essentially what happens is that a band new VPS is created (based on the same OS from the same original template used to create your broken VPS), with the filesystem of the original VPS being mounted in /repair.

Whatever happened to your VPS may be very easily fixed by your VPS provider. Have you contacted them? For example they may have a backup of it, or they may be able to investigate the cause and fix it very easily. It definitely sounds like something got broken in terms of networking (or firewall) when you updated via yum (though I don't know what!).

BruceLee's suggestion looks very interesting!

Another thing for you to consider is to compare the potentially broken networking scripts (and maybe firewall) in /repair/etc/sysconfig/(whatever) with those in /etc/sysconfig/(whatever) of the working Repair-Mode VPS.