Apache segmentation fault

General Discussion of atomic repo and development projects.

Ask for help here with anything else not covered by other forums.
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re:

Unread post by faris »

aus-city wrote: You put the script into your rc.local so its run on bootup.
Faris's dumbo question of the day - does this need to be run as a background task when started via rc.local ?

i.e.
/path/to/scriptname &
or
perl /path/to/scriptname &
or
what?

I presume not, and that anything in rc.local is just run as an independant process so it can just be /path/to/scriptname

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Apache segmentation fault

Unread post by scott »

It doesnt matter really, rc.local runs last so everything else has started already at that point
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

Interesting. Thanks.

Incidentally I've reverted to the full ruleset over the weekend. So far memory usage is up (as expected), semaphores are up (as expected) but things are stable and no segfaults.

I'll keep an eye on things on Monday, when the load will get to normal levels, and see what happens.
I've got that script running on one VPS to see if it makes any difference if there's a problem.

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

I just tested and it kinda does need an & at the end otherwise you end up with s99 and rc 3 showing up in the processlist permanently.

I guess it doesn't matter though.

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

Well, with the full ruleset active, no segfaults. No problems.

So far!

Please cross your fingers that all is now well again.

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

Errr...

I thought I'd test the restart script.....so I added an approrpriate error to the error_log

Unfortunately it didn't really work!
Here's what happened. Looks like the semaphore cleaning didn't do its thing or something?

Code: Select all

[Tue Mar 03 16:24:31 2009] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Tue Mar 03 16:24:31 2009] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Tue Mar 03 16:24:31 2009] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Tue Mar 03 16:24:31 2009] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Tue Mar 03 16:24:31 2009] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Tue Mar 03 16:24:31 2009] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Tue Mar 03 16:24:31 2009] [alert] Child 32051 returned a Fatal error... Apache is exiting!
[Tue Mar 03 16:24:31 2009] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Tue Mar 03 16:24:38 2009] [emerg] (22)Invalid argument: couldn't release the accept mutex
And unfortunately apache wasn't restarted. Obviously psmon would have taken care of it if I hadn't done so manually.

Indeed, I'm wondering if psmon got in the way - if psmon restarted apache before the the semaphores were cleaned or something?

EDIT:

Yeah, I think it was psmon. I got this at the same time

Code: Select all

Command executed: /sbin/service httpd restart Exit value: 1 Signal number: 0 Dumped core?: 0

Stopping httpd: [FAILED] Starting httpd: [FAILED] 
Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

Ah. Found the REAL problem!

Code: Select all

system "/etc/init.d/apache stop";
(and similarly for start)

Should be httpd not apache

Nevertheless, stopping psmon seems liek a good idea.

Here's my revised code:

Code: Select all

sub act {
    # stop psmon and give it a chance to stop
    system "/etc/init.d/psmon stop";
    sleep 2;
    # stop apache
    #system "killall -9 httpd";
    system "/etc/init.d/httpd stop";
    # clear semaphores for apache
    system "ipcs -s | grep $apacheuser | awk ' { print $2 } ' | xargs ipcrm sem";
    #start apache clean
    system "/etc/init.d/httpd start";
    #give it some time to start, just in case
    sleep 10;
    #restart psmon now
    system "/etc/init.d/psmon start";
    # email admin
    system "echo 'Service httpd restarted due to processing error at " . scalar(localtime) . "' | mail $adminemail -s $subjectemail";

    # don't restart again sooner than 5 minutes, in case of mad apache
    sleep 300;
}

EDIT ... ok ok. So what I had was old code. But I just downloaded it. I thought it had been fixed and re-uploaded?

Anyway, it works.

But I note that I'm getting

Code: Select all

Use of uninitialized value in concatenation (.) or string at restart_apache.pl line 68, <GEN0> line 1.
usage: ipcrm [ [-q msqid] [-m shmid] [-s semid]
          [-Q msgkey] [-M shmkey] [-S semkey] ... ]
deprecated usage: ipcrm {shm | msg | sem} id ...
Changing sem to -s doesn't work for some reason though. I'm missing something.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
hostingguy
Forum Regular
Forum Regular
Posts: 661
Joined: Mon Oct 29, 2007 6:51 pm

Re: Apache segmentation fault

Unread post by hostingguy »

Try this

Code: Select all

system  "for i in `ipcs -s | grep $apacheuser | awk '{print $2}'` ; do ipcrm -s $i; done";
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

Lovely. Thanks.

That makes more sense too.

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

Well, even that didn't work. Perl was complaining about potential syntax errors.

There's nothing wrong with the syntax though..

I've had to put the apache stop/start etc and semaphore cleaning in a bash script, and call that from the perl script.
It is quite happy with that :-)

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
sebas
Forum User
Forum User
Posts: 85
Joined: Thu Feb 12, 2009 8:53 pm
Location: Mexico
Contact:

Re: Apache segmentation fault

Unread post by sebas »

Ok I managed to have dumps, I also installed -debuginfo, but when I do use gdb I get:

gdb core.22310
GNU gdb Fedora (6.8-27.el5)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...
"/disk1/apache2-gdb-dump/core.22310": not in executable format: File format not recognized

It might be important to say that I installed httpd-debuinfo once I had the dump.

Thanks for your help.
CentOS release 6.5 (Final)
Plesk psa-11.5.30-cos6.build115130819.13
ASL 3.2.18-37
Linux 2.6.32-358.11.1.el6.x86_64
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Apache segmentation fault

Unread post by scott »

Youd do:

gdb /usr/sbin/httpd <path to core>
sebas
Forum User
Forum User
Posts: 85
Joined: Thu Feb 12, 2009 8:53 pm
Location: Mexico
Contact:

Re: Apache segmentation fault

Unread post by sebas »

Ok here is what I get:
This GDB was configured as "x86_64-redhat-linux-gnu"...

warning: Can't read pathname for load map: Input/output error.
Loaded symbols for /usr/sbin/httpd
Reading symbols from /lib64/libm.so.6...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpcre.so.0...done.
Loaded symbols for /lib64/libpcre.so.0
Reading symbols from /lib64/libselinux.so.1...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /usr/lib64/libaprutil-1.so.0...done.
Loaded symbols for /usr/lib64/libaprutil-1.so.0
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /usr/lib64/libldap-2.3.so.0...done.
Loaded symbols for /usr/lib64/libldap-2.3.so.0
Reading symbols from /usr/lib64/liblber-2.3.so.0...done.
Loaded symbols for /usr/lib64/liblber-2.3.so.0
Reading symbols from /lib64/libdb-4.3.so...done.
Loaded symbols for /lib64/libdb-4.3.so
Reading symbols from /lib64/libexpat.so.0...done.
Loaded symbols for /lib64/libexpat.so.0
Reading symbols from /usr/lib64/libapr-1.so.0...done.
Loaded symbols for /usr/lib64/libapr-1.so.0
Reading symbols from /lib64/libpthread.so.0...done.
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/libsepol.so.1...done.
Loaded symbols for /lib64/libsepol.so.1
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /usr/lib64/libpq.so.4...done.
Loaded symbols for /usr/lib64/libpq.so.4
Reading symbols from /usr/lib64/libsqlite3.so.0...done.
Loaded symbols for /usr/lib64/libsqlite3.so.0
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /usr/lib64/libsasl2.so.2...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /lib64/libssl.so.6...done.
Loaded symbols for /lib64/libssl.so.6
Reading symbols from /lib64/libcrypto.so.6...done.
Loaded symbols for /lib64/libcrypto.so.6
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libnsl.so.1...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /usr/lib64/libkrb5support.so.0...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /lib64/librt.so.1...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /usr/lib64/httpd/modules/mod_auth_basic.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_auth_basic.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_auth_basic.so
Reading symbols from /usr/lib64/httpd/modules/mod_auth_digest.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_auth_digest.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_auth_digest.so
Reading symbols from /usr/lib64/httpd/modules/mod_authn_file.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authn_file.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authn_file.so
Reading symbols from /usr/lib64/httpd/modules/mod_authn_alias.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authn_alias.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authn_alias.so
Reading symbols from /usr/lib64/httpd/modules/mod_authn_anon.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authn_anon.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authn_anon.so
Reading symbols from /usr/lib64/httpd/modules/mod_authn_dbm.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authn_dbm.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authn_dbm.so
Reading symbols from /usr/lib64/httpd/modules/mod_authn_default.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authn_default.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authn_default.so
Reading symbols from /usr/lib64/httpd/modules/mod_authz_host.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authz_host.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authz_host.so
Reading symbols from /usr/lib64/httpd/modules/mod_authz_user.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authz_user.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authz_user.so
Reading symbols from /usr/lib64/httpd/modules/mod_authz_owner.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authz_owner.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authz_owner.so
Reading symbols from /usr/lib64/httpd/modules/mod_authz_groupfile.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authz_groupfile.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authz_groupfile.so
Reading symbols from /usr/lib64/httpd/modules/mod_authz_dbm.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authz_dbm.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authz_dbm.so
Reading symbols from /usr/lib64/httpd/modules/mod_authz_default.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authz_default.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authz_default.so
Reading symbols from /usr/lib64/httpd/modules/mod_ldap.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_ldap.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_ldap.so
Reading symbols from /usr/lib64/httpd/modules/mod_authnz_ldap.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_authnz_ldap.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_authnz_ldap.so
Reading symbols from /usr/lib64/httpd/modules/mod_include.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_include.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_include.so
Reading symbols from /usr/lib64/httpd/modules/mod_log_config.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_log_config.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_log_config.so
Reading symbols from /usr/lib64/httpd/modules/mod_logio.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_logio.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_logio.so
Reading symbols from /usr/lib64/httpd/modules/mod_env.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_env.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_env.so
Reading symbols from /usr/lib64/httpd/modules/mod_ext_filter.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_ext_filter.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_ext_filter.so
Reading symbols from /usr/lib64/httpd/modules/mod_mime_magic.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_mime_magic.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_mime_magic.so
Reading symbols from /usr/lib64/httpd/modules/mod_expires.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_expires.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_expires.so
Reading symbols from /usr/lib64/httpd/modules/mod_deflate.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_deflate.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_deflate.so
Reading symbols from /usr/lib64/httpd/modules/mod_headers.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_headers.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_headers.so
Reading symbols from /usr/lib64/httpd/modules/mod_usertrack.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_usertrack.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_usertrack.so
Reading symbols from /usr/lib64/httpd/modules/mod_setenvif.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_setenvif.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_setenvif.so
Reading symbols from /usr/lib64/httpd/modules/mod_mime.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_mime.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_mime.so
Reading symbols from /usr/lib64/httpd/modules/mod_dav.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_dav.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_dav.so
Reading symbols from /usr/lib64/httpd/modules/mod_status.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_status.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_status.so
Reading symbols from /usr/lib64/httpd/modules/mod_autoindex.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_autoindex.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_autoindex.so
Reading symbols from /usr/lib64/httpd/modules/mod_info.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_info.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_info.so
Reading symbols from /usr/lib64/httpd/modules/mod_dav_fs.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_dav_fs.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_dav_fs.so
Reading symbols from /usr/lib64/httpd/modules/mod_vhost_alias.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_vhost_alias.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_vhost_alias.so
Reading symbols from /usr/lib64/httpd/modules/mod_negotiation.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_negotiation.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_negotiation.so
Reading symbols from /usr/lib64/httpd/modules/mod_dir.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_dir.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_dir.so
Reading symbols from /usr/lib64/httpd/modules/mod_actions.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_actions.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_actions.so
Reading symbols from /usr/lib64/httpd/modules/mod_speling.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_speling.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_speling.so
Reading symbols from /usr/lib64/httpd/modules/mod_userdir.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_userdir.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_userdir.so
Reading symbols from /usr/lib64/httpd/modules/mod_alias.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_alias.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_alias.so
Reading symbols from /usr/lib64/httpd/modules/mod_rewrite.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_rewrite.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_rewrite.so
Reading symbols from /usr/lib64/httpd/modules/mod_proxy.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_proxy.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy.so
Reading symbols from /usr/lib64/httpd/modules/mod_proxy_balancer.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_proxy_balancer.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_balancer.so
Reading symbols from /usr/lib64/httpd/modules/mod_proxy_ftp.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_proxy_ftp.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_ftp.so
Reading symbols from /usr/lib64/httpd/modules/mod_proxy_http.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_proxy_http.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_http.so
Reading symbols from /usr/lib64/httpd/modules/mod_proxy_connect.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_proxy_connect.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_connect.so
Reading symbols from /usr/lib64/httpd/modules/mod_cache.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_cache.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_cache.so
Reading symbols from /usr/lib64/httpd/modules/mod_suexec.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_suexec.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_suexec.so
Reading symbols from /usr/lib64/httpd/modules/mod_disk_cache.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_disk_cache.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_disk_cache.so
Reading symbols from /usr/lib64/httpd/modules/mod_file_cache.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_file_cache.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_file_cache.so
Reading symbols from /usr/lib64/httpd/modules/mod_cgi.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_cgi.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_cgi.so
Reading symbols from /usr/lib64/httpd/modules/mod_version.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_version.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_version.so
Reading symbols from /usr/lib64/httpd/modules/mod_jk.so...done.
Loaded symbols for /usr/lib64/httpd/modules/mod_jk.so
Reading symbols from /usr/lib64/httpd/modules/mod_security2.so...done.
Loaded symbols for /etc/httpd/modules/mod_security2.so
Reading symbols from /usr/lib64/libxml2.so.2...done.
Loaded symbols for /usr/lib64/libxml2.so.2
Reading symbols from /usr/lib64/httpd/modules/mod_unique_id.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_unique_id.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_unique_id.so
Reading symbols from /usr/lib64/httpd/modules/mod_bw.so...done.
Loaded symbols for /usr/lib64/httpd/modules/mod_bw.so
Reading symbols from /usr/lib64/httpd/modules/mod_fcgid.so...done.
Loaded symbols for /usr/lib64/httpd/modules/mod_fcgid.so
Reading symbols from /usr/lib64/httpd/modules/mod_evasive20.so...done.
Loaded symbols for /etc/httpd/modules/mod_evasive20.so
Reading symbols from /usr/lib64/httpd/modules/mod_perl.so...done.
Loaded symbols for /etc/httpd/modules/mod_perl.so
Reading symbols from /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so...done.
Loaded symbols for /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
Reading symbols from /lib64/libutil.so.1...done.
Loaded symbols for /lib64/libutil.so.1
Reading symbols from /usr/lib64/httpd/modules/libphp5.so...done.
Loaded symbols for /etc/httpd/modules/libphp5.so
Reading symbols from /usr/lib64/libgmp.so.3...done.
Loaded symbols for /usr/lib64/libgmp.so.3
Reading symbols from /usr/lib64/libcurl.so.3...done.
Loaded symbols for /usr/lib64/libcurl.so.3
Reading symbols from /usr/lib64/libbz2.so.1...done.
Loaded symbols for /usr/lib64/libbz2.so.1
Reading symbols from /usr/lib64/libidn.so.11...done.
Loaded symbols for /usr/lib64/libidn.so.11
Reading symbols from /usr/lib64/httpd/modules/mod_proxy_ajp.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_proxy_ajp.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_proxy_ajp.so
Reading symbols from /usr/lib64/httpd/modules/mod_python.so...done.
Loaded symbols for /etc/httpd/modules/mod_python.so
Reading symbols from /usr/lib64/libpython2.4.so.1.0...done.
Loaded symbols for /usr/lib64/libpython2.4.so.1.0
Reading symbols from /usr/lib64/httpd/modules/mod_ssl.so...Reading symbols from /usr/lib/debug/usr/lib64/httpd/modules/mod_ssl.so.debug...done.
done.
Loaded symbols for /etc/httpd/modules/mod_ssl.so
Reading symbols from /usr/lib64/libdistcache.so.1...done.
Loaded symbols for /usr/lib64/libdistcache.so.1
Reading symbols from /usr/lib64/libnal.so.1...done.
Loaded symbols for /usr/lib64/libnal.so.1
Reading symbols from /usr/lib64/php/modules/apc.so...done.
Loaded symbols for /usr/lib64/php/modules/apc.so
Reading symbols from /usr/lib64/php/modules/bcmath.so...done.
Loaded symbols for /usr/lib64/php/modules/bcmath.so
Reading symbols from /usr/lib64/php/modules/dba.so...done.
Loaded symbols for /usr/lib64/php/modules/dba.so
Reading symbols from /usr/lib64/php/modules/dbase.so...done.
Loaded symbols for /usr/lib64/php/modules/dbase.so
Reading symbols from /usr/lib64/php/modules/dom.so...done.
Loaded symbols for /usr/lib64/php/modules/dom.so
Reading symbols from /usr/lib64/php/modules/gd.so...done.
Loaded symbols for /usr/lib64/php/modules/gd.so
Reading symbols from /usr/lib64/libt1.so.5...done.
Loaded symbols for /usr/lib64/libt1.so.5
Reading symbols from /usr/lib64/libfreetype.so.6...done.
Loaded symbols for /usr/lib64/libfreetype.so.6
Reading symbols from /usr/lib64/libX11.so.6...done.
Loaded symbols for /usr/lib64/libX11.so.6
Reading symbols from /usr/lib64/libXpm.so.4...done.
Loaded symbols for /usr/lib64/libXpm.so.4
Reading symbols from /usr/lib64/libpng12.so.0...done.
Loaded symbols for /usr/lib64/libpng12.so.0
Reading symbols from /usr/lib64/libjpeg.so.62...done.
Loaded symbols for /usr/lib64/libjpeg.so.62
Reading symbols from /usr/lib64/libXau.so.6...done.
Loaded symbols for /usr/lib64/libXau.so.6
Reading symbols from /usr/lib64/libXdmcp.so.6...done.
Loaded symbols for /usr/lib64/libXdmcp.so.6
Reading symbols from /usr/lib64/php/modules/imap.so...done.
Loaded symbols for /usr/lib64/php/modules/imap.so
Reading symbols from /usr/lib64/libc-client.so.1...done.
Loaded symbols for /usr/lib64/libc-client.so.1
Reading symbols from /lib64/libpam.so.0...done.
Loaded symbols for /lib64/libpam.so.0
Reading symbols from /lib64/libaudit.so.0...done.
Loaded symbols for /lib64/libaudit.so.0
Reading symbols from /usr/lib64/php/modules/json.so...done.
Loaded symbols for /usr/lib64/php/modules/json.so
Reading symbols from /usr/lib64/php/modules/ldap.so...done.
Loaded symbols for /usr/lib64/php/modules/ldap.so
Reading symbols from /usr/lib64/php/modules/mbstring.so...done.
Loaded symbols for /usr/lib64/php/modules/mbstring.so
Reading symbols from /usr/lib64/php/modules/mcrypt.so...done.
Loaded symbols for /usr/lib64/php/modules/mcrypt.so
Reading symbols from /usr/lib64/libmcrypt.so.4...done.
Loaded symbols for /usr/lib64/libmcrypt.so.4
Reading symbols from /usr/lib64/php/modules/mhash.so...done.
Loaded symbols for /usr/lib64/php/modules/mhash.so
Reading symbols from /usr/lib64/libmhash.so.2...done.
Loaded symbols for /usr/lib64/libmhash.so.2
Reading symbols from /usr/lib64/php/modules/mysql.so...done.
Loaded symbols for /usr/lib64/php/modules/mysql.so
Reading symbols from /usr/lib64/mysql/libmysqlclient.so.15...done.
Loaded symbols for /usr/lib64/mysql/libmysqlclient.so.15
Reading symbols from /usr/lib64/php/modules/mysqli.so...done.
Loaded symbols for /usr/lib64/php/modules/mysqli.so
Reading symbols from /usr/lib64/php/modules/ncurses.so...done.
Loaded symbols for /usr/lib64/php/modules/ncurses.so
Reading symbols from /usr/lib64/libpanel.so.5...done.
Loaded symbols for /usr/lib64/libpanel.so.5
Reading symbols from /usr/lib64/libncurses.so.5...done.
Loaded symbols for /usr/lib64/libncurses.so.5
Reading symbols from /usr/lib64/php/modules/odbc.so...done.
Loaded symbols for /usr/lib64/php/modules/odbc.so
Reading symbols from /usr/lib64/libodbc.so.1...done.
Loaded symbols for /usr/lib64/libodbc.so.1
Reading symbols from /usr/lib64/libodbcpsql.so.2...done.
Loaded symbols for /usr/lib64/libodbcpsql.so.2
Reading symbols from /usr/lib64/php/modules/pdo.so...done.
Loaded symbols for /usr/lib64/php/modules/pdo.so
Reading symbols from /usr/lib64/php/modules/pdo_mysql.so...done.
Loaded symbols for /usr/lib64/php/modules/pdo_mysql.so
Reading symbols from /usr/lib64/php/modules/pdo_odbc.so...done.
Loaded symbols for /usr/lib64/php/modules/pdo_odbc.so
Reading symbols from /usr/lib64/php/modules/pdo_pgsql.so...done.
Loaded symbols for /usr/lib64/php/modules/pdo_pgsql.so
Reading symbols from /usr/lib64/php/modules/pdo_sqlite.so...done.
Loaded symbols for /usr/lib64/php/modules/pdo_sqlite.so
Reading symbols from /usr/lib64/php/modules/pgsql.so...done.
Loaded symbols for /usr/lib64/php/modules/pgsql.so
Reading symbols from /usr/lib64/php/modules/pspell.so...done.
Loaded symbols for /usr/lib64/php/modules/pspell.so
Reading symbols from /usr/lib64/libaspell.so.15...done.
Loaded symbols for /usr/lib64/libaspell.so.15
Reading symbols from /usr/lib64/libpspell.so.15...done.
Loaded symbols for /usr/lib64/libpspell.so.15
Reading symbols from /usr/lib64/libstdc++.so.6...done.
Loaded symbols for /usr/lib64/libstdc++.so.6
Reading symbols from /lib64/libgcc_s.so.1...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /usr/lib64/php/modules/snmp.so...done.
Loaded symbols for /usr/lib64/php/modules/snmp.so
Reading symbols from /usr/lib64/libnetsnmp.so.10...done.
Loaded symbols for /usr/lib64/libnetsnmp.so.10
Reading symbols from /lib64/libwrap.so.0...done.
Loaded symbols for /lib64/libwrap.so.0
Reading symbols from /usr/lib64/php/modules/soap.so...done.
Loaded symbols for /usr/lib64/php/modules/soap.so
Reading symbols from /usr/lib64/php/modules/sqlite.so...done.
Loaded symbols for /usr/lib64/php/modules/sqlite.so
Reading symbols from /usr/lib64/php/modules/suhosin.so...done.
Loaded symbols for /usr/lib64/php/modules/suhosin.so
Reading symbols from /usr/lib64/php/modules/tidy.so...done.
Loaded symbols for /usr/lib64/php/modules/tidy.so
Reading symbols from /usr/lib64/libtidy-0.99.so.0...done.
Loaded symbols for /usr/lib64/libtidy-0.99.so.0
Reading symbols from /usr/lib64/php/modules/xdebug.so...done.
Loaded symbols for /usr/lib64/php/modules/xdebug.so
Reading symbols from /usr/lib64/php/modules/xmlreader.so...done.
Loaded symbols for /usr/lib64/php/modules/xmlreader.so
Reading symbols from /usr/lib64/php/modules/xmlrpc.so...done.
Loaded symbols for /usr/lib64/php/modules/xmlrpc.so
Reading symbols from /usr/lib64/php/modules/xmlwriter.so...done.
Loaded symbols for /usr/lib64/php/modules/xmlwriter.so
Reading symbols from /usr/lib64/php/modules/xsl.so...done.
Loaded symbols for /usr/lib64/php/modules/xsl.so
Reading symbols from /usr/lib64/libexslt.so.0...done.
Loaded symbols for /usr/lib64/libexslt.so.0
Reading symbols from /usr/lib64/libxslt.so.1...done.
Loaded symbols for /usr/lib64/libxslt.so.1
Reading symbols from /usr/lib64/libgcrypt.so.11...done.
Loaded symbols for /usr/lib64/libgcrypt.so.11
Reading symbols from /usr/lib64/libgpg-error.so.0...done.
Loaded symbols for /usr/lib64/libgpg-error.so.0
Reading symbols from /usr/lib64/php/modules/zip.so...done.
Loaded symbols for /usr/lib64/php/modules/zip.so
Reading symbols from /lib64/libnss_dns.so.2...done.
Loaded symbols for /lib64/libnss_dns.so.2
Reading symbols from /usr/lib64/gconv/ISO8859-1.so...done.
Loaded symbols for /usr/lib64/gconv/ISO8859-1.so
Core was generated by `/usr/sbin/httpd'.
Program terminated with signal 11, Segmentation fault.
[New process 11507]
#0 0x000069c3b6254006 in do_bind_function () from /etc/httpd/modules/libphp5.so


I have lots of core dumps, many of which give the same output.

Other programs causing sementation fault are:
Core was generated by `/usr/libexec/paxtest/anonmap'.
Core was generated by `/usr/libexec/paxtest/execbss'.
Core was generated by `/usr/libexec/paxtest/execdata'.
Core was generated by `/usr/libexec/paxtest/execheap'.
Core was generated by `/usr/libexec/paxtest/execstack'.
Core was generated by `/usr/libexec/paxtest/mprotanon'.
Core was generated by `/usr/libexec/paxtest/mprotbss'.
Core was generated by `/usr/libexec/paxtest/mprotdata'.
Core was generated by `/usr/libexec/paxtest/mprotheap'.
Core was generated by `/usr/libexec/paxtest/mprotshbss'.
Core was generated by `/usr/libexec/paxtest/mprotshdata'.
Core was generated by `/usr/libexec/paxtest/mprotstack'.
Core was generated by `/usr/libexec/paxtest/shlibbss'.
Core was generated by `/usr/libexec/paxtest/shlibdata'.
CentOS release 6.5 (Final)
Plesk psa-11.5.30-cos6.build115130819.13
ASL 3.2.18-37
Linux 2.6.32-358.11.1.el6.x86_64
scott
Atomicorp Staff - Site Admin
Atomicorp Staff - Site Admin
Posts: 8355
Joined: Wed Dec 31, 1969 8:00 pm
Location: earth
Contact:

Re: Apache segmentation fault

Unread post by scott »

Yup, this has nothing to do with mod_security or ASL:

Program terminated with signal 11, Segmentation fault.
[New process 11507]
#0 0x000069c3b6254006 in do_bind_function () from /etc/httpd/modules/libphp5.so

PHP died for some reason
faris
Long Time Forum Regular
Long Time Forum Regular
Posts: 2321
Joined: Thu Dec 09, 2004 11:19 am

Re: Apache segmentation fault

Unread post by faris »

Just to keep people updated....

I got a couple of segfaults after a rule update a few days ago. No rule processing failures.

aus-city --- have you tried disabling the domain-blacklist.txt checks? That 100% sorted the problem for me previously. I'm currently running with them enabled, however. The thing is...they never seem to get fired. So they are much less important than anything else *for our sites*.

Faris.
--------------------------------
<advert>
If you want to rent a UK-based VPS that comes with friendly advice and support from a fellow ART fan, please get in touch.
</advert>
Post Reply