Description:
My current setup is Server version: 5.6.10-enterprise-commercial-advanced MySQL Enterprise Server - Advanced Edition (Commercial). I have data in my database.
I installed the Galera package [galera-3, mysql-wsrep-5.6] to support a cluster.
Installed:
galera-3.x86_64 0:25.3.16-2.el6 mysql-wsrep-5.6.x86_64 0:5.6.30-25.15.el6 mysql-wsrep-client-5.6.x86_64 0:5.6.30-25.15.el6
Dependency Installed:
lsof.x86_64 0:4.82-5.el6 mysql-wsrep-server-5.6.x86_64 0:5.6.30-25.15.el6 rsync.x86_64 0:3.0.6-12.el6
Replaced:
mysql.x86_64 0:5.1.73-7.el6
------
After this installation, the new version of mysql became active
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.30 MySQL Community Server (GPL), wsrep_25.15
Next, I started mysql using my pre-existing my.cnf and it did not start due to some issues with the audit plugin. I disabled audit plugin and then mysql started properly. I will tackle the audit plugin issue separately.
Next, I tried to invoke mysql_upgrade and that crashed the server. Here is the log from mysql server
Version: '5.6.30' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL), wsrep_25.15
=====================================
13:29:33 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=500
thread_count=1
connection_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 206856 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x5041180
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f5860164d38 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x35)[0x90cd55]
/usr/sbin/mysqld(handle_fatal_signal+0x494)[0x6859b4]
/lib64/libpthread.so.0(+0xf7e0)[0x7f585fd947e0]
/usr/sbin/mysqld(thd_get_ha_data+0x9)[0x6c7ec9]
/usr/sbin/mysqld(_Z20thd_binlog_trx_resetP3THD+0x31)[0x8b99e1]
/usr/sbin/mysqld(_Z25wsrep_cleanup_transactionP3THD+0x1d)[0x7bb97d]
/usr/sbin/mysqld(_Z17wsrep_post_commitP3THDb+0xa7)[0x7bba77]
/usr/sbin/mysqld(_Z17trans_commit_stmtP3THD+0x90)[0x7a3db0]
/usr/sbin/mysqld[0x8488a3]
/usr/sbin/mysqld(_ZN19Sql_cmd_check_table7executeEP3THD+0xc2)[0x849522]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x2414)[0x706f34]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x36f)[0x70b07f]
/usr/sbin/mysqld[0x70b284]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1826)[0x70d416]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x1eb)[0x70ec7b]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x167)[0x6d6817]
/usr/sbin/mysqld(handle_one_connection+0x47)[0x6d69a7]
/usr/sbin/mysqld(pfs_spawn_thread+0x12a)[0xb31d6a]
/lib64/libpthread.so.0(+0x7aa1)[0x7f585fd8caa1]
/lib64/libc.so.6(clone+0x6d)[0x7f585e4a5aad]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f57800050f0): is an invalid pointer
Connection ID (thread ID): 4
Status: NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
---
Here is the screen output from mysql_upgrade
mysql_upgrade --password
Enter password:
Looking for 'mysql' as: mysql
Looking for 'mysqlcheck' as: mysqlcheck
Running 'mysqlcheck with default connection arguments
Warning: Using a password on the command line interface can be insecure.
Running 'mysqlcheck with default connection arguments
Warning: Using a password on the command line interface can be insecure.
mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... FOR UPGRADE'
mysql.columns_priv OK
mysql.db OK
mysql.event OK
mysql.func OK
mysql.general_log OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
FATAL ERROR: Error during call to mysql_check for upgrading the tables names on mysql db
[root@localhost plugin]# mysql_upgrade --password cloupia
Enter password:
Looking for 'mysql' as: mysql
Looking for 'mysqlcheck' as: mysqlcheck
Running 'mysqlcheck with default connection arguments
Warning: Using a password on the command line interface can be insecure.
Running 'mysqlcheck with default connection arguments
Warning: Using a password on the command line interface can be insecure.
mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... FOR UPGRADE'
mysql.columns_priv OK
mysql.db OK
mysql.event OK
mysql.func OK
mysql.general_log OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
FATAL ERROR: Error during call to mysql_check for upgrading the tables names on mysql db
[root@localhost plugin]# cat /etc/redhat-release
CentOS release 6.7 (Final)
---
How to repeat:
I ran the above test, running mysql_upgrade multiple times and it still crashed mysqld every time.
I'm not sure if mysql_upgrade is needed to be run but I'm following the documentation from Galera and it recommends running mysql_upgrade. It is quite likely, there is no upgrade needed because the mysql version is almost the same (5.6.13 -> 5.6.30) but I would expect that mysqld woudln't crash.
If needed, I can provide the details of the galera upgrade log that shows all the rpms installed.
