Bug #40688 mysql crash
Submitted: 13 Nov 2008 6:49 Modified: 9 Jan 2009 7:57
Reporter: Bartek okan Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:5.1.31 OS:Linux (centos 5)
Assigned to: CPU Architecture:Any
Tags: crash, MySQL, replication

[13 Nov 2008 6:49] Bartek okan
Description:
081113  7:44:41 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
key_buffer_size=1572864000
read_buffer_size=1048576
max_used_connections=81
max_threads=2000
threads_connected=39
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5652312 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x2aac34b9aa20
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x45e490e0 thread_stack 0x40000
/srv/mysql/libexec/mysqld(my_print_stacktrace+0x24) [0x868404]
/srv/mysql/libexec/mysqld(handle_segfault+0x322) [0x5ae182]
/lib64/libpthread.so.0 [0x39b200de80]
/lib64/libc.so.6(memmove+0x90) [0x39b18794c0]
/srv/mysql/libexec/mysqld(my_vsnprintf+0x353) [0x87d3f3]
/srv/mysql/libexec/mysqld(Log_to_file_event_handler::log_error(loglevel, char const*, __va_list_tag*)+0x21) [0x64afd1]
/srv/mysql/libexec/mysqld(sql_print_warning(char const*, ...)+0xa5) [0x64adc5]
/srv/mysql/libexec/mysqld(THD::binlog_query(THD::enum_binlog_query_type, char const*, unsigned long, bool, bool, THD::killed_state)+0x1c5) [0x59d955]
/srv/mysql/libexec/mysqld(mysql_update(THD*, TABLE_LIST*, List<Item>&, List<Item>&, Item*, unsigned int, st_order*, unsigned long long, enum_duplicates, bool)+0xb9d) [0x6439ed]
/srv/mysql/libexec/mysqld(mysql_execute_command(THD*)+0x4bd8) [0x5c1008]
/srv/mysql/libexec/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x228) [0x5c3f98]
/srv/mysql/libexec/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x3f1) [0x5c4391]
/srv/mysql/libexec/mysqld(do_command(THD*)+0xe4) [0x5c5504]
/srv/mysql/libexec/mysqld(handle_one_connection+0x5d7) [0x5b6137]
/lib64/libpthread.so.0 [0x39b2006307]
/lib64/libc.so.6(clone+0x6d) [0x39b18d1ded]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x14c43720 = UPDATE smf_sessions
                SET data = 'rand_code|s:32:\"d1091ebd7858fc7a3345fd150ecc9b97\";ban|a:5:{s:12:\"last_checked\";i:1226558572;s:9:\"ID_MEMBER\";s:4:\"4682\";s:2:\"ip\";s:12:\"83.1$
                WHERE session_id = '6ad312b63185f2ee7a01ed208deebae6'
                LIMIT 1
thd->thread_id=116762
thd->killed=NOT_KILLED

How to repeat:
run mysql 5.1.31 builded from source (bazaar)
[13 Nov 2008 6:54] MySQL Verification Team
Hi Bartek, please send output of:
show create table smf_sessions;
select @@global.binlog_format;

thanks,
[13 Nov 2008 7:10] Bartek okan
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table        | Create Table                                                                                                                                                                                                          |
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| smf_sessions | CREATE TABLE `smf_sessions` (
  `session_id` char(32) NOT NULL DEFAULT '',
  `last_update` int(10) unsigned NOT NULL,
  `data` mediumtext NOT NULL,
  PRIMARY KEY (`session_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

select @@global.binlog_format;
+------------------------+
| @@global.binlog_format |
+------------------------+
| STATEMENT              |
+------------------------+

hmm is it because of statement ? hmm.. should i switch to mixed ?
[13 Nov 2008 15:13] Bartek okan
shit stoped after switching to MIXED but yet old bug come back i dont remember its number:

      Last_SQL_Error: Could not execute Delete_rows event on table testing.mybb_sessions; Can't find record in 'mybb_sessions', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysqlbin.000005, end_log_pos 586265043

| mybb_sessions | CREATE TABLE `mybb_sessions` (
  `sid` varchar(32) NOT NULL DEFAULT '',
  `uid` int(10) unsigned NOT NULL DEFAULT '0',
  `ip` varchar(40) NOT NULL DEFAULT '',
  `time` bigint(30) NOT NULL DEFAULT '0',
  `location` varchar(150) NOT NULL DEFAULT '',
  `useragent` varchar(100) NOT NULL DEFAULT '',
  `anonymous` int(1) NOT NULL DEFAULT '0',
  `nopermission` int(1) NOT NULL DEFAULT '0',
  `location1` int(10) NOT NULL DEFAULT '0',
  `location2` int(10) NOT NULL DEFAULT '0',
  `loginattempts` tinyint(2) NOT NULL DEFAULT '1',
  `failedlogin` bigint(30) NOT NULL DEFAULT '0',
  PRIMARY KEY (`sid`),
  KEY `location1` (`location1`),
  KEY `location2` (`location2`),
  KEY `time` (`time`),
  KEY `uid` (`uid`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 |

bin=MIXED

replication between two 5.1.31 (bzr) (centos 5 64bit )

if you need any other information like just ask
[13 Nov 2008 20:20] Bartek okan
http://bugs.mysql.com/bug.php?id=40004 - still exist 

Could not execute Update_rows event on table test.e107_online; handler error HA_ERR_END_OF_FILE; the event's master log mysqlbin.000005, end_log_pos 993011008

and for about 10 hours replication goes to hell, just more ane more of those error shows up, and it is impossible to recover from it 

exist in 5.1.28(bin)/5.1.29(bin)/5.1.30(bzr)/5.1.31(bzr)

all daabases such as *online*, *session* getting crazy with MIXED replication
i can`t use STATEMENT because server instantly crush after each 5 minutes of runing... when sever is runned standalone (without replication), its ok...
when i try to setup master-to-master:

-HA_ERR_END_OF_FILE
-HA_ERR_KEY_NOT_FOUND

Those two error on tables like *online*, *session* which in structure are very close to example higher.. broke the replication to death :)
[17 Nov 2008 20:40] Bartek okan
are there any work around it ?
It crash under 5.0 GA too

its about month from first statemnt of this bug, you state that is fixed.. but isn`t ? is there any ETA for it, this is very serious cricital bug...
[9 Dec 2008 7:57] Valeriy Kravchuk
Nobody claimed that this bug (repeatable on both 5.1 and 5.0 with statement-based replication) is fixed. We can not even repeat it in our environment yet. Please, send your entire my.cnf, the results of SHOW GLOBAL STATUS and the results of:

free

Linux command.

As for bug #40004, indeed, we claimed it is fixed in 5.1.30 and newer versions. It is NOT related directly to this report (you have primary key and use statement-based replication), but if you have a repeatable test case for bug #40004 send it here and I'll reopen it.
[10 Jan 2009 0:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".