Bug #9277 4.1.11bk crashes assertion failure; log_open() false?
Submitted: 18 Mar 2005 12:56 Modified: 6 Apr 2005 14:08
Reporter: Jan Lindström Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S2 (Serious)
Version:4.1.11-bk OS:Linux (Linux)
Assigned to: Heikki Tuuri CPU Architecture:Any

[18 Mar 2005 12:56] Jan Lindström
Description:
I used ./BUILD/compile-pentium-debug --with-debug=full for build and after
some time of ibtests mysqld crashes on assertion.

jan@hundin:~/mysql-4.1/sql> ./mysqld
050318 14:04:30 [Warning] You have enabled the binary log, but you haven't setserver-id to a non-zero value: we force server id to 1; updates will be loggedto the binary log, but connections from slaves will not be accepted.
InnoDB: !!!!!!!!!!!!!! UNIV_DEBUG switched on !!!!!!!!!!!!!!!
InnoDB: The first specified data file /home/jan/sqldata/ibdata1 did not exist:
InnoDB: a new database to be created!
050318 14:04:30  InnoDB: Setting file /home/jan/sqldata/ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
050318 14:04:31  InnoDB: Log file /home/jan/sqldata/ib_logfile0 did not exist:new to be created
InnoDB: Setting log file /home/jan/sqldata/ib_logfile0 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
050318 14:04:50  InnoDB: Log file /home/jan/sqldata/ib_logfile1 did not exist:new to be created
InnoDB: Setting log file /home/jan/sqldata/ib_logfile1 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
050318 14:05:18  InnoDB: Started; log sequence number 0 0
./mysqld: ready for connections.
Version: '4.1.11-debug-log'  socket: '/home/jan/bugsocket'  port: 3306  Sourcedistribution
050318 14:05:58InnoDB: Assertion failure in thread 245776 in file log0log.c line 1280
InnoDB: Failing assertion: loop_count < 5
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=36700160
read_buffer_size=33550336
max_used_connections=12
max_connections=100
threads_connected=12
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2394732 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x8c679e8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6b3ee0a8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8169949
0x4006896c
0x831bd84
0x82f7c5f
0x8211040
0x8204aba
0x81da821
0x8204c3e
0x8185c98
0x81872a3
0x817fe8f
0x817f714
0x817eade
0x40062f60
0x401f5327
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
InnoDB: Thread 114696 stopped in file sync0sync.c line 325
thd->query at 0x8cfba90 = commit
thd->thread_id=22
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
InnoDB: Thread 278546 stopped in file ha_innodb.cc line 522
InnoDB: Thread 147466 stopped in file ../include/rem0rec.ic line 252
InnoDB: Thread 213006 stopped in file ../include/rem0rec.ic line 591
InnoDB: Thread 180236 stopped in file ../include/mach0data.ic line 116
InnoDB: Thread 327701 stopped in file ha_innodb.cc line 522
InnoDB: Thread 196621 stopped in file ha_innodb.cc line 522
InnoDB: Thread 294931 stopped in file sync0sync.c line 325
InnoDB: Thread 229391 stopped in file ../include/mach0data.ic line 34
InnoDB: Thread 163851 stopped in file ../include/ut0byte.ic line 331
InnoDB: Thread 262161 stopped in file log0log.c line 1280

How to repeat:
Run normal ibtests while and you should get this assertion failure. Note that
I did not get allways this assertion failure when I  used gdb. Ibtests I did use were:

perl ibtest3 >out3 &
perl ibtest3a >out3a &
perl ibtest5 >out5 &
perl ibtest5a >out5a &
perl ibtest7 >out7 &
perl ibtest7a >out7a &
perl ibtest8 >out8 &
perl ibtest8a >out8a &
perl ibtest9 >out 9 &
perl ibtest9a >out9a &
perl ibtest12 >out12 &
perl ibtest12a >out12a &
perl ibtest13 >out13 &

Suggested fix:
N/A.
[18 Mar 2005 13:02] Jan Lindström
Here is stack trace from gdb (note that lsn is most likely corrupted):

050318 14:15:30InnoDB: Assertion failure in thread 262161 in file log0log.c line 1280
InnoDB: Failing assertion: loop_count < 5
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 262161 (LWP 30019)]
0x0831bd84 in log_write_up_to (lsn={high = 1177507224, low = 5}, wait=92,
    flush_to_disk=1) at log0log.c:1280
1280            ut_ad(loop_count < 5);
Current language:  auto; currently c
(gdb) where
#0  0x0831bd84 in log_write_up_to (lsn={high = 1177507224, low = 5}, wait=92,
    flush_to_disk=1) at log0log.c:1280
#1  0x082f7c5f in trx_commit_complete_for_mysql (trx=0x462f5468)
    at trx0trx.c:1553
#2  0x08211040 in innobase_commit_complete(void*) (trx_handle=0x462f5468)
    at ha_innodb.cc:1346
#3  0x08204aba in ha_commit_complete(THD*) (thd=0x8c55a10) at handler.cc:453
#4  0x081da821 in MYSQL_LOG::write(THD*, st_io_cache*, bool) (this=0x85d5dc0,
    thd=0x8c55a10, cache=0x8c56a88, commit_or_rollback=true) at log.cc:1707
#5  0x08204c3e in ha_commit_trans(THD*, st_thd_trans*) (thd=0x8c55a10,
    trans=0x8c56b4c) at handler.cc:516
#6  0x08185c98 in mysql_execute_command(THD*) (thd=0x8c55a10)
    at sql_parse.cc:3579
#7  0x081872a3 in mysql_parse(THD*, char*, unsigned) (thd=0x8c55a10,
    inBuf=0x8c85b38 "commit", length=6) at sql_parse.cc:4187
#8  0x0817fe8f in dispatch_command(enum_server_command, THD*, char*, unsigned)
    (command=COM_QUERY, thd=0x8c55a10, packet=0x8c9d009 "commit",
    packet_length=7) at sql_parse.cc:1505
#9  0x0817f714 in do_command(THD*) (thd=0x8c55a10) at sql_parse.cc:1318
#10 0x0817eade in handle_one_connection (arg=0x8c55a10) at sql_parse.cc:1050
#11 0x40062f60 in pthread_start_thread () from /lib/i686/libpthread.so.0
#12 0x400630fe in pthread_start_thread_event () from /lib/i686/libpthread.so.0
#13 0x401f5327 in clone () from /lib/i686/libc.so.6
(gdb) up
#1  0x082f7c5f in trx_commit_complete_for_mysql (trx=0x462f5468)
    at trx0trx.c:1553
1553                            log_write_up_to(lsn, LOG_WAIT_ONE_GROUP, TRUE);
(gdb) p lsn
$1 = {high = 1177507224, low = 5}
(gdb)
[18 Mar 2005 14:04] Jan Lindström
This bug is not inside the InnoDB. This is because MySQL does not call
innobase_commit() or innobase_report_binlog_offset_and_commit() before
calling innobase_commit_complete(). trx->commit_lsn field is updated in trx->commit_off_kernel() which is called from innobase_commit() and
innobase_commit() is also called inside of innobase_report_binlog_offset_and_commit(). That is reason why lsn in corrupted.
For example in the below trx->commit_lsn = {high = 0, low = 62448}.

050318 15:04:33InnoDB: Assertion failure in thread 147466 in file trx0trx.c line 1543
InnoDB: Failing assertion: trx->commit_off_kernel == TRUE
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 147466 (LWP 6501)]
0x082f7eb3 in trx_commit_complete_for_mysql (trx=0x462dcc68) at trx0trx.c:1543
1543            ut_a(trx->commit_off_kernel == TRUE);
Current language:  auto; currently c
(gdb) where
#0  0x082f7eb3 in trx_commit_complete_for_mysql (trx=0x462dcc68)
    at trx0trx.c:1543
#1  0x08211044 in innobase_commit_complete(void*) (trx_handle=0x462dcc68)
    at ha_innodb.cc:1346
#2  0x08204aba in ha_commit_complete(THD*) (thd=0x6b3399f0) at handler.cc:453
#3  0x081da1e6 in MYSQL_LOG::write(Log_event*) (this=0x85d54c0,
    event_info=0x6b2c750c) at log.cc:1546
#4  0x0821ee67 in mysql_create_table(THD*, char const*, char const*, st_ha_create_information*, List<create_field>&, List<Key>&, bool, unsigned) (
    thd=0x6b3399f0, db=0x6b32abb0 "test", table_name=0x6b31e5f0 "alex1",
    create_info=0x6b339db4, fields=@0x6b339cf0, keys=@0x6b339ce4,
    tmp_table=false, select_field_count=0) at sql_table.cc:1422
#5  0x081827f3 in mysql_execute_command(THD*) (thd=0x6b3399f0)
    at sql_parse.cc:2481
#6  0x081872a3 in mysql_parse(THD*, char*, unsigned) (thd=0x6b3399f0,
    inBuf=0x6b31e2b0 "CREATE TABLE alex1 (tiny1 tinyint, tiny2 tinyint, small1smallint, symbole varchar(20) NOT NULL default '', symbole2 char(2),symbole3 varchar(20),symbole4 varchar(20),symbole5 varchar(20),symbole6 var"...,
    length=790) at sql_parse.cc:4187
#7  0x0817fe8f in dispatch_command(enum_server_command, THD*, char*, unsigned)
    (command=COM_QUERY, thd=0x6b3399f0,
    packet=0x6b342dd9 "CREATE TABLE alex1 (tiny1 tinyint, tiny2 tinyint, small1 smallint, symbole varchar(20) NOT NULL default '', symbole2 char(2),symbole3 v---Type <return> to continue, or q <return> to quit---
archar(20),symbole4 varchar(20),symbole5 varchar(20),symbole6 var"...,
    packet_length=791) at sql_parse.cc:1505
#8  0x0817f714 in do_command(THD*) (thd=0x6b3399f0) at sql_parse.cc:1318
#9  0x0817eade in handle_one_connection (arg=0x8c2a448) at sql_parse.cc:1050
#10 0x40062f60 in pthread_start_thread () from /lib/i686/libpthread.so.0
#11 0x400630fe in pthread_start_thread_event () from /lib/i686/libpthread.so.0
#12 0x401f5327 in clone () from /lib/i686/libc.so.6
[18 Mar 2005 15:29] Heikki Tuuri
Hi!

Below is Jan's my.cnf and his binlog. The bug could be explained if log_open() is false in log.cc where it should be true.

But I am not able to repeat this with compile-pentium-debug or

compile-pentium-debug --debug-full

Jan, please rebuild mysqld and tests again.

Regards,

Heikki

heikki@hundin:~> ./mysqlbinlog /home/jan/sqldata/binlog.000001
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
# at 4
#050318 15:16:18 server id 1  log_pos 4         Start: binlog v 3, server v 4.1.
11-debug-log created 050318 15:16:18 at startup
# at 79
#050318 15:17:02 server id 1  log_pos 79        Query   thread_id=4     exec_tim
e=0     error_code=0
SET TIMESTAMP=1111151822;
create database `test`;
# at 136
#050318 15:17:05 server id 1  log_pos 136       Query   thread_id=6     exec_tim
e=4     error_code=0
use test;
SET TIMESTAMP=1111151825;
CREATE TABLE alex3 (tiny1 tinyint, tiny2 tinyint, small1 smallint, symbole varch
ar(20) NOT NULL default '', symbole2 char(2),symbole3 varchar(20),symbole4 varch
ar(20),symbole5 varchar(20),symbole6 varchar(20),symbole7 varchar(20),date datet
ime default '0000-00-00 00:00:00',first double(17,5) default '0.00000',high doub
le(17,5) NOT NULL default '0.00000', low double(17,5) NOT NULL default '0.00000'
,last double(17,5) NOT NULL default '0.00000',volume int(11) NOT NULL default '0
', UNIQUE KEY  (symbole, date),KEY idx_symbole (symbole),KEY idx_symbole2 (symbo
le2),KEY idx_symbole3 (symbole3, symbole4),KEY idx_symbole4 (symbole4, symbole5,
 symbole6),KEY idx_symbole5 (symbole7, symbole), KEY idx_date (date), KEY idx_ti
ny (tiny2, tiny1), KEY idx_small (small1, tiny1)) TYPE=InnoDB # jhghjghjhj;
# at 965
#050318 15:17:09 server id 1  log_pos 965       Query   thread_id=19    exec_tim
e=1     error_code=0
SET TIMESTAMP=1111151829;
create table `Aktivität`(a int) type = innodb;
# at 1045
#050318 15:17:10 server id 1  log_pos 1045      Query   thread_id=18    exec_tim
e=3     error_code=0
SET TIMESTAMP=1111151830;
CREATE TABLE alex1 (tiny1 tinyint, tiny2 tinyint, small1 smallint, symbole varch
ar(20) NOT NULL default '', symbole2 char(2),symbole3 varchar(20),symbole4 varch
ar(20),symbole5 varchar(20),symbole6 varchar(20),symbole7 varchar(20),date datet
ime default '0000-00-00 00:00:00',first double(17,5) default '0.00000',high doub
le(17,5) NOT NULL default '0.00000', low double(17,5) NOT NULL default '0.00000'
,last double(17,5) NOT NULL default '0.00000',volume int(11) unsigned NOT NULL d
efault '0', UNIQUE KEY  (symbole, date),KEY idx_symbole (symbole),KEY idx_symbol
e2 (symbole2),KEY idx_symbole3 (symbole3, symbole4),KEY idx_symbole4 (symbole4,
symbole5, symbole6),KEY idx_symbole5 (symbole7, symbole), KEY idx_date (date), K
EY idx_tiny (tiny2, tiny1), KEY idx_small (small1, tiny1)) TYPE=InnoDB;

heikki@hundin:/home/jan> cat .my.cnf
# Example mysql config file.
# Copy this file to c:\my.cnf to set global options
#
# One can use all long options that the program supports.
# Run the program with --help to get a list of available options

# This will be passed to all mysql clients
[client]
#default-character-set=utf8
socket=/home/jan/bugsocket

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# The MySQL server
[mysqld]
#default-character-set=utf8
language=/home/jan/varchar/mysql-5.0/sql/share/english
default-table-type=innodb
log-bin=binlog
set-variable = max_binlog_size=100M
socket=/home/jan/bugsocket
basedir=/home/jan/
datadir=/home/jan/sqldata
set-variable = table_cache=256
set-variable = sort_buffer_size=32M
set-variable = read_buffer_size=32M
set-variable = read_rnd_buffer_size=32M
set-variable = thread_cache=8
set-variable = thread_concurrency=2
set-variable = key_buffer=35M
innodb_file_per_table
#innodb_locks_unsafe_for_binlog
innodb_data_home_dir = /home/jan/sqldata
innodb_data_file_path = ibdata1:10M:autoextend
set-variable = innodb_mirrored_log_groups=1
innodb_log_group_home_dir = /home/jan/sqldata
innodb_log_arch_dir = /home/jan/sqldata
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
set-variable = innodb_buffer_pool_size=500M
set-variable = innodb_additional_mem_pool_size=100M
set-variable = innodb_file_io_threads=4
# Set .._log_file_size to 25 % of buffer pool size
set-variable = innodb_log_file_size=125M
set-variable = innodb_log_files_in_group=2
set-variable = innodb_log_buffer_size=10M
#innodb_flush_log_at_trx_commit=2
set-variable = innodb_lock_wait_timeout=50
# InnoDB does a full purge and an insert buffer merge
# before a shutdown. May be slow hope to solve problem
#set-variable = innodb_fast_shutdown=0
#set-variable = lower_case_table_names=1
#innodb_lock_wait_timeout=5
innodb_thread_concurrency=100

[mysqldump]
quick
set-variable = max_allowed_packet=16M

[mysql]
#default-character-set=utf8
no-auto-rehash
socket=/home/jan/bugsocket
#port=3307

[mysqladmin]
socket=/home/jan/bugsocket
#port=3307

[isamchk]
set-variable= key=16M

[client_fltk]
[18 Mar 2005 15:32] Heikki Tuuri
Hi!

Now I got a crash, but a different one!

Maybe something wrong in the --debug-full build?

--Heikki

heikki@hundin:~/mysql-4.1/sql> ./mysqld
InnoDB: !!!!!!!!!!!!!! UNIV_DEBUG switched on !!!!!!!!!!!!!!!
050318 16:46:17  InnoDB: Started; log sequence number 0 12376350
050318 16:46:17 [Warning] mysql.user table is not updated to new password format
; Disabling new password usage until mysql_fix_privilege_tables is run
./mysqld: ready for connections.
Version: '4.1.11-debug-log'  socket: '/home/heikki/bugsocket'  port: 3307  Sourc
e distribution
mysqld: ha_innodb.cc:2787: virtual int ha_innobase::update_row(const mysql_byte*
, mysql_byte*): Assertion `prebuilt->template_type == 0' failed.
mysqld got signal 6;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=36700160
read_buffer_size=131072
max_used_connections=5
max_connections=100
threads_connected=5
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 253436
 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x8c68308
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6e460e08, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8169949
0x4006896c
0x40142b71
0x4006600b
0x40142904
0x40143e8c
0x4013be84
0x821337a
0x81c4ed6
0x81c41b8
0x81836a2
0x81872a3
0x817fe8f
0x817f714
0x817eade
0x40062f60
0x401f5327
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow
instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8c69f90 = REPLACE INTO alex1 VALUES(78, 58, 874,'aa16818','bb','
c16818','d9678t','e35954f','g222','h874',date_format('2001-04-03 12:54:22','%Y-%
m-%d %H:%i'),7.31,7.31,7.31,7.31,200)
thd->thread_id=5
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
heikki@hundin:~/mysql-4.1/sql>
[18 Mar 2005 15:38] Heikki Tuuri
Hi!

Again a different crash! Maybe --with-debug-full causes memory corruption?

Regards,

Heikki

heikki@hundin:~/mysql-4.1/sql> ./mysqld
InnoDB: !!!!!!!!!!!!!! UNIV_DEBUG switched on !!!!!!!!!!!!!!!
InnoDB: The first specified data file /home/heikki/data/ibdata1 did not exist:
InnoDB: a new database to be created!
050318 16:48:50  InnoDB: Setting file /home/heikki/data/ibdata1 size to 40 MB
InnoDB: Database physically writes the file full: wait...
050318 16:48:52  InnoDB: Log file /home/heikki/data/ib_logfile0 did not exist: n
ew to be created
InnoDB: Setting log file /home/heikki/data/ib_logfile0 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
050318 16:49:05  InnoDB: Log file /home/heikki/data/ib_logfile1 did not exist: n
ew to be created
InnoDB: Setting log file /home/heikki/data/ib_logfile1 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
050318 16:49:33  InnoDB: Started; log sequence number 0 0
050318 16:49:33 [Warning] mysql.user table is not updated to new password format
; Disabling new password usage until mysql_fix_privilege_tables is run
./mysqld: ready for connections.
Version: '4.1.11-debug-log'  socket: '/home/heikki/bugsocket'  port: 3307  Sourc
e distribution
050318 16:50:28InnoDB: Assertion failure in thread 311313 in file lock0lock.c li
ne 4903
InnoDB: Failing assertion: mode != 4 || lock_table_has(thr_get_trx(thr), index->
table, 2)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=36700160
read_buffer_size=131072
max_used_connections=19
InnoDB: Thread 196621 stopped in file os0sync.c line 501
InnoDB: Thread 213006 stopped in file sync0sync.c line 325
InnoDB: Thread 573465 stopped in file ../include/rem0rec.ic line 116
max_connections=100
threads_connected=19
InnoDB: Thread 704535 stopped in file ../include/hash0hash.ic line 21
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 253436
 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

InnoDB: Thread 147466 stopped in file ../include/page0page.ic line 389
thd=0x8ccb668
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6e52f9a8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8169949
0x4006896c
0x83530f2
0x82ad89d
0x82b48c1
Stack trace seems successful - bottom reached
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow
instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8ccd228 = select * from blobt3 where c like 'khd%'
thd->thread_id=11
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
safe_mutex: Trying to destroy a mutex that was locked at log.cc, line 1243 at lo
g.cc, line 159
[18 Mar 2005 17:40] Jan Lindström
I did a new build with ./BUILD/compile-pentium-debug --with-debug=full and still
I get the same assert failure:

jan@hundin:~/mysql-4.1/sql> ./mysqld
050318 18:48:13 [Warning] You have enabled the binary log, but you haven't setserver-id to a non-zero value: we force server id to 1; updates will be loggedto the binary log, but connections from slaves will not be accepted.
InnoDB: !!!!!!!!!!!!!! UNIV_DEBUG switched on !!!!!!!!!!!!!!!
InnoDB: The first specified data file /home/jan/sqldata/ibdata1 did not exist:
InnoDB: a new database to be created!
050318 18:48:13  InnoDB: Setting file /home/jan/sqldata/ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
050318 18:48:14  InnoDB: Log file /home/jan/sqldata/ib_logfile0 did not exist:new to be created
InnoDB: Setting log file /home/jan/sqldata/ib_logfile0 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
050318 18:48:28  InnoDB: Log file /home/jan/sqldata/ib_logfile1 did not exist:new to be created
InnoDB: Setting log file /home/jan/sqldata/ib_logfile1 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
050318 18:48:57  InnoDB: Started; log sequence number 0 0
./mysqld: ready for connections.
Version: '4.1.11-debug-log'  socket: '/home/jan/bugsocket'  port: 3306  Sourcedistribution
050318 18:49:15InnoDB: Assertion failure in thread 294931 in file log0log.c line 1280
InnoDB: Failing assertion: loop_count < 5
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=36700160
read_buffer_size=33550336
max_used_connections=10
max_connections=100
threads_connected=10
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2394732 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x8cab420
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6b4810a8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8169949
0x4006896c
0x831bfb0
0x82f7e84
0x8211044
0x8204aba
0x81da821
0x8204c3e
0x8185c98
0x81872a3
0x817fe8f
0x817f714
0x817eade
0x40062f60
0x401f5327
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8cee4c0 = commit
thd->thread_id=24
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
InnoDB: Thread 278546 stopped in file ../include/hash0hash.ic line 21
InnoDB: Thread 147466 stopped in file ../include/mem0mem.ic line 61
InnoDB: Thread 229391 stopped in file ha_innodb.cc line 522
[19 Mar 2005 8:50] Jan Lindström
Today I created a new clone and compiled with ./BUILD/compile-pentium-debug --with-debug=full in hundin.mysql.fi and still it crashes on assertion failure:

jan@hundin:~> cd new/mysql-4.1/sql
jan@hundin:~/new/mysql-4.1/sql> ./mysqld
050319 10:02:18 [Warning] You have enabled the binary log, but you haven't setserver-id to a non-zero value: we force server id to 1; updates will be loggedto the binary log, but connections from slaves will not be accepted.
InnoDB: !!!!!!!!!!!!!! UNIV_DEBUG switched on !!!!!!!!!!!!!!!
InnoDB: The first specified data file /home/jan/sqldata/ibdata1 did not exist:
InnoDB: a new database to be created!
050319 10:02:18  InnoDB: Setting file /home/jan/sqldata/ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
050319 10:02:19  InnoDB: Log file /home/jan/sqldata/ib_logfile0 did not exist:new to be created
InnoDB: Setting log file /home/jan/sqldata/ib_logfile0 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
050319 10:02:34  InnoDB: Log file /home/jan/sqldata/ib_logfile1 did not exist:new to be created
InnoDB: Setting log file /home/jan/sqldata/ib_logfile1 size to 125 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
050319 10:03:10  InnoDB: Started; log sequence number 0 0
./mysqld: ready for connections.
Version: '4.1.11-debug-log'  socket: '/home/jan/bugsocket'  port: 3306  Sourcedistribution
050319 10:03:31InnoDB: Assertion failure in thread 294931 in file log0log.c line 1280
InnoDB: Failing assertion: loop_count < 5
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
InnoDB: Thread 114696 stopped in file sync0sync.c line 677
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=36700160
read_buffer_size=33550336
max_used_connections=12
InnoDB: Thread 147466 stopped in file ha_innodb.cc line 522
max_connections=100
InnoDB: Thread 229391 stopped in file ../include/hash0hash.ic line 21
InnoDB: Thread 262161 stopped in file log0log.c line 1280
threads_connected=12
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2394732 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x8cd8f00
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6b4810a8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8169949
0x4006896c
0x831bd84
0x82f7c5f
0x8211040
0x8204aba
0x81da821
0x8204c3e
0x8185c98
0x81872a3
0x817fe8f
0x817f714
0x817eade
0x40062f60
0x401f5327
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8d0e0e8 = commit
thd->thread_id=24
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
InnoDB: Thread 327701 stopped in file ha_innodb.cc line 522
[20 Mar 2005 8:12] Heikki Tuuri
Jan,

looks like compile-pentium and compile-pentium-debug create ok binaries, while

compile-pentium-debug --with-debug-full

makes badly broken binaries where memory gets corrupt here and there.

We have to study this more. You could try disabling InnoDB's debug code by putting:

#undef UNIV_DEBUG

to the start of univ.i. Maybe some of InnoDB's debug code causes serious memory corruption. Or, it might be something else in the build.

Regards,

Heikki
[21 Mar 2005 20:55] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/23248
[21 Mar 2005 21:06] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/23249
[21 Mar 2005 21:07] Heikki Tuuri
Jan,

the log flush assertion is probably fixed by the patch I just bk committed.

But that does not explain other crashes that I got.

This must be tested extensively. Do we still get assertion failures after the patch?

Regards,

Heikki
[22 Mar 2005 15:39] Heikki Tuuri
Hi!

The log write assertion failure no longer happens after my patch last night.

And I am not able to repeat the crashes I got on Friaday in other places. Maybe something was wrong in my build on Friday.

I am moving this bug report to the 'patch approved' state.

Regards,

Heikki