| Bug #19424 | InnoDB: Possibly a memory overrun of the buffer being freed (64-bit Visual C) | ||
|---|---|---|---|
| Submitted: | 28 Apr 2006 13:10 | Modified: | 18 Jun 2010 12:42 |
| Reporter: | Andrew Hind | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: InnoDB storage engine | Severity: | S1 (Critical) |
| Version: | 5.0.21-winx64, 5.0.19-winx64, 5.1 | OS: | Windows (Windows Server 2003 x64, XP x64) |
| Assigned to: | Iggy Galarza | CPU Architecture: | Any |
[28 Apr 2006 13:40]
Valeriy Kravchuk
Thank you for a problem report. Please, send your my.ini files. Any ideas on how to repeat (or what's going on while you get that overrun) are welcomed.
[3 May 2006 12:16]
Heikki Tuuri
Andrew, please post all the error printouts. This could be bad RAM. Please run memtestx86 or similar. Regards, Heikki
[3 May 2006 16:27]
Andrew Hind
Here it is again.
I have not had chance to run the memory test suggested although we did run the SISoftware Sandra lite stuff when we first started seeing the problem.
I have also noticed similar problems on restarting mysql.
I have tried the innodb_force_recovery option and needed 1,3,4 so far if this helps.
The latest error is below
D:\MySQL Server 5.0\bin>mysqld --console
060503 15:08:12 [Warning] Changed limits: max_open_files: 2048 max_connections:
100 table_cache: 969
060503 15:08:13 InnoDB: Started; log sequence number 9 1391892912
060503 15:08:14 [Note] mysqld: ready for connections.
Version: '5.0.19-log' socket: '' port: 3306 Source distribution
InnoDB: Error: Memory area size 1024, next area size 0 not a power of 2!
InnoDB: Possibly a memory overrun of the buffer being freed here.
InnoDB: Apparent memory corruption: mem dump len 500; hex 0000a26b7316000000002
c6c731600000000b66c731600000000406d731600000000ca6d731600000000546e731600000000d
e6e731600000000686f731600000000f26f7316000000007c7073160000000000003f800b8000010
0140e9480000000000169d1313134363637323434323636363a65376231373861632d646162652d3
13164612d393935342d3933663031633037656163300080d265656136322d646162652d313164612
d393935342d3933663031633037656163300000003f800b80000100140efc80000000000169d2313
134363637323434323636363a65376231373861632d646162652d313164612d393935342d3933660
004000000000000000000000000000000000000000000009268a5200000000030646174612e63009
60000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000068020000000000000000000000000000000000000000000068010000000000007
00000000000000000000000000000000000000000000000030000000000000003000000000000002
0ed42060000000014000000000000001c0000000000000040000000000000000aaec950000000000
9000000000000000c000000000000000f01210000000000960000000000000000000000000000000
1000000000000000300; asc ¢ks ,ls ¶ls @ms Êms Tns Þns
hos òos |ps ? " iÑ1146672442666:e7b178ac-dabe-11da-
9954-93f01c07eac0 Òeea62-dabe-11da-9954-93f01c07eac0 ? ü iÒ114667
2442666:e7b178ac-dabe-11da-9954-93f 'h¥ 0data.c -
h h p
íB @ ®ÉP
! - ;
InnoDB: Scanning
backward trying to find previous allocated mem blocks
Mem block at - 1000, file w0ins.c, line 96
Mem block at - 1256, file w0sel.c, line 2934
Mem block at - 1512, file w0sel.c, line 2934
Mem block at - 1768, file w0sel.c, line 2934
Mem block at - 2024, file w0sel.c, line 2934
Mem block at - 2280, file w0sel.c, line 2934
Mem block at - 2536, file w0sel.c, line 2934
Mem block at - 2792, file w0sel.c, line 2934
Mem block at - 3048, file w0sel.c, line 2934
Freed mem block at - 4072, file r0sea.c, line 1163
InnoDB: Scanning forward trying to find next allocated mem blocks
Freed mem block at + 24, file 0data.c, line 150
Freed mem block at + 536, file r0sea.c, line 1164
Freed mem block at + 792, file r0sea.c, line 1164
Freed mem block at + 1048, file 0data.c, line 150
Freed mem block at + 2072, file r0sea.c, line 505
Freed mem block at + 2328, file r0sea.c, line 1163
Freed mem block at + 2584, file r0sea.c, line 1164
Freed mem block at + 3096, file e0cur.c, line 923
Freed mem block at + 3608, file r0sea.c, line 1164
Freed mem block at + 4120, file r0sea.c, line 1164
060503 17:15:41InnoDB: Assertion failure in thread 424 in file .\mem\mem0pool.c
line 505
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
InnoDB: Thread 2276 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b
uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111
InnoDB: Thread 3956 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b
uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111
InnoDB: Thread 3780 stopped in file .\os\os0sync.c line 487
InnoDB: Thread 2344 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b
uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111
[3 May 2006 16:27]
Andrew Hind
I am now getting this in restart too (thi is not always the case)
D:\MySQL Server 5.0\bin>mysqld --console
060503 17:20:10 [Warning] Changed limits: max_open_files: 2048 max_connections:
100 table_cache: 969
060503 17:20:10 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
060503 17:20:11 InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 9 2675259001.
InnoDB: Doing recovery: scanned up to log sequence number 9 2680501760
InnoDB: Doing recovery: scanned up to log sequence number 9 2685744640
InnoDB: Doing recovery: scanned up to log sequence number 9 2690987520
InnoDB: Doing recovery: scanned up to log sequence number 9 2696230400
InnoDB: Doing recovery: scanned up to log sequence number 9 2701473280
InnoDB: Doing recovery: scanned up to log sequence number 9 2706716160
InnoDB: Doing recovery: scanned up to log sequence number 9 2711959040
InnoDB: Doing recovery: scanned up to log sequence number 9 2717201920
InnoDB: Doing recovery: scanned up to log sequence number 9 2722444800
InnoDB: Doing recovery: scanned up to log sequence number 9 2727687680
InnoDB: Doing recovery: scanned up to log sequence number 9 2732930560
InnoDB: Doing recovery: scanned up to log sequence number 9 2738173440
InnoDB: Doing recovery: scanned up to log sequence number 9 2743416320
InnoDB: Doing recovery: scanned up to log sequence number 9 2748659200
InnoDB: Doing recovery: scanned up to log sequence number 9 2753902080
InnoDB: Doing recovery: scanned up to log sequence number 9 2759144960
InnoDB: Doing recovery: scanned up to log sequence number 9 2764387840
InnoDB: Doing recovery: scanned up to log sequence number 9 2769630720
InnoDB: Doing recovery: scanned up to log sequence number 9 2774873600
InnoDB: Doing recovery: scanned up to log sequence number 9 2780116480
InnoDB: Doing recovery: scanned up to log sequence number 9 2785359360
InnoDB: Doing recovery: scanned up to log sequence number 9 2790602240
InnoDB: Doing recovery: scanned up to log sequence number 9 2795845120
InnoDB: Doing recovery: scanned up to log sequence number 9 2801088000
InnoDB: Doing recovery: scanned up to log sequence number 9 2801562649
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 579 row operations to undo
InnoDB: Trx id counter is 0 4357632
060503 17:20:29 InnoDB: Starting an apply batch of log records to the database.
..
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20 21 InnoDB: Error: Memory area size 4096, next area size 0 not a power of 2!
InnoDB: Possibly a memory overrun of the buffer being freed here.
InnoDB: Apparent memory corruption: mem dump len 500; hex 0000604d3706000000000
0000000000000009268a5200000000074306d656d2e63002f0000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000d8000000000000000
00000000000000000000000000000007000000000000000700000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
010000000000000000000000000000060303706000000009268a5200000000074306d656d2e63002
f0000000000000000000000000000000000000000000000000000000000000078403706000000000
000000000000000100c0000000000000000000000000000000000000000000008080000000000007
0000000000000000000000000000000e8703706000000000000000000000000ffffffffffffffff0
00000000000000018583706000000000300000000000000000100000000000008000000000000000
00000000000000000000000000000000000000000000000e8143706000000000d000000000000001
1000000000000000100; asc `M7 'h¥ t0mem.c /
Ø p p
`07 'h¥ t0mem.c /
x@7 p
èp7 ÿÿÿÿÿÿÿÿ X7
è 7 ;
InnoDB: Scanning backward trying to find previous allocated mem blocks
Freed mem block at - 232, file t0mem.c, line 47
Freed mem block at - 1000, file t0mem.c, line 194
Freed mem block at - 1256, file t0mem.c, line 47
Freed mem block at - 2024, file t0mem.c, line 47
Freed mem block at - 2280, file t0mem.c, line 47
Mem block at - 4072, file t0mem.c, line 47
Freed mem block at - 4328, file t0mem.c, line 47
Freed mem block at - 5096, file t0mem.c, line 194
Freed mem block at - 5352, file t0mem.c, line 194
Freed mem block at - 6376, file t0mem.c, line 47
InnoDB: Scanning forward trying to find next allocated mem blocks
Freed mem block at + 24, file t0mem.c, line 47
Freed mem block at + 1816, file t0mem.c, line 194
Freed mem block at + 2840, file t0mem.c, line 47
Freed mem block at + 3096, file t0mem.c, line 47
Freed mem block at + 3864, file t0mem.c, line 47
Freed mem block at + 4120, file t0mem.c, line 47
Freed mem block at + 5656, file t0mem.c, line 194
Freed mem block at + 5912, file t0mem.c, line 47
Freed mem block at + 6936, file t0mem.c, line 194
Freed mem block at + 7192, file t0mem.c, line 194
060503 17:20:34InnoDB: Assertion failure in thread 3416 in file .\mem\mem0pool.c
line 505
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
InnoDB: Thread 3692 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b
uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111
[9 May 2006 1:40]
Jorge del Conde
I was unable to reproduce this bug under WinXP64 using the attatched my.ini file: C:\my\mysql\data>mysql -uroot mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 to server version: 5.0.21-log Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> show variables like "myisam%"; +---------------------------+---------------+ | Variable_name | Value | +---------------------------+---------------+ | myisam_data_pointer_size | 6 | | myisam_max_sort_file_size | 2147483647 | | myisam_recover_options | OFF | | myisam_repair_threads | 1 | | myisam_sort_buffer_size | 16777216 | | myisam_stats_method | nulls_unequal | +---------------------------+---------------+ 6 rows in set (0.00 sec) mysql> I tested this under 5.0.19 and 5.0.21
[9 May 2006 8:41]
Andrew Hind
I have now run memtest86 v3.2 on the same machine without any problem for 24 hours
[9 May 2006 10:37]
MySQL Verification Team
Andrew, please tell us what kind of queries ran at the time of the assertion failure? Perhaps enable the general query log to fetch those details..
[9 May 2006 12:17]
Heikki Tuuri
Reopening this because I have not yet analyzed properly the printouts. The printouts do look loike random memory corruption.
[10 May 2006 7:45]
Andrew Hind
Another .....
I have not had chance to add query logging yet.
I am trying to get some performance data out...
Note I started out the data files etc deleted - it is the easiest way for me to restart at the moment rather than recovering and dropping then creating the database.
Regards
Andy
D:\MySQL Server 5.0\bin>mysqld --console
060509 12:32:26 [Warning] Changed limits: max_open_files: 2048 max_connections:
100 table_cache: 969
InnoDB: The first specified data file .\ibdata1 did not exist:
InnoDB: a new database to be created!
060509 12:32:26 InnoDB: Setting file .\ibdata1 size to 50 MB
InnoDB: Database physically writes the file full: wait...
060509 12:32:26 InnoDB: Log file .\ib_logfile0 did not exist: new to be created
InnoDB: Setting log file .\ib_logfile0 size to 128 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
060509 12:32:28 InnoDB: Log file .\ib_logfile1 did not exist: new to be created
InnoDB: Setting log file .\ib_logfile1 size to 128 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
060509 12:32:30 InnoDB: Started; log sequence number 0 0
060509 12:32:30 [Note] mysqld: ready for connections.
Version: '5.0.19-log' socket: '' port: 3306 Source distribution
InnoDB: Error: Memory area size 2048, next area size 0 not a power of 2!
InnoDB: Possibly a memory overrun of the buffer being freed here.
InnoDB: Apparent memory corruption: mem dump len 500; hex 000003000000000000001
84c99340000000024000000000000000c000000000000000f012100000000006c000000000000002
c0100000000000001000000000000000300000000000000000000000000000003000000000000000
300000000000000e8873b06000000000f012100000000006c0000000000000073706163655370616
54c99340000000009000000000000000c000000000000000f0121000000000096000000000000000
001000000000000010000000000000003000000000000006e4c9934000000000b000000000000000
c000000000000000f012100000000002c0100000000000070883b060000000001000000000000000
008000000000000000000000000000060983b06000000009268a5200000000074306d656d2e63002
f0000000000000000000000000000000000000000000000000000000000000078943a06000000000
000000000000000d00500000000000000000000000000000000000000000000b8030000000000007
0000000000000000000000000000000494255465f44554d4d5900000000000004000000000000000
000000000000000ffffffffffffffff0000000000000000188c3b06000000000c000000000000000
f012100000000009600000000000000040000000000000001000000000000000300000000000000e
8943a06000000000300; asc L 4 $ ! l ,
è╬; ! l spaceSpaeL 4
! - nL 4
! , p ; ` ; 'h¥ t0mem.c /
x": Ð ¸ p
IBUF_DUMMY ÿÿÿÿÿÿÿÿ O;
! - è": ;
InnoDB: Scanning backward trying to find previous allocated mem blocks
Mem block at - 2024, file w0sel.c, line 2934
Freed mem block at - 2536, file mysql.c, line 592
Freed mem block at - 3048, file 0pcur.c, line 29
Freed mem block at - 4072, file mysql.c, line 592
Freed mem block at - 6120, file odb.cpp, line 3189
Mem block at - 10216, file mysql.c, line 592
Freed mem block at - 11240, file n0dyn.c, line 33
Mem block at - 12264, file w0sel.c, line 2934
Mem block at - 14312, file w0sel.c, line 2934
Freed mem block at - 16360, file mysql.c, line 592
InnoDB: Scanning forward trying to find next allocated mem blocks
Freed mem block at + 24, file t0mem.c, line 47
Freed mem block at + 2072, file 0data.c, line 150
Freed mem block at + 3096, file 0data.c, line 150
Freed mem block at + 4120, file r0sea.c, line 1163
Freed mem block at + 4376, file r0sea.c, line 1164
Freed mem block at + 5144, file , line 0
Mem block at + 6168, file mysql.c, line 592
Freed mem block at + 8216, file t0mem.c, line 194
Freed mem block at + 9240, file t0mem.c, line 47
Mem block at + 10264, file w0ins.c, line 96
060509 22:09:20InnoDB: Assertion failure in thread 6668 in file .\mem\mem0pool.c
line 505
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
InnoDB: Thread 6076 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b
uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111
InnoDB: Thread 6704 stopped in file .\os\os0sync.c line 487
InnoDB: Thread 6716 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b
uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111
[13 May 2006 12:16]
Michail Epikhin
I have problems of a similar nature:
060513 15:11:08 InnoDB: Started; log sequence number 17 1806942364
060513 15:11:09 [Note] MySQL: ready for connections.
Server crash, restarting
Version: '5.0.21' socket: '' port: 3306 Source distribution
060513 15:25:21 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
060513 15:25:22 InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 17 1807123618.
InnoDB: Doing recovery: scanned up to log sequence number 17 1812366336
...
InnoDB: 4 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 16095 row operations to undo
InnoDB: Trx id counter is 0 750336
060513 15:27:17 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 InnoDB: Error: Memory area size 2048, next area size 0 not a power of 2!
InnoDB: Possibly a memory overrun of the buffer being freed here.
InnoDB: Apparent memory corruption: mem dump len 500; hex 000000000000000000001000000000000000ffffffffffffffff0000000000000000b081670700000000030000000000000000000000000000003200000000000000000000000000000000000000000000000000000000000000e864670700000000000000000000000070000000000000001100000000000000ffffffffffffffff0000000000000000b881670700000000030000000000000000000000000000001400000000000000000000000000000000000000000000000000000000000000e864670700000000000000000000000000000000000000001200000000000000ffffffffffffffff0000000000000000c0816707000000000008000000000000000000000000000000000000000000009268a5200000000074306d656d2e63002f0000000000000000000000000000000000000000000000000000000000000078646707000000000000000000000000d005000000000000000000000000000000000000000000004803000000000000700000000000000000000000000000004c4f475f44554d4d590000000000000001000000000000000000000000000000ffffffffffffffff0000000000000000b07b670700000000030000000000000000000000000000001400000000000000000000000000000000000000000000000000000000000000e864670700000000f870; asc яяяяяяяя °Ѓg 2 иdg p яяяяяяяя ёЃg иdg яяяяяяяя АЃg ’hҐ t0mem.c / xdg Р H p LOG_DUMMY яяяяяяяя °{g иdg шp;
InnoDB: Scanning backward trying to find previous allocated mem blocks
Freed mem block at - 1000, file t0mem.c, line 209
Freed mem block at - 1256, file t0mem.c, line 209
Mem block at - 2024, file t0mem.c, line 47
Mem block at - 3048, file t0mem.c, line 47
Mem block at - 4072, file t0mem.c, line 47
Freed mem block at - 4328, file t0mem.c, line 47
Mem block at - 5096, file t0mem.c, line 47
Mem block at - 5352, file t0mem.c, line 47
Mem block at - 5608, file t0mem.c, line 47
Mem block at - 6120, file purge.c, line 49
InnoDB: Scanning forward trying to find next allocated mem blocks
Freed mem block at + 24, file t0mem.c, line 47
Freed mem block at + 536, file t0mem.c, line 209
Freed mem block at + 792, file t0mem.c, line 47
Freed mem block at + 1048, file t0mem.c, line 209
Mem block at + 2072, file t0mem.c, line 47
Freed mem block at + 2840, file t0mem.c, line 209
Freed mem block at + 3096, file t0mem.c, line 47
Freed mem block at + 3352, file t0mem.c, line 209
Freed mem block at + 4120, file t0mem.c, line 209
Mem block at + 5144, file t0mem.c, line 209
060513 15:27:18InnoDB: Assertion failure in thread 2436 in file .\mem\mem0pool.c line 505
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mys
[13 May 2006 12:39]
MySQL Verification Team
Andrew, please can you try identify if the ASCII in your first post comes from any particular table ("7a-d6b2-11da-8777-ed4b95c0") Can you show us that table structure ?
Michail, please give us more info, such as OS, 32-bit/64-bit, or which queries are running at time of crash? Do you use UTF8? Thanks,
[13 May 2006 13:23]
Michail Epikhin
Sorry my English. Hardware - Dual Xeon, 8GB RAM, OS - Windows 2003 x64 R2, cp1251 only. I can`t identify crash query - many application work with DB. If I run mysqld-debug crach never not occur, if I run mysqld-max-nt and set table_cache = 4096, innodb_additional_mem_pool_size = 16M crach occur without delay, else mysqld-max-nt work 1-2 day and crach. 32 bit version work good at this hardware(using AWE) about 1 year. Thank you.
[15 May 2006 5:29]
Heikki Tuuri
Hi! I still need to analyze the hex dumps. Could this be a bug in InnoDB which only manifests on a 64-bit Windows, or could this be a Visual C compiler bug on 64-bit Windows? One possible explanation is a bug in InnoDB mutexes. sync0sync.ic implements the InnoDB mutex using embedded assembler code. Since 64-bit processors have different registers, the code might not work right there. We should test using the Windows CriticalSection() function call to implement the InnoDB mutex, just like we use pthread_mutex() on Unix. Regards, Heikki
[15 May 2006 5:35]
Heikki Tuuri
One could note that 64-bit Windows binaries of MySQL have been available only for a couple of months. This could also be a bug of InnoDB's 64-bit cleanness on Windows.
[15 May 2006 9:18]
Heikki Tuuri
Looks like InnoDB in a 64-bit Windows compilation does NOT use inline assembler to implement its mutex. Thus we are left with two main candidates for the bug: 1. A Visual C compiler bug in 64 bits 2. InnoDB unclean 64-bit code in mem0pool.c or some other InnoDB file univ.i: #if (defined(WIN32) || defined(_WIN32) || defined(WIN64) || defined(_WIN64)) &&\ !defined(MYSQL_SERVER) && !defined(__WIN__) #undef __WIN__ #define __WIN__ #include <windows.h> #if !defined(WIN64) && !defined(_WIN64) #define UNIV_CAN_USE_X86_ASSEMBLER #endif
[15 May 2006 9:20]
Heikki Tuuri
Andrew, if you run mysqld-debug.exe like Michail, does the crash happen? If that binary does not crash, that suggests a Visual C compiler bug. Regards, Heikki
[15 May 2006 9:38]
Michail Epikhin
If necessary, I can run any test application on my server.
[15 May 2006 11:28]
Andrew Hind
Hi Here is the DB schema as requested. It is not the same schema as for the original error, but it does have the same problem. To get on with stuff we have rebuilt the machine as 32 bit and gone back to 32 bit mysql for now, so I can not try the debug version. I have had no problems with the 32 bit mysql. Regards Andy -- MySQL Administrator dump 1.4 -- -- ------------------------------------------------------ -- Server version 5.0.20-community /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; /*!40101 SET NAMES utf8 */; /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */; /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */; /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */; -- -- Create schema alfresco -- CREATE DATABASE /*!32312 IF NOT EXISTS*/ alfresco; USE alfresco; DROP TABLE IF EXISTS `access_control_entry`; CREATE TABLE `access_control_entry` ( `id` bigint(20) NOT NULL auto_increment, `acl_id` bigint(20) NOT NULL, `permission_id` bigint(20) NOT NULL, `authority_id` varchar(100) NOT NULL, `allowed` bit(1) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `acl_id` (`acl_id`,`permission_id`,`authority_id`), KEY `FKF064DF7560601995` (`permission_id`), KEY `FKF064DF75B25A50BF` (`authority_id`), KEY `FKF064DF75B9553F6C` (`acl_id`), CONSTRAINT `FKF064DF75B9553F6C` FOREIGN KEY (`acl_id`) REFERENCES `access_control_list` (`id`), CONSTRAINT `FKF064DF7560601995` FOREIGN KEY (`permission_id`) REFERENCES `permission` (`id`), CONSTRAINT `FKF064DF75B25A50BF` FOREIGN KEY (`authority_id`) REFERENCES `authority` (`recipient`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `access_control_list`; CREATE TABLE `access_control_list` ( `id` bigint(20) NOT NULL auto_increment, `node_id` bigint(20) default NULL, `inherits` bit(1) NOT NULL, PRIMARY KEY (`id`), KEY `FK18486D3B7F2C8017` (`node_id`), CONSTRAINT `FK18486D3B7F2C8017` FOREIGN KEY (`node_id`) REFERENCES `node` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `applied_patch`; CREATE TABLE `applied_patch` ( `id` varchar(32) NOT NULL, `description` text, `fixes_from_schema` int(11) default NULL, `fixes_to_schema` int(11) default NULL, `applied_to_schema` int(11) default NULL, `target_schema` int(11) default NULL, `applied_on_date` datetime default NULL, `applied_to_server` varchar(64) default NULL, `was_executed` bit(1) default NULL, `succeeded` bit(1) default NULL, `report` text, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `auth_ext_keys`; CREATE TABLE `auth_ext_keys` ( `id` varchar(100) NOT NULL, `externalKey` varchar(100) NOT NULL, PRIMARY KEY (`id`,`externalKey`), KEY `FK31D3BA097B7FDE43` (`id`), CONSTRAINT `FK31D3BA097B7FDE43` FOREIGN KEY (`id`) REFERENCES `authority` (`recipient`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `authority`; CREATE TABLE `authority` ( `recipient` varchar(100) NOT NULL, PRIMARY KEY (`recipient`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `child_assoc`; CREATE TABLE `child_assoc` ( `id` bigint(20) NOT NULL auto_increment, `parent_node_id` bigint(20) default NULL, `child_node_id` bigint(20) default NULL, `type_qname` varchar(255) NOT NULL, `qname` varchar(255) NOT NULL, `is_primary` bit(1) default NULL, `assoc_index` int(11) default NULL, PRIMARY KEY (`id`), KEY `FKC6EFFF3274173FF4` (`child_node_id`), KEY `FKC6EFFF328E50E582` (`parent_node_id`), CONSTRAINT `FKC6EFFF328E50E582` FOREIGN KEY (`parent_node_id`) REFERENCES `node` (`id`), CONSTRAINT `FKC6EFFF3274173FF4` FOREIGN KEY (`child_node_id`) REFERENCES `node` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `node`; CREATE TABLE `node` ( `id` bigint(20) NOT NULL auto_increment, `protocol` varchar(50) NOT NULL, `identifier` varchar(100) NOT NULL, `uuid` varchar(36) NOT NULL, `type_qname` varchar(255) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `protocol` (`protocol`,`identifier`,`uuid`), KEY `FK33AE02D24ADD25` (`protocol`,`identifier`), CONSTRAINT `FK33AE02D24ADD25` FOREIGN KEY (`protocol`, `identifier`) REFERENCES `store` (`protocol`, `identifier`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `node_aspects`; CREATE TABLE `node_aspects` ( `node_id` bigint(20) NOT NULL, `qname` varchar(200) default NULL, KEY `FK2B91A9DE7F2C8017` (`node_id`), CONSTRAINT `FK2B91A9DE7F2C8017` FOREIGN KEY (`node_id`) REFERENCES `node` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `node_assoc`; CREATE TABLE `node_assoc` ( `id` bigint(20) NOT NULL auto_increment, `source_node_id` bigint(20) default NULL, `target_node_id` bigint(20) default NULL, `type_qname` varchar(255) NOT NULL, PRIMARY KEY (`id`), KEY `FK5BAEF398B69C43F3` (`source_node_id`), KEY `FK5BAEF398A8FC7769` (`target_node_id`), CONSTRAINT `FK5BAEF398A8FC7769` FOREIGN KEY (`target_node_id`) REFERENCES `node` (`id`), CONSTRAINT `FK5BAEF398B69C43F3` FOREIGN KEY (`source_node_id`) REFERENCES `node` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `node_properties`; CREATE TABLE `node_properties` ( `node_id` bigint(20) NOT NULL, `actual_type` varchar(15) NOT NULL, `multi_valued` bit(1) NOT NULL, `persisted_type` varchar(15) NOT NULL, `boolean_value` bit(1) default NULL, `long_value` bigint(20) default NULL, `float_value` float default NULL, `double_value` double default NULL, `string_value` text, `serializable_value` blob, `qname` varchar(200) NOT NULL, PRIMARY KEY (`node_id`,`qname`), KEY `FKC962BF907F2C8017` (`node_id`), CONSTRAINT `FKC962BF907F2C8017` FOREIGN KEY (`node_id`) REFERENCES `node` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `node_status`; CREATE TABLE `node_status` ( `protocol` varchar(50) NOT NULL, `identifier` varchar(100) NOT NULL, `guid` varchar(36) NOT NULL, `node_id` bigint(20) default NULL, `change_txn_id` varchar(56) NOT NULL, `deleted` bit(1) NOT NULL, PRIMARY KEY (`protocol`,`identifier`,`guid`), KEY `FK38ECB8CF7F2C8017` (`node_id`), CONSTRAINT `FK38ECB8CF7F2C8017` FOREIGN KEY (`node_id`) REFERENCES `node` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `permission`; CREATE TABLE `permission` ( `id` bigint(20) NOT NULL auto_increment, `type_qname` varchar(200) NOT NULL, `name` varchar(100) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `type_qname` (`type_qname`,`name`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `store`; CREATE TABLE `store` ( `protocol` varchar(50) NOT NULL, `identifier` varchar(100) NOT NULL, `root_node_id` bigint(20) default NULL, PRIMARY KEY (`protocol`,`identifier`), KEY `FK68AF8E122DBA5BA` (`root_node_id`), CONSTRAINT `FK68AF8E122DBA5BA` FOREIGN KEY (`root_node_id`) REFERENCES `node` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; DROP TABLE IF EXISTS `version_count`; CREATE TABLE `version_count` ( `protocol` varchar(100) NOT NULL, `identifier` varchar(100) NOT NULL, `version_count` int(11) NOT NULL, PRIMARY KEY (`protocol`,`identifier`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET SQL_MODE=@OLD_SQL_MODE */; /*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */; /*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */; /*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */; /*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */; /*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */; /*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
[20 May 2006 16:07]
Dany Hajjar
I have a very similar configuration (win 2003 X64 8 GIG ram) and I can consistently reproduce this bug by highly stressing the SQL. Under normal load, everything functions as expected. However, with when a special application stress test is used to simulate large number of user interactions, MySQL crashes almost 5 seconds after the start. Memory allocation by MySQL seams to jump dramatically until the assert error shown in this bug report is generated. Using mysqld-debug, the problem doesn't appear however, this may not mean much as performance under this configuration cannot be simulated in a similar manner to mysqld. Let me know if I can help my producing any further logs or debug data.
[24 May 2006 18:12]
Dany Hajjar
Is there a significant performance boost to running 64 bit MySQL on win 64 or is it resonable to run MySQL 32 bit on top of Win 64 as a workaround to this issue?
[4 Jun 2006 0:36]
James Day
It's reasonable to run 32 bit MySQL unless you need the extra memory allocation made available by 64 bit addressing. If you don't need the extra memory allocation capability the 32 bit version is likely to be faster.
[4 Jun 2006 4:59]
Dany Hajjar
Thanks James. If I run sql32 now and this issue is resolved, would migration be as simple as running the new binaries with the existing data and log files? Regards, Dany
[4 Jun 2006 4:59]
Dany Hajjar
Thanks James. If I run sql32 now and this issue is resolved, would migration to 64bit be as simple as running the new binaries with the existing data and log files? Regards, Dany
[6 Jun 2006 21:57]
James Day
Dany, Yes, should be no more work than that. Do make a backup first in case you're the first to discover some new bug. James Day Support Engineer, MySQL AB
[27 Jul 2006 16:41]
Heikki Tuuri
One could test compiling the binary with no optimizations in the /innobase/mem directory. If this is a compiler bug, it most probably surfaces in that code.
[30 Jul 2006 15:50]
Heikki Tuuri
I will analyze the hex dumps next week if they reveal anything. A workaround might be to make innodb_additional_mem_pool_size as small as possible (512k). The assertion failures happen there. And when new versions of 64-bit Visual C compiler appear, the problem may disappear. --Heikki
[31 Jul 2006 9:38]
Michail Epikhin
Can I use another version of compiler for problem solving? Can I get solution of MySQL server with support Win64 for MS Visual Studio 2003(2005)?
[1 Aug 2006 5:28]
Heikki Tuuri
Michail, I think the 32-bit binary runs ok on a 64-bit Windows. And the mysqld-debug.exe of the 64-bit distro also runs ok. If you have the 64-bit Microsoft Visual C compiler you can try building mysqld yourself and removing compiler optimizations from /innobase/mem or some other directories. If it produces a good binary, then we would know that the compiler bug is there. You can also try to work around the bug by setting innodb_additional_mem_pool_size to 512 kB. Maybe the bug does not exhibit if we do allocation with the OS malloc. Regards, Heikki
[1 Aug 2006 6:29]
Heikki Tuuri
The hex dumps look ok. Unfortunately, because we only print 250 bytes forward from the mem_area_t block start, they do not show the most interesting bytes which would be at the start of the next area block. If this is a genuine memory overrun, then those overruns would happen for mem_block_t blocks allocated in many different parts of code! And if this is a genuine memory overrun, then the overwriting bytes seem to be zero, and there are no more than 24 bytes of overrun, because the header of the next mem_block_t seems to be ok. We can make InnoDB to print longer hex dumps and add more assertions to mem0pool.c and mem0mem.c to diagnose this further. An interesting binary would be one where UNIV_MEM_DEBUG is defined in /innobase/include/univ.i Another interesting binary is one where UNIV_MUST_NOT_INLINE is defined in univ.i. The compiler bug might be in inlining.
[25 Aug 2006 20:56]
Heikki Tuuri
We could test defining UNIV_MUST_NOT_INLINE in 64-bit Windows builds. That will slow down the binary, but at least we would then know if the compiler (?) bug is associated with function inlining.
[4 Sep 2006 6:38]
Michail Epikhin
I built mysqld using VS2005 and Microsoft Platform SDK for Windows Server 2003 R2(Win Svr 2003 x64 Build Env (Retail)) and remove compiler optimizations from /innobase/mem. I can't repeat crash, server works good ~3 days.
[6 Sep 2006 10:10]
Heikki Tuuri
Sunny should study this once he has the Win 64-bit environment up.
[4 Oct 2006 14:56]
Heikki Tuuri
Mikhail, thank you. Maybe we can work around the problem simply by removing compiler optimizations from /innobase/mem Regards, Heikki
[24 Oct 2006 10:31]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/14241 ChangeSet@1.2295, 2006-10-24 12:31:18+02:00, joerg@trift2. +1 -0 innobase/CMakeLists.txt : Change done by Ignacio Galarza (igalarza@mysql.com) Bug #19424 InnoDB: possibly a memory overrun of the buffer being freed (Windows 64) Fix is to reduce optimization if the compiler is "Visual Studio 8 2005 Win64".
[25 Oct 2006 7:00]
Michail Epikhin
Sorry my English. I cant answer to Timothy Smith by email. I use: MS VS 2005, MySQL source for Windows from http://dev.mysql.com/downloads/mysql/5.0.html#downloads (I build version 5.0.24a and 5.0.26, all work fine), Microsoft Platform SDK for Windows Server 2003 R2. My steps: 1. Convert project from 7.1 to 8.0 2. Disable optimization (/Od) on file mem0mem.c and mem0pool.c (not for all innobase) 3. Start Windows Server 2003 64-bit Build Environment (Set Win Svr 2003 x64 Build Env (Retail)) 4. Start VS from Build Environment (devenv /useenv) 5. Build "nt" configuration of MySQL We have large transaction: about 60000 rows delete/insert on one transaction, 1000-5000 rows update on one transaction, and about 10000000 updates per day. If needed I can send my solution and binary for you testing.
[25 Oct 2006 20:36]
Timothy Smith
Michail, Thanks a lot for the details. The patch to remove optimizations in InnoDB has been approved, and will be in future releases. We would appreciate any feedback you have with the new release, or if you notice similar problems in the future. Regards, Timothy
[29 Oct 2006 23:12]
Henric Carlström
When could we see this new build at the download page? We are in urgent need of a working version of the x64 binary. :-\
[1 Nov 2006 21:15]
Timothy Smith
Hi, Henric, I don't have a good answer for you. The 5.0.28 enterprise binaries do have this 'fix' applied. Unfortunately it didn't make it into the 5.0.27 community source release, which came fast on the heels of 5.0.26 with an ABI compatibility fix. I don't have info on when the next 5.0.x community source release will be made. Of course, the code is always available from mysql.bkbits.net; and this particular patch is also trivial to apply by hand. (See the commit e-mail linked above.) Also, I don't believe this change will be in the 5.1.12 release. Finally, this change is not a sufficient fix for the bug. Although it does appear to remove some crashing behavior in certain contexts, we don't have a clear test case for this, so we can't say "yes, this is a compiler bug that we are working around" and not "this is a bug in MySQL which is more likely to surface when the code is optimized". So we are actively working on this bug, still. Regards, Timothy
[3 Nov 2006 22:12]
Iggy Galarza
I cannot reproduce the behaviour described. I can successfully restore the given large database and delete all the rows. In order to move this bug along, I have made three different versions of the mysql-5.0.29 (BETA) Win64 executables. 1. Release configuration built and packaged with debug symbols. ftp://ftp.mysql.com/pub/mysql/download/19424/mysql-noinstall-5.0.29-winx64-rel_symbols.zip 2. Release configuration built without compiler optimizations for innobase/mem/* and packaged with debug symbols ftp://ftp.mysql.com/pub/mysql/download/19424/mysql-noinstall-5.0.29-winx64-rel_symbols_noo... 3. Release configuration with UNIV_MUST_NOT_INLINE defined in univ.i and packaged with debug symbols ftp://ftp.mysql.com/pub/mysql/download/19424/mysql-noinstall-5.0.29-winx64-rel_symbols_inl... (Note - This is current as of today and is intended only for testing this bug) The debug symbols include should give us more insight into this problem. If you can reproduce the problem described, please try to do so with each of the 3 versions from above and report your results. Please save the debug dump (from at least version 1 above) for further examination.
[3 Nov 2006 22:47]
Timothy Smith
Michail, others, One brief comment - if you have built your own binary with success, please double-check that it is truly a 64-bit binary. The solution available in the Windows source distribution generates a 32-bit executable. You can check this by looking in the Task Manager. A 32-bit executable will show up as "mysqld.exe * 32". This is important, as we're quite certain that the bug is 64-bit specific, so testing a 32-bit binary will only confuse the results. Thank you, Timothy
[5 Nov 2006 12:57]
Michail Epikhin
I use: innodb_buffer_pool_size = 6G. It can't work on 32 bit version Windows. To build 64-bit binary from source distribution I use: http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/30887.pdf
[6 Nov 2006 20:24]
Timothy Smith
Michail, Thank you for informing us a bit more about how you built your binary. I hope my previous comment was not too insulting - of course my goal is simply to ensure that we have accurate information, not to offend someone who is helping us a lot! If you can possibly test the builds that Ignacio made available for download, that would be very much appreciated. It was built the same way as our standard downloads (using cmake and the CMakeLists.txt files which are available in our BitKeeper trees). The most useful feedback, I think, would be to compare these known builds in your situation. If we could find out "Build A crashes after only a few minutes; Build B has been running for two days without problem, and Build C also ran for two days before I decided to test another", that would be wonderful information. Currently, none of these builds are crashing in any of our test environments. We'll continue exploring this on our side, but if you could check these binaries in your environment it would be most helpful. We need more data from people who are actually experiencing this problem, in order to move forward effectively. Thanks, Timothy
[10 Nov 2006 13:22]
Michail Epikhin
I build and use musqld-nt binary, but in the test builds that Ignacio made I cant find this (mysqld only). Is mysqld binary work correctly on Win2003 x64 R2? I can start testing tomorrow.
[10 Nov 2006 16:56]
Iggy Galarza
Michail, Thank you very much for your continued assistance with this issue. I created the mysqld.exe 5.0 using Visual Studio 2005 running on Windows XP Pro 64 and did my testing there. The mysql-5.0 64-bit Windows solution does not contain an NT configuration. Instead, one of the configuration options that can be used by CMake is __NT__. I verified that all of the binaries I made set __NT__ = TRUE. (For more information on how the mysql-5.0 64-bit Windows binary is created, please see the file win/README in the mysql-5.1-beta source distribution.) The binaries should behave properly on Windows 2003 x64. Cheers! ~Ignacio
[12 Nov 2006 11:02]
Michail Epikhin
I run my crash-recovery test with you test binary on my database server. Test: I have crashed mysql database, that correctly recover with binary without bug. Results: 1. Release configuration built and packaged with debug symbols: 061112 12:38:43 InnoDB: Started; log sequence number 304 4135208696 061112 12:38:44 [ERROR] Can't open shared library 'myudf.dll' (errno: 0 ) 061112 12:38:44 [Note] C:\MySQLTest\bin\mysqld: ready for connections. Version: '5.0.29-debug' socket: '' port: 3306 Source distribution 061112 12:41:09 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. ... InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 InnoDB: Error: Memory area size 1024, next area size 0 not a power of 2! InnoDB: Possibly a memory overrun of the buffer being freed here. InnoDB: Apparent memory corruption: mem dump len 500; hex 000000000000000000000000000000000000e874470a000000004882470a00000000000000000000000000000000000000000400000000000000560100000000000000000000000000000000000000000000d80000000000000000000000000000000000000000000000700000000000000070000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000400000000000000000000000000006054470a000000009268a5200000000074306d656d2e6300d100000000000000000000000000000000000000000000000000000000000000782f470a000000000000000000000000200200000000000000000000000000000000000000000000b801000000000000700000000000000000000000000000004c4f475f44554d4d5900470a00000000782f470a000000000000000000000000b851470a00000000f07d980000000000e83c470a00000000000000000000000000000000000000001400000000000000020000000000000002000000000000000200000000000000c851470a000000000100; asc иtG H‚G V Ш p p `TG ’hҐ t0mem.c С x/G ё p LOG_DUMMY G x/G ёQG р} и<G ИQG ; InnoDB: Scanning backward trying to find previous allocated mem blocks Mem block at - 2024, file t0mem.c, line 47 Mem block at - 4072, file t0mem.c, line 47 Mem block at - 5096, file t0mem.c, line 47 Mem block at - 6120, file t0mem.c, line 47 Mem block at - 8168, file purge.c, line 177 Mem block at - 8424, file t0mem.c, line 209 Mem block at - 8680, file t0mem.c, line 47 Mem block at - 9192, file purge.c, line 49 Mem block at - 10216, file purge.c, line 177 Mem block at - 12264, file x0trx.c, line 108 InnoDB: Scanning forward trying to find next allocated mem blocks Freed mem block at + 24, file t0mem.c, line 209 Freed mem block at + 1048, file t0mem.c, line 209 Freed mem block at + 1304, file t0mem.c, line 1 Mem block at + 2072, file t0mem.c, line 209 Mem block at + 4120, file t0mem.c, line 47 Freed mem block at + 8216, file t0mem.c, line 47 Freed mem block at + 9240, file t0mem.c, line 47 Freed mem block at + 10264, file t0mem.c, line 47 Mem block at + 12312, file t0mem.c, line 47 Freed mem block at + 16408, file t0mem.c, line 47 061112 12:41:18InnoDB: Assertion failure in thread 2700 in file .\mem\mem0pool.c line 505 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs. Crash. 2. Release configuration built without compiler optimizations for innobase/mem/* and packaged with debug symbols: 061112 12:51:16 InnoDB: Database was not shut down normally! ... 061112 12:51:57 InnoDB: Rollback of non-prepared transactions completed Start database. 3. Release configuration with UNIV_MUST_NOT_INLINE defined in univ.i and packaged with debug symbols: 061112 13:26:11 InnoDB: Database was not shut down normally! ... InnoDB: 1 transaction(s) which must be rolled back or cleaned up InnoDB: in total 159003 row operations to undo InnoDB: Trx id counter is 0 9574656 061112 13:26:46 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 0 1 2 InnoDB: Error: Memory area size 2048, next area size 0 not a power of 2! InnoDB: Possibly a memory overrun of the buffer being freed here. InnoDB: Apparent memory corruption: mem dump len 500; hex 000004000000000000005201000000000000206f430a000000004882430a00000000000000000000000000000000000000000400000000000000560100000000000000000000000000000000000000000000d800000000000000000000000000000000000000000000007000000000000000700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000009268a5200000000074306d656d2e6300d1000000000000000000000000000000000000000000000000000000000000007850430a000000000000000000000000b00400000000000000000000000000000000000000000000980100000000000070000000000000000000000000000000f840430a000000005045430a0000000000000000000000000000000000000000040000000000000000000000000000006041430a000000005845430a000000000000000000000000000000000000000008000000000000000400000000000000c841430a000000006045430a000000000000; asc R oC H‚C V Ш p p ’hҐ t0mem.c С xPC ° p ш@C PEC `AC XEC ИAC `EC ; InnoDB: Scanning backward trying to fin Crash. If necessary, I can run other you test, but I can’t prolong run any of test builds on my database server.
[13 Nov 2006 11:58]
Magnus Blåudd
Very very nice. Seems like you almost got a reproducible test case for this problem. Do you think it's possible to send us the script that makes this problem occur? Also one question about the tests that is performed - is the data that causes the crash generated by the defect binary? What I mean is do you think the problem is in recovering or are the corrupt logs produced by this problem?
[13 Nov 2006 12:57]
Heikki Tuuri
http://bugs.mysql.com/bug.php?id=24207 is probably a duplicate of this, and the user can repeat the crash with a simple sql-bench run.
[24 Nov 2006 6:04]
Michail Epikhin
How to repeat crach: Using good binaries (like 32 bit) create table: CREATE TABLE `farm`.`synonym` ( `SynonymCode` int(11) unsigned NOT NULL auto_increment, `FirmCode` int(11) unsigned NOT NULL default '0', `Synonym` char(255) default NULL, `FullCode` int(11) unsigned default NULL, `ShortCode` int(11) unsigned default NULL, `Code` char(20) default NULL, `Junk` char(20) NOT NULL, `Date` datetime default NULL, `UserName` char(40) default NULL, PRIMARY KEY (`SynonymCode`), KEY `ShortCode_IDX` (`ShortCode`), KEY `FullCode_IDX` (`FullCode`), KEY `Synonym_IDX` (`Synonym`), KEY `Code_IDX` (`Code`), KEY `FirmCode_IDX` (`FirmCode`)) ENGINE=InnoDB DEFAULT CHARSET=cp1251 PACK_KEYS=0 DELAY_KEY_WRITE=1 ROW_FORMAT=FIXED; Insert into table ~100000 rows. Start good binary, start READ-COMMITTED transaction, delete ~30000 rows, insert ~1000 rows, the kill mysqld. Start defect binary - server crach on recovering. After crash, start good binary - database recover OK.
[24 Nov 2006 9:02]
Magnus Blåudd
Thank you very much!
[1 Dec 2006 14:47]
Heikki Tuuri
http://bugs.mysql.com/bug.php?id=24247, which is VERY easy to repeat might be the same bug as this.
[14 Dec 2006 17:17]
Iggy Galarza
Michail, Thank you for the steps to reproduce the problem. I am unable to reproduce the behavior described.
Will you please review the following test plan and make suggestions?
1. Create mysql-5.0 win32 Release build and start mysqld as a process (mysql-test-run.pl --start-and-exit)
2. Execute the following in the mysql client.
# Create the db and table.
CREATE DATABASE farm;
USE farm;
CREATE TABLE `farm`.`synonym` (
`SynonymCode` int(11) unsigned NOT NULL auto_increment,
`FirmCode` int(11) unsigned NOT NULL default '0',
`Synonym` char(255) default NULL,
`FullCode` int(11) unsigned default NULL,
`ShortCode` int(11) unsigned default NULL,
`Code` char(20) default NULL,
`Junk` char(20) NOT NULL,
`Date` datetime default NULL,
`UserName` char(40) default NULL,
PRIMARY KEY (`SynonymCode`),
KEY `ShortCode_IDX` (`ShortCode`),
KEY `FullCode_IDX` (`FullCode`),
KEY `Synonym_IDX` (`Synonym`),
KEY `Code_IDX` (`Code`),
KEY `FirmCode_IDX` (`FirmCode`))
ENGINE=InnoDB DEFAULT CHARSET=cp1251 PACK_KEYS=0 DELAY_KEY_WRITE=1 ROW_FORMAT=FIXED;
#Create some procedures for filling the table.
delimiter //
CREATE PROCEDURE fill_synonym()
BEGIN
DECLARE v1 INT DEFAULT 1;
WHILE v1 < 100001 DO
INSERT INTO synonym(FirmCode, Synonym, FullCode, ShortCode, Code, Junk, Date, UserName)
VALUES(v1, CONCAT('synonym',CAST(v1 AS char)), v1, v1,
CONCAT('code',CAST(v1 AS char)), CONCAT('junk',CAST(v1 AS char)), now(),
CONCAT('user',CAST(v1 AS char)));
SET v1 = v1 + 1;
END WHILE;
END
//
CREATE PROCEDURE fill_syn_small()
BEGIN
DECLARE v1 INT DEFAULT 100001;
WHILE v1 < 101001 DO
INSERT INTO synonym(FirmCode, Synonym, FullCode, ShortCode, Code, Junk, Date, UserName)
VALUES(v1, CONCAT('synonym',CAST(v1 AS char)), v1, v1,
CONCAT('code',CAST(v1 AS char)), CONCAT('junk',CAST(v1 AS char)), now(),
CONCAT('user',CAST(v1 AS char)));
SET v1 = v1 + 1;
END WHILE;
END
//
delimiter ;
# Fill the table with 100,000 rows.
CALL fill_synonym();
3. Close the mysql client. Shutdown the server (mysqladmin.exe --user=root -P 9306 status; mysqladmin.exe --user=root -P 9306 shutdown;)
4. Restart the mysqld process. (mysql-test-run.pl --start-and-exit --start-dirty)
5. Execute the following in the mysql client.
# Create the test conditions.
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
START TRANSACTION;
# Delete some data.
DELETE FROM synonym WHERE SynonymCode > 14999 AND SynonymCode < 20001;
DELETE FROM synonym WHERE SynonymCode > 29999 AND SynonymCode < 35001;
DELETE FROM synonym WHERE SynonymCode > 44999 AND SynonymCode < 50001;
DELETE FROM synonym WHERE SynonymCode > 59999 AND SynonymCode < 65001;
DELETE FROM synonym WHERE SynonymCode > 74999 AND SynonymCode < 80001;
DELETE FROM synonym WHERE SynonymCode > 89999 AND SynonymCode < 95001;
# Insert 1000 rows.
CALL fill_syn_small();
6. While the mysql client is still running and before committing the transaction, kill the mysqld process.
7. Build mysql-5.0 win64 Release build.
8. Start the mysqld server. (mysql-test-run.pl --start-and-exit --start-dirty)
[14 Dec 2006 17:44]
Iggy Galarza
All InnoDB compiler optimizations have been removed from mysql-5.0 Win64 since 5.0.28. The preceding changeset is not a final fix, it only re-enables compiler optimizations on all except innodb/mem/* based on customer recommendation.
[22 Dec 2006 16:00]
MySQL Verification Team
heikki, iggy, can you say roughly what performance impact the above 'fix' will have on innodb? Has anyone done any benchmarks to check it?
[22 Dec 2006 17:31]
Heikki Tuuri
Shane, if the code optimization is only removed from mem0*, then the impact is very small. If it is removed from all InnoDB code, then mysqld would run maybe only at 50 % of the speed of a normal build. Regards, Heikki
[15 Jan 2007 15:00]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/18115 ChangeSet@1.2361, 2007-01-15 10:00:29-05:00, iggy@recycle.(none) +1 -0 Bug#19424 InnoDB: Possibly a memory overrun of the buffer being freed (Win64) - Re-enabling optimization on all except innodb/mem/*.
[15 Jan 2007 15:30]
Iggy Galarza
This bug deserves a status update at this point. The previous changeset (made to 5.0.28) removed the compiler optimizations from innobase/* which can lead to poor performance. The latest changeset removes Win64 compiler options for innobase/mem/* in mysql-5.0 only. I have not been able to reproduce the problem as described (noted my testing efforts above). This latest change was made due to the testing done by Michail Epikhin, where he confirms that the problem he can reproduce is resolved by removing the compiler optimizations. This is at best a stop gap effort. This bug cannot have a resolution until the problem can be readily reproduced.
[17 Jan 2007 14:43]
Heikki Tuuri
Iggy, I think that if the removal of compiler optimizations in /innobase/mem* resolves this, then we can mark the bug as closed. The performance hit from disabling optimizations there is small. If this is a compiler bug, finding the bug can require too much work. If the bug reappears, then we start to study this once again. But we still need to wait for a few months, so that we get feedback from users. I am putting this to the 'Need feedback' status. We wait for feedback from any InnoDB user on Win64. Regards, Heikki
[18 Feb 2007 0:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[21 Feb 2007 7:51]
MySQL Verification Team
using a self-built 5.0.36BK snapshot, with and without optimizations in innobase/mem i couldn't repeat this memory corruption after 24 hours of intensive tests, using 3GB buffer pool size. Did anybody else repeat the corruptions recently?
[21 Feb 2007 12:54]
Heikki Tuuri
Shane, what 64-bit Visual C++ compiler version did you use? If this is a compiler bug, then Microsoft might have fixed it lately. I have not heard anyone reporting this memory corruption bug in the past 2 months. Regards, Heikki
[21 Feb 2007 13:06]
MySQL Verification Team
Heikki, I used Visual Studio 2005, 8.0.50727.42. We shall have to wait for any comments from users/customers. No news is good news.
[20 Mar 2007 0:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[20 Mar 2007 13:55]
Heikki Tuuri
Optimistically, I am closing this bug report. Removing compiler optimizations from /innobase/mem probably fixed this. It may also be that Microsoft has fixed the latest Visual C++ compiler.
[3 May 2007 13:43]
MySQL Verification Team
Heikki, Iggy, In which released versions are *all* innodb optimizations removed? And which released versions have just the mem/* optimizations removed?
[3 May 2007 15:24]
Iggy Galarza
innobase/* opts were removed in 5.0.28 innobase/* opts were re-enabled except for innobase/mem/* in 5.0.34
[10 Feb 2008 19:00]
Louis Breda van
Hello, Please fix this bug soon, since it is a show stopper. The proposed fix "to remove compiler optimizations in the /innobase/mem directory" would hopefully help :> However, this is a quick workaround. Please haver the compiler fixed!! Sincerely, Louis
[10 Feb 2008 19:00]
Louis Breda van
Hello, Please fix this bug soon, since it is a show stopper. The proposed fix "to remove compiler optimizations in the /innobase/mem directory" would hopefully help :> However, this is a quick workaround. Please have the compiler fixed!! Sincerely, Louis
[10 Feb 2008 19:00]
Louis Breda van
Hello, Please fix this bug soon, since it is a show stopper. The proposed fix "to remove compiler optimizations in the /innobase/mem directory" would hopefully help :> However, this is a quick workaround. Please have the compiler fixed!! Sincerely, Louis
[11 Feb 2008 16:23]
Timothy Smith
Louis, This bug is closed; the optimizations have already been disabled in innobase/CMakeLists.txt for mem/*. If you're experiencing similar symptoms, please post to forums.mysql.com or open a MySQL support issue to get help in diagnosing it. If you have a repeatable test case showing a problem, please open a new bug report giving full details (exactly what version of MySQL and operating system, exactly what commands are run in what sequence, and the full output which shows the problem you're having). Regards, Timothy
[26 Jan 2009 16:58]
Vasil Dimov
Setting state to "Documenting" because the fix for this has already been committed in: ------------------------------------------------------------ revno: 1810.2473.1 committer: iggy@recycle.(none) timestamp: Mon 2007-01-15 10:00:29 -0500 message: Bug#19424 InnoDB: Possibly a memory overrun of the buffer being freed (Win64) - Re-enabling optimization on all except innodb/mem/*. and ------------------------------------------------------------ revno: 2617 committer: tsmith@ramayana.hindu.god timestamp: Fri 2008-05-09 00:38:17 -0600 message: Bug #34297: MySQL Server crashes when processing large table Remove optimizations on innobase/mem/* to avoid apparent compiler bug which causes memory overruns. See also bug 19424, and probably bug 36366. This is done in 5.1+; 5.0 already has this workaround in place.
[26 Jan 2009 19:30]
Paul DuBois
Noted in 5.0.34 changelog. InnoDB: Optimizations removed in MySQL 5.0.28 were re-enabled except for files under the innobase/mem directory. (This is a fine-tuning of optimization disabling.)
[5 May 2010 15:02]
Bugs System
Pushed into 5.1.47 (revid:joro@sun.com-20100505145753-ivlt4hclbrjy8eye) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[6 May 2010 2:06]
Paul DuBois
Push resulted from incorporation of InnoDB tree. No changes pertinent to this bug. Re-closing.
[28 May 2010 5:57]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100524190136-egaq7e8zgkwb9aqi) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (pib:16)
[28 May 2010 6:26]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100524190941-nuudpx60if25wsvx) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[28 May 2010 6:54]
Bugs System
Pushed into 5.5.5-m3 (revid:alik@sun.com-20100524185725-c8k5q7v60i5nix3t) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[29 May 2010 2:47]
Paul DuBois
Push resulted from incorporation of InnoDB tree. No changes pertinent to this bug. Re-closing.
[15 Jun 2010 8:08]
Bugs System
Pushed into 5.5.5-m3 (revid:alik@sun.com-20100615080459-smuswd9ooeywcxuc) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (merge vers: 5.1.47) (pib:16)
[15 Jun 2010 8:23]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100615080558-cw01bzdqr1bdmmec) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (pib:16)
[17 Jun 2010 12:00]
Bugs System
Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[17 Jun 2010 12:41]
Bugs System
Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[17 Jun 2010 13:27]
Bugs System
Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
Description: 060426 13:58:09 [Note] mysqld: ready for connections. Version: '5.0.19-log' socket: '' port: 3306 Source distribution InnoDB: Error: Memory area size 256, next area size 0 not a power of 2! InnoDB: Possibly a memory overrun of the buffer being freed here. InnoDB: Apparent memory corruption: mem dump len 500; hex 000000000000000000006 0653c0600000000b307952d0000000072306c6f632e6300a80000000000000001000000000000007 8643c060000000078643c060000000000000000000000000000000000000000a0000000000000000 0000000000000000000000000000000a00000000000000070000000000000000000000000000000f c06000000000000feffffffffffffff0a00008000000000000000000000000000000000000000008 2c91200000000001c0000000000000027000000000000004b000000000000007f000000000000007 c00000000000000820000000000000083000000000000008b000000000000008f000000000000000 001000000000000000000000000000000000000000000009268a52000000000773073656c2e6300b b0900000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000e80000000000000000000000000000000000000000000000e8000000000000007 0000000000000000000000000000000636f6e74656e7455726c3d73746f72653a2f2f323030362f3 42f32362f31342f38386633363262312d643532632d313164612d616563662d66393339626665373 03666372e62696e7c6d696d65747970653d746578742f706c61696e7c73697a653d333139317c656 e636f64696e673d5554; asc `e< ³ - r0loc.c ¨ xd< xd< p ü þÿÿÿÿÿÿÿ 'É ' K | ' ƒ < 'h¥ w0sel.c » è è p contentUrl=store://2006/4/26/14/88f362b1-d52c-11da-aecf-f939bfe706 f7.bin|mimetype=text/plain|size=3191|encoding=UT; InnoDB: Scanning backward trying to find previous allocated mem blocks Mem block at - 232, file r0loc.c, line 168 Mem block at - 744, file 0pcur.c, line 29 Mem block at - 1256, file w0upd.c, line 289 Freed mem block at - 3304, file 0pcur.c, line 29 Freed mem block at - 4328, file w0ins.c, line 96 Freed mem block at - 4840, file w0upd.c, line 289 Mem block at - 5352, file w0ins.c, line 96 Mem block at - 7400, file w0ins.c, line 96 Mem block at - 9448, file odb.cpp, line 3189 Freed mem block at - 11496, file 0data.c, line 150 InnoDB: Scanning forward trying to find next allocated mem blocks Freed mem block at + 24, file w0sel.c, line 2491 Freed mem block at + 280, file e0cur.c, line 923 Freed mem block at + 792, file r0btr.c, line 1587 Freed mem block at + 1304, file r0sea.c, line 1164 Freed mem block at + 1816, file 0data.c, line 150 Mem block at + 2840, file x0trx.c, line 176 Mem block at + 6936, file mysql.c, line 592 Freed mem block at + 7960, file 0data.c, line 150 Freed mem block at + 10008, file 0page.c, line 369 Freed mem block at + 10776, file odb.cpp, line 3189 060426 15:49:48InnoDB: Assertion failure in thread 316 in file .\mem\mem0pool.c line 505 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html InnoDB: about forcing recovery. InnoDB: Thread 1424 stopped in file .\row\row0mysql.c line 169 InnoDB: Thread 2312 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111 InnoDB: Thread 2632 stopped in file .\os\os0sync.c line 487 InnoDB: Thread 2128 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111 D:\MySQL Server 5.0\bin>mysqld --console 060426 16:03:25 [Warning] Changed limits: max_open_files: 2048 max_connections: 100 table_cache: 969 060426 16:03:26 InnoDB: Started; log sequence number 2 1929811830 060426 16:03:27 [Note] mysqld: ready for connections. Version: '5.0.19-log' socket: '' port: 3306 Source distribution InnoDB: Error: Memory area size 1024, next area size 0 not a power of 2! InnoDB: Possibly a memory overrun of the buffer being freed here. InnoDB: Apparent memory corruption: mem dump len 500; hex 64612d383737372d65643 46239356330313135620900c1a8a80b5c3a362d643662322d313164612d383737372d65643462393 56330313135620900c1a8a80bbd3a382d643662322d313164612d383737372d65643462393563303 13135620900c1a81bc30537612d643662322d313164612d383737372d65643462393563303131356 20900c1a8a80c7f90573f0600000000e8603f0600000000372d6564346239356330313135620900c 1a8a80ce03a652d44554d4d5900316444554d4d59002d6544554d4d5900303131356231313562626 163657353746f7265776f726b737061636580000000002889ea6e7480b8362d643632662d3131640 004000000000000000000000000000060dc3c06000000009268a5200000000030646174612e63009 60000000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000680200000000000000000000000000000000000000000000a8010000000000007 00000000000000000000000000000000000000000000000030000000000000003000000000000002 05d3f0600000000e8603f0600000000000000000000000000000000000000000aae8950000000000 9000000000000000c000000000000000f01210000000000960000000000000000000000000000000 1000000000000000300; asc da-8777-ed4b95c0115b Á¨¨ \:6-d6b2-11da-8777-ed4b95c0 115b Á¨¨ ½:8-d6b2-11da-8777-ed4b95c0115b Á¨ à 7a-d6b2-11da-8777-ed4b95c0 115b Á¨¨ W? è`? 7-ed4b95c0115b Á¨¨ à:e-DUMMY 1dDUMMY -eDUMMY 0 115b115bbacesStoreworkspace (%ênt ¸6-d62f-11d `Ü< 'h¥ 0data.c - h ¨ p ]? è`? ®%P ! - ; InnoDB: Scanning backward trying to find previous allocated mem blocks Mem block at - 1000, file w0sel.c, line 2934 Mem block at - 2024, file w0sel.c, line 2934 Mem block at - 3048, file w0sel.c, line 2934 Mem block at - 4072, file w0sel.c, line 2934 Mem block at - 5096, file w0sel.c, line 2934 Mem block at - 6120, file w0sel.c, line 2934 Mem block at - 7144, file w0sel.c, line 2934 Freed mem block at - 8168, file t0mem.c, line 47 Freed mem block at - 9192, file t0mem.c, line 194 Freed mem block at - 10216, file DUMMY, line 1296913732 InnoDB: Scanning forward trying to find next allocated mem blocks Freed mem block at + 24, file 0data.c, line 150 Freed mem block at + 1048, file 0ibuf.c, line 3102 Freed mem block at + 4120, file DUMMY, line 1296913732 Freed mem block at + 5144, file t0mem.c, line 194 Freed mem block at + 6168, file n0dyn.c, line 33 Freed mem block at + 7192, file n0dyn.c, line 33 Freed mem block at + 8216, file n0dyn.c, line 33 Freed mem block at + 9240, file n0dyn.c, line 33 Freed mem block at + 13336, file n0dyn.c, line 33 Freed mem block at + 16664, file e0cur.c, line 923 060428 13:31:26InnoDB: Assertion failure in thread 2996 in file .\mem\mem0pool.c line 505 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html InnoDB: about forcing recovery. InnoDB: Thread 2784 stopped in file .\os\os0sync.c line 309 InnoDB: Thread 3076 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111 InnoDB: Thread 4004 stopped in file .\os\os0sync.c line 487 InnoDB: Thread 1552 stopped in file c:\cygwin\home\mysqldev\build\mysql-5.0.19-b uild-x64\mysql-5.0.19\innobase\include\sync0sync.ic line 111 InnoDB: Thread 3660 stopped in file .\mem\mem0mem.c line 290 How to repeat: This has not been repeatable with the test I have at the moment. It has happened at least 5 times after various times.