Bug #53898 --with-debug causes glibc to dump 'double free or corruption (fasttop)' messages
Submitted: 21 May 2010 18:56 Modified: 28 May 2010 5:56
Reporter: Andreas Mock Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Server: Tests Severity:S7 (Test Cases)
Version:5.1.47 OS:Linux (OpenSuSE 11.2)
Assigned to: Assigned Account CPU Architecture:Any
Tags: corruption, glibc, regression, test

[21 May 2010 18:56] Andreas Mock
Description:
configured (./configure --with-debug --prefix=/opt/mysql --with-extra-charsets=all --enable-thread-safe-client --enable-local-infile --enable-assembler --with-readline --without-libedit --with-mysqld-user=mysql --with-plugins=partition,federated,heap,innobase,innodb_plugin,myisam,myisammrg), compiled (make) and tested (make test) the last versions (mysql-5.1.44  mysql-5.1.45  mysql-5.1.46) on OpenSuSE 11.2 64bit without problems.

With version mysql-5.1.47 I do get glibc dumps of the following kind when doing 'make test':
--------------8<--------------
*** glibc detected *** /opt/data/tmp/mysql-5.1.47/sql/mysqld: double free or corruption (!prev): 0x087eaad8 ***
======= Backtrace: =========
/lib/libc.so.6[0xb747850b]
/lib/libc.so.6[0xb7479de4]
/lib/libc.so.6(cfree+0x6d)[0xb747cfdd]
/lib/libc.so.6(fclose+0x154)[0xb74680e4]
/opt/data/tmp/mysql-5.1.47/sql/mysqld[0x84f1401]
/opt/data/tmp/mysql-5.1.47/sql/mysqld[0x84f1520]
/opt/data/tmp/mysql-5.1.47/sql/mysqld(my_end+0x185)[0x84c9a65]
/opt/data/tmp/mysql-5.1.47/sql/mysqld(unireg_abort+0x1a2)[0x81eaca9]
/opt/data/tmp/mysql-5.1.47/sql/mysqld(main+0xab4)[0x81f0f3d]
/lib/libc.so.6(__libc_start_main+0xfe)[0xb7421ace]
/opt/data/tmp/mysql-5.1.47/sql/mysqld[0x812f781]
======= Memory map: ========
08048000-08774000 r-xp 00000000 fd:00 142909     /opt/data/tmp/mysql-5.1.47/sql/mysqld
08774000-08775000 r--p 0072b000 fd:00 142909     /opt/data/tmp/mysql-5.1.47/sql/mysqld
08775000-087d4000 rw-p 0072c000 fd:00 142909     /opt/data/tmp/mysql-5.1.47/sql/mysqld
087d4000-08880000 rw-p 00000000 00:00 0          [heap]
b6392000-b6393000 ---p 00000000 00:00 0
b6393000-b63c3000 rw-p 00000000 00:00 0
b63c3000-b63c4000 ---p 00000000 00:00 0
b63c4000-b63f4000 rw-p 00000000 00:00 0
b63f4000-b63ff000 r-xp 00000000 fd:00 11524      /lib/libnss_files-2.10.1.so
b63ff000-b6400000 r--p 0000a000 fd:00 11524      /lib/libnss_files-2.10.1.so
b6400000-b6401000 rw-p 0000b000 fd:00 11524      /lib/libnss_files-2.10.1.so
b6c08000-b6c09000 ---p 00000000 00:00 0
b6c09000-b740b000 rw-p 00000000 00:00 0
b740b000-b7564000 r-xp 00000000 fd:00 11507      /lib/libc-2.10.1.so
b7564000-b7565000 ---p 00159000 fd:00 11507      /lib/libc-2.10.1.so
b7565000-b7567000 r--p 00159000 fd:00 11507      /lib/libc-2.10.1.so
b7567000-b7568000 rw-p 0015b000 fd:00 11507      /lib/libc-2.10.1.so
b7568000-b756b000 rw-p 00000000 00:00 0
b756b000-b7589000 r-xp 00000000 fd:00 20582      /lib/libgcc_s.so.1
b7589000-b758a000 r--p 0001d000 fd:00 20582      /lib/libgcc_s.so.1
b758a000-b758b000 rw-p 0001e000 fd:00 20582      /lib/libgcc_s.so.1
b758b000-b75b2000 r-xp 00000000 fd:00 11515      /lib/libm-2.10.1.so
b75b2000-b75b3000 r--p 00027000 fd:00 11515      /lib/libm-2.10.1.so
b75b3000-b75b4000 rw-p 00028000 fd:00 11515      /lib/libm-2.10.1.so
b75b4000-b769b000 r-xp 00000000 fd:00 129724     /usr/lib/libstdc++.so.6.0.12
b769b000-b769c000 ---p 000e7000 fd:00 129724     /usr/lib/libstdc++.so.6.0.12
b769c000-b76a0000 r--p 000e7000 fd:00 129724     /usr/lib/libstdc++.so.6.0.12
b76a0000-b76a1000 rw-p 000eb000 fd:00 129724     /usr/lib/libstdc++.so.6.0.12
b76a1000-b76a9000 rw-p 00000000 00:00 0
b76a9000-b76be000 r-xp 00000000 fd:00 11518      /lib/libnsl-2.10.1.so
b76be000-b76bf000 r--p 00014000 fd:00 11518      /lib/libnsl-2.10.1.so
b76bf000-b76c0000 rw-p 00015000 fd:00 11518      /lib/libnsl-2.10.1.so
b76c0000-b76c2000 rw-p 00000000 00:00 0
b76c2000-b76cf000 r-xp 00000000 fd:00 11511      /lib/libcrypt-2.10.1.so
b76cf000-b76d0000 r--p 0000c000 fd:00 11511      /lib/libcrypt-2.10.1.so
b76d0000-b76d1000 rw-p 0000d000 fd:00 11511      /lib/libcrypt-2.10.1.so
b76d1000-b76f8000 rw-p 00000000 00:00 0
b76f8000-b76fb000 r-xp 00000000 fd:00 11513      /lib/libdl-2.10.1.so
b76fb000-b76fc000 r--p 00002000 fd:00 11513      /lib/libdl-2.10.1.so
b76fc000-b76fd000 rw-p 00003000 fd:00 11513      /lib/libdl-2.10.1.so
b76fd000-b7713000 r-xp 00000000 fd:00 11533      /lib/libpthread-2.10.1.so
b7713000-b7714000 r--p 00015000 fd:00 11533      /lib/libpthread-2.10.1.so
b7714000-b7715000 rw-p 00016000 fd:00 11533      /lib/libpthread-2.10.1.so
b7715000-b7717000 rw-p 00000000 00:00 0
b771c000-b771d000 rw-p 00000000 00:00 0
b771e000-b771f000 rw-p 00000000 00:00 0
b771f000-b773d000 r-xp 00000000 fd:00 11500      /lib/ld-2.10.1.so
b773d000-b773e000 r--p 0001d000 fd:00 11500      /lib/ld-2.10.1.so
b773e000-b773f000 rw-p 0001e000 fd:00 11500      /lib/ld-2.10.1.so
bf8e9000-bf8fe000 rw-p 00000000 00:00 0          [stack]
ffffe000-fffff000 r-xp 00000000 00:00 0          [vdso]
--------------8<--------------

I could identify some tests causing this dump:

binlog.binlog_incident
main.flush_read_lock_kill
binlog.binlog_killed_simulate
rpl.rpl_killed_ddl
rpl.rpl_incident
rpl.rpl_known_bugs_detection
rpl.rpl_loaddata_fatal
rpl.rpl_temporary_errors

Sometimes it happens that the bootstrap of one test fails with the upper kind of dump and the test fails completely.

I can't judge how severe the bug is. But as sometimes the test framework is not able to start and bootstrap one test because starting of mysql fails with such a glibc dump, IMHO it's really not unsevere.

How to repeat:
- Download sources to an OpenSuSE 11.2 machine.
- Install necessary libraries and tools
- Untar sources
- Configure with configure statement given above
- make
- make test

I installed OpenSuSE 11.2 32bit extra for this in a virtual machine.
The same errors appear.

I did the same procedure on OpenSuSE 11.1 64bit.
The same errors occur.
[21 May 2010 19:04] MySQL Verification Team
Thank you for the bug report. Probably duplicate of?

http://bugs.mysql.com/bug.php?id=53794

Thanks in advance.
[21 May 2010 19:35] Andreas Mock
Hi Miguel,

I found that one too, but two reasons caused me to file an extra bug.

a) I thought Ingo was talking about current developer branches (5.1.48) and not about a released (and as stable assumed) version. (5.1.47)
b) His bug report's severity was S3, but as I said: Sometimes even mysql doesn't come up to bootstrap a test with exactly this glibc dump. Memory corruption and memory leaks are IMHO bad beasts and if you get signs of that someone should investigate this properly.

So I hope I didn't annoyed with this bug report.

Best regards
Andreas
[24 May 2010 16:56] Ryan Mack
I posted my take on bug 53794.

Summary: Quick fix is to comment out this line in _db_end_():

dbug.c:1521: FreeState (cs, discard, 0);

I wouldn't swear that's the correct thing to do but it works for me.

Thanks, Ryan
[25 May 2010 8:48] Sveta Smirnova
Thank you for the report.

Verified as described.
[25 May 2010 8:49] Sveta Smirnova
Not repeatable with next-mr
[27 May 2010 22:21] Omer Barnir
triage: setting to SR51MRU (same as bug#52884) - might be a duplicate
[28 May 2010 5:56] Sergei Glukhov
duplicated with Bug#52884