MySQL Bugs: #20337: crash during stress testing within transaction

Bug #20337	crash during stress testing within transaction
Submitted:	8 Jun 2006 10:50	Modified:	3 Aug 2006 21:27
Reporter:	Matthias Leich	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S1 (Critical)
Version:	5.1	OS:	Linux (Linux)
Assigned to:	Assigned Account	CPU Architecture:	Any

Description:
Crash in stress testsuite "system_2":
- Neither triggers nor stored procedures are used.
  (The monitoring thread does nothing, aka does not execute
   tolog.test ....)
- all three test tables use the storage engine InnoDB

Please have a look into the attached stacktrace and the general_log.CSV.

My environment:
   - PC(Pentium M, x86-32Bit) with Linux(SuSE 9.3)
   - MySQL compiled (compile-pentium-debug-max) from source
        Version 5.0 ChangeSet@1.2173, 2006-06-06

How to repeat:
Please 
- clone the internal QA tree 
   bk-internal.mysql.com:/home/bk/mysql-test-extra-5.1
- cd <test_extra>/mysql-test/suite/system_2
- adjust the attached configuration file (innodb2.env) to
   your configuration (paths to files ...)
   and remove the testcases from monitor.test
- run
       nice -20 ./run_systest2 --config=innodb2.env

stack trace

Attachment: stacktracex (application/octet-stream, text), 4.05 KiB.

log

Attachment: general_log.tgz (application/x-compressed-tar, text), 122.25 KiB.

stress test configuration file

Attachment: innodb2.env (application/octet-stream, text), 7.49 KiB.

Thank you for a problem report. Sorry, is it 5.0-BK or 5.1-BK that should be tested? 5.1 is in Version field, but you mentioned 5.0 in description.

Sorry, it's 5.1 BK

and ChangeSet@1.2179.1.1, 2006-06-05

Stress testsuite system_2

Attachment: system_2.tgz (application/x-compressed-tar, text), 46.24 KiB.

Matthias,

how quickly does it crash?

Is this repeatable?

Can you run inside gdb?

Regards,

Heikki

Hello Heikki,
- in average 20 % of all tests get a crash
- most or all of the crashes occur after a short runtime (< 120s) after
   the initialization of the test, but that causes for example more than 5000
   lines in var/master-data/mysql/general_log.CSV
- the backtraces of the crashes and also the "guilty" SQL statements vary
   Sometimes I see routine names containing "deadlock", sometimes not.
- It is to be expected that the mixup of testscripts and the random 
   computation of primary keys leads to masses of  duplicate primary key
   errors and deadlocks within transactions consisting of several data 
   modifying statements (affecting in maximum one record) or even 
   within single modifying statements (affecting in maximum more than
   one record).

Osku,

please study this. This could be related to the lock table corruption we saw a week ago (field index in record lock was 0x3) in a similar stress test of a 5.1 build.

Regards,

Heikki

Osku,

can you repeat the crash on our latest version of 5.1?

Regards,

Heikki

Matthias,

can you provide the complete test setting to us?

Regards,

Heikki

This may be related to other crash reports from multithreaded tests of 5.1.

These bug reports suggest that some serious bug was introduced in 5.1.11:

http://bugs.mysql.com/bug.php?id=20213
http://bugs.mysql.com/bug.php?id=20337
http://bugs.mysql.com/bug.php?id=20595
http://bugs.mysql.com/bug.php?id=21322

Can you test if 5.1.9 exhibits the same crashes?

Regards,

Heikki

Heikki,

I made a bunch of attempts to reproduce the problem with MySQL 5.1.9.
InnoDB worked perfect, no crashes or suspicious server responses.

Regards,

Matthias

Heikki,

I made a bunch of attempts to reproduce the problem with MySQL 5.1.9.
InnoDB worked perfect, no crashes or suspicious server responses.

Regards,

Matthias

Matthias,

thank you. It is the large patches between 5.1.9 and 5.1.11 that have broken MySQL/InnoDB. We are concentrating on reproducing
http://bugs.mysql.com/bug.php?id=20213
on our own computers.

Regards,

Heikki

Probably a duplicate of http://bugs.mysql.com/bug.php?id=20213