| Bug #36792 | Assertion table->in_use == current_thd in Field_string::store, line 6259 | ||
|---|---|---|---|
| Submitted: | 19 May 2008 6:51 | Modified: | 29 Jul 2008 11:41 |
| Reporter: | Philip Stoev | Email Updates: | |
| Status: | Duplicate | Impact on me: | |
| Category: | MySQL Server: Backup | Severity: | S1 (Critical) |
| Version: | 6.0-bk | OS: | Any |
| Assigned to: | Assigned Account | CPU Architecture: | Any |
[19 May 2008 9:43]
Philip Stoev
Test case for bug 36792
Attachment: bug36792.zip (application/x-zip-compressed, text), 1.18 KiB.
[19 May 2008 9:48]
Philip Stoev
To reproduce the bug, please unpack the attached archive so that the .txt files are in mysql-test/ and the .test files are in mysql-test/t and then run: $ perl ./mysql-test-run.pl --stress --stress-init-file=bug36792_init.txt --stress-test-file=bug36792_run.txt --stress-test-duration=120 --stress-threads=2 --skip-ndb --mysqld=--skip-innodb The crash will happen immediately. Partitioning + Falcon is required to reproduce the bug. Innodb and non-partitioned tables are not affected.
[25 Jul 2008 9:13]
Øystein Grøvlen
I suspect that this error is related to attempting to do two backups in parallel. master.err contains the following:
080724 14:22:10 [Note] Backup: Starting backup process
080724 14:22:10 [Note] Backup: Backing up selected databases
080724 14:22:10 [Note] Backup: Starting backup process
080724 14:22:10 [ERROR] Backup: Can't execute this command because another BACKUP/RESTORE operation is in progress
080724 14:22:10 [Note] Backup: Backup completed
Probably the thread for the second backup is not terminated in the appropriate way. It seems the assert failure occurs for a later connection. The full list of operation performed by the stress test before failure is:
Time Id Command Argument
080724 14:22:09 1 Connect root@localhost on test
1 Query CREATE TABLE inter1 (
t1_autoinc INTEGER NOT NULL AUTO_INCREMENT,
t1_uuid CHAR(36),
t2_uuid CHAR(36),
t1_blob MEDIUMTEXT,
PRIMARY KEY (t1_autoinc),
KEY (t1_uuid),
KEY (t2_uuid)
) ENGINE = Falcon
1 Query ALTER TABLE inter1 PARTITION BY HASH(t1_autoinc)
080724 14:22:10 1 Query SHOW WARNINGS
1 Quit
2 Connect root@localhost on test
2 Query SELECT CONCAT('backup', CONNECTION_ID())
2 Query BACKUP DATABASE test TO "/home/og136792/mysql/shared/mysql-6.0-backup-clean/mysql-test/var/backup2"
3 Connect root@localhost on test
3 Query SELECT CONCAT('backup', CONNECTION_ID())
3 Query BACKUP DATABASE test TO "/home/og136792/mysql/shared/mysql-6.0-backup-clean/mysql-test/var/backup3"
3 Query SHOW WARNINGS
3 Quit
080724 14:22:11 4 Connect root@localhost on test
4 Query SET @uuid = CONVERT(UUID() USING latin1)
4 Query INSERT INTO inter1 (t1_uuid) VALUES (@uuid)
4 Query UPDATE inter1 SET t1_blob = 'a' WHERE inter1.t1_uuid = @uuid
4 Query SHOW WARNINGS
4 Quit
[25 Jul 2008 14:54]
Øystein Grøvlen
I have been able to reproduce the bug with the following test script. (Had to fix some other concurrency bugs before I got so far. More on that later.) Note that column types and partitioning seems to be significant in order to reproduce bug. ========================================================================= CREATE DATABASE backup_concurrent; USE backup_concurrent; #Create table. Make it partitioned --echo Creating Table CREATE TABLE inter1 ( t1_autoinc INTEGER NOT NULL AUTO_INCREMENT, t1_uuid CHAR(36), PRIMARY KEY (t1_autoinc) ) ENGINE = Falcon; ALTER TABLE inter1 PARTITION BY HASH(t1_autoinc); --echo Sending First Backup connect(backup1,localhost,root,,); send BACKUP DATABASE backup_concurrent TO 'backup1'; --echo Starting Second Backup, will fail since previous still active connection default; --error ER_BACKUP_RUNNING BACKUP DATABASE backup_concurrent TO 'backup2'; --echo Inserting Data INSERT INTO inter1 (t1_uuid) VALUES (1); --echo Starting third backup BACKUP DATABASE backup_concurrent TO 'backup3';
[28 Jul 2008 8:47]
Øystein Grøvlen
Turns out this has nothing to do with parallel backup commands. I was able to reproduce the assert with two sequential backups as follows: ================================================== CREATE DATABASE backup_concurrent; USE backup_concurrent; #Create table. Make it partitioned --echo Creating Table CREATE TABLE inter1 ( t1_autoinc INTEGER NOT NULL AUTO_INCREMENT, t1_uuid CHAR(36), PRIMARY KEY (t1_autoinc) ) ENGINE = Falcon; ALTER TABLE inter1 PARTITION BY HASH(t1_autoinc); --echo First Backup BACKUP DATABASE backup_concurrent TO 'backup1'; --echo Inserting Data INSERT INTO inter1 (t1_uuid) VALUES (1); --echo Starting second backup BACKUP DATABASE backup_concurrent TO 'backup2';
[29 Jul 2008 11:16]
Øystein Grøvlen
Turns out first backup is not necessary, either. The following test script provokes the assert: CREATE DATABASE backup_concurrent; USE backup_concurrent; #Create table. Make it partitioned --echo Creating Table CREATE TABLE inter1 ( t1_autoinc INTEGER NOT NULL AUTO_INCREMENT, t1_uuid CHAR(36), PRIMARY KEY (t1_autoinc) ) ENGINE = Falcon; ALTER TABLE inter1 PARTITION BY HASH(t1_autoinc); --echo Inserting Data INSERT INTO inter1 (t1_uuid) VALUES (1); --echo Starting Backup BACKUP DATABASE backup_concurrent TO 'backup1';
[29 Jul 2008 11:41]
Øystein Grøvlen
This is a duplicate of Bug#33566
[12 Aug 2008 13:53]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/51418 2681 Jorgen Loland 2008-08-12 Bug#35117 - Backup: Server crash for backup of CSV engine with CHAR data type Bug#33566 - Backup: crash with partitions and Falcon Bug#36792 - Assertion table->in_use == current_thd in Field_string::store, line 6259 Patch applies to all three bugs. Before: With default driver, the table->in_use variable is set to point to the locking thread when a table is locked. The variable points to the wrong thread (i.e., the locking thread) when backup later tries to get the data from the table. Now: Before getting data from a table, the default driver sets the table->in_use pointer to the backup thread.
[13 Aug 2008 6:15]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/51473 2681 Jorgen Loland 2008-08-13 Bug#35117 - Backup: Server crash for backup of CSV engine with CHAR data type Bug# 33566 - Backup: crash with partitions and Falcon Bug# 36792 - Assertion table->in_use == current_thd in Field_string::store, line 6259 Patch attached to 35117 but applies to all three bugs. Before: With default driver, the table->in_use variable is set to point to the locking thread when a table is locked. The variable points to the wrong thread (i.e., the locking thread) when backup later tries to get the data from the table. Now: Before getting data from a table, the default driver sets the table->in_use pointer to the backup thread.
[18 Aug 2008 8:23]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/51822 2675 Jorgen Loland 2008-08-18 Bug#35117 - Backup: Server crash for backup of CSV engine with CHAR data type Bug 33566 - Backup: crash with partitions and Falcon Bug 36792 - Assertion table->in_use == current_thd in Field_string::store, line 6259 Patch attached to 35117 but applies to all three bugs. Before: With default driver, the table->in_use variable is set to point to the locking thread when a table is locked. The variable points to the wrong thread (i.e., the locking thread) when backup later tries to get the data from the table. Now: Before getting data from a table, the default driver sets the table->in_use pointer to the backup thread.
Description: When executing a DML workload which contains BACKUP, the server asserts as follows: #0 0x00110402 in __kernel_vsyscall () #1 0x00bdc617 in pthread_kill () from /lib/libpthread.so.0 #2 0x0844e0b3 in write_core (sig=6) at stacktrace.c:302 #3 0x082a5314 in handle_segfault (sig=6) at mysqld.cc:2626 #4 <signal handler called> #5 0x00110402 in __kernel_vsyscall () #6 0x00a6e690 in raise () from /lib/libc.so.6 #7 0x00a6ff91 in abort () from /lib/libc.so.6 #8 0x00a6793e in __assert_fail () from /lib/libc.so.6 #9 0x0827c39e in Field_string::store (this=0x9f7f700, from=0xae1595f4 "126a40dd-767a-102b-af45-000c29c8314c\0019\022C\\3╞ющщщщщщщщ\220\226\025╝п\225\025╝ \201╦\b`", length=36, cs=0x8a495e0) at field.cc:6259 #10 0x084b58c9 in StorageInterface::decodeRecord (this=0x9f7f7b8, buf=0x9f7f5e0 "Ы\003") at ha_falcon.cpp:2513 #11 0x084b8f3b in StorageInterface::rnd_next (this=0x9f7f7b8, buf=0x9f7f5e0 "Ы\003") at ha_falcon.cpp:584 #12 0x083df860 in ha_partition::rnd_next (this=0x9f7f3d0, buf=0x9f7f5e0 "Ы\003") at ha_partition.cc:3266 #13 0x087af238 in default_backup::Backup::get_data (this=0xad8060f8, buf=@0xad84e19c) at be_default.cc:364 #14 0x087ab484 in backup::Backup_pump::pump (this=0xad84e178, howmuch=0xaf03df0c) at data_backup.cc:1251 #15 0x087ac233 in backup::Scheduler::step (this=0xaf03e1c8) at data_backup.cc:761 #16 0x087acf1e in backup::write_table_data (thd=0x9f7d1f0, info=@0xaf03e304, s=@0xa097a28) at data_backup.cc:597 #17 0x087a4709 in mysql_backup (thd=0x9f7d1f0, info=@0xaf03e304, s=@0xa097a28) at kernel.cc:467 #18 0x087a5afb in execute_backup_command (thd=0x9f7d1f0, lex=0x9f7e2b8) at kernel.cc:345 #19 0x082b6410 in mysql_execute_command (thd=0x9f7d1f0) at sql_parse.cc:2158 #20 0x082bedcc in mysql_parse (thd=0x9f7d1f0, inBuf=0xa084a40 "BACKUP DATABASE test TO \"/build/mysql-6.0/mysql-test/var/backup41\"", length=66, found_semicolon=0xaf041270) at sql_parse.cc:5751 #21 0x082bf83e in dispatch_command (command=COM_QUERY, thd=0x9f7d1f0, packet=0x9f67479 "BACKUP DATABASE test TO \"/build/mysql-6.0/mysql-test/var/backup41\"", packet_length=66) at sql_parse.cc:1045 #22 0x082c0b10 in do_command (thd=0x9f7d1f0) at sql_parse.cc:722 #23 0x082ae0e5 in handle_one_connection (arg=0x9f7d1f0) at sql_connect.cc:1134 #24 0x00bd750b in start_thread () from /lib/libpthread.so.0 #25 0x00b18b2e in clone () from /lib/libc.so.6 The assertion is in sql/field.cc, line 6259: /* Copy a string and fill with space */ int Field_string::store(const char *from,uint length,CHARSET_INFO *cs) { ASSERT_COLUMN_MARKED_FOR_WRITE; uint copy_length; const char *well_formed_error_pos; const char *cannot_convert_error_pos; const char *from_end_pos; /* See the comment for Field_long::store(long long) */ DBUG_ASSERT(table->in_use == current_thd); How to repeat: A simplified test case will be provided shortly.