MySQL Bugs: #12186: NULL VARCHAR UCS2 column in InnoDB table crashes the server

Bug #12186	NULL VARCHAR UCS2 column in InnoDB table crashes the server
Submitted:	26 Jul 2005 19:52	Modified:	28 Jul 2005 11:32
Reporter:	Hakan Küçükyılmaz	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S1 (Critical)
Version:	5.0.11	OS:	Linux (Linux)
Assigned to:	Marko Mäkelä	CPU Architecture:	Any

Description:
A "SELECT * FROM ..." crashes the server if table is UCS2.

How to repeat:
 CREATE TABLE "TMP_OSQLJVER1" (
  "K1VC4" varchar(4) NOT NULL,
  "K2INT" int(11) NOT NULL,
  "FVC8" varchar(8) default NULL,
  "FVB16" binary(16) default NULL,
  "FSHORT" smallint(6) default NULL,
  "FINT" int(11) default NULL,
  "FBIGINT" bigint(20) default NULL,
  "FDEC5_2" decimal(5,2) default NULL,
  "FDOUBLE" double default NULL,
  "FDATE" date default NULL,
  "FTIME" time default NULL,
  "FTSTAMP" datetime default NULL,
  "FCLOB" longtext,
  "FBLOB" blob,
  PRIMARY KEY  ("K1VC4","K2INT")
) ENGINE=InnoDB DEFAULT CHARSET=ucs2;

INSERT INTO TMP_OSQLJVER1 (K1VC4, K2INT) VALUES (1, 1);
SELECT * FROM TMP_OSQLJVER1;

-- sql_mode="ANSI_QUOTES,NO_BACKSLASH_ESCAPES,TRADITIONAL,IGNORE_SPACE"

Backtrace from log:

050726 21:51:19InnoDB: Assertion failure in thread 163851 in file row0sel.c line 2541
InnoDB: Failing assertion: !(len & 1)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=2093056
max_used_connections=1
max_connections=100
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 425583 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x8bb2278
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6d9988c8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8154c78
0x4005a96c
0x829b0cf
0x829d7ce
0x820ae44
0x820b268
0x820b3c2
0x81fa559
0x81ac4d1
0x81ab48d
0x81ab024
0x819caf2
0x819d706
0x81990d2
0x8169547
0x8170640
0x8167d0a
0x8167714
0x8166a58
0x40054f60
0x401e7327
New value of fp=(nil) failed sanity check, terminating stack trace!
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do 
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8bdc3a0 = SELECT * FROM TMP_OSQLJVER1
thd->thread_id=7
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0
050726 21:51:19  mysqld restarted
050726 21:51:19 [Warning] No argument was provided to --log-bin, and --log-bin-index was not used; so replication may break when this MySQL server acts as a master and has his hostname changed!! Please use '--log-bin=lmy001-bin' to avoid this problem.
050726 21:51:20  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
050726 21:51:20  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 2 4227498255.
InnoDB: Doing recovery: scanned up to log sequence number 2 4227498255
InnoDB: Last MySQL binlog file position 0 1173, file name ./lmy001-bin.000086
050726 21:51:20  InnoDB: Started; log sequence number 2 4227498255
050726 21:51:20 [Note] Recovering after a crash using lmy001-bin
050726 21:51:20 [Note] Starting crash recovery...
050726 21:51:20 [Note] Crash recovery finished.
050726 21:51:20 [Note] /usr/local/mysql-5.0-debug/libexec/mysqld: ready for connections.
Version: '5.0.11-beta-debug-log'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution

Simplified test case:

CREATE TABLE TMP_OSQLJVER1 (   FVC8 VARCHAR(8) default NULL ) ENGINE=InnoDB DEFAULT CHARSET=ucs2;
INSERT INTO TMP_OSQLJVER1 VALUES ();
SELECT * FROM TMP_OSQLJVER1;

Also, ALTER TABLE on this table will crash the server:

ALTER TABLE TMP_OSQLJVER1 MODIFY FVC8 CHAR(8);
ALTER TABLE TMP_OSQLJVER1 ADD i INT;

Notes:

* Behavior does not occur with utf8 character set
* Behavior only occurs with VARCHAR columns (not CHAR, TEXT, or VARBINARY columns)
* Behavior also occurs on Windows
* Behavior is NOT present in MyISAM storage engine
* Behavior is present in MySQL 5.0.9
* Behavior is NOT present in MySQL 4.1.13

lmy001:~ # resolve_stack_dump -s /tmp/mysqld.sym -n /tmp/mysqld.stack | c++filt 
0x8154c78 handle_segfault + 272
0x4005a96c _end + 934076172
0x829b0cf row_sel_store_mysql_rec + 1063
0x829d7ce row_search_for_mysql + 7950
0x820ae44 ha_innobase::index_read(char*, char const*, unsigned int, ha_rkey_function) + 286
0x820b268 ha_innobase::index_next_same(char*, char const*, unsigned int) + 10
0x820b3c2 ha_innobase::rnd_init(bool) + 20
0x81fa559 rr_sequential(st_read_record*) + 13
0x81ac4d1 test_if_quick_select(st_join_table*) + 45
0x81ab48d sub_select(JOIN*, st_join_table*, bool) + 65
0x81ab024 do_select(JOIN*, List<Item>*, st_table*, Procedure*) + 488
0x819caf2 JOIN::exec() + 5608
0x819d706 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orderSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 300
0x81990d2 handle_select(THD*, st_lex*, select_result*, unsigned long) + 58
0x8169547 mysql_execute_command(THD*) + 1501
0x8170640 mysql_parse(THD*, char*, unsigned int) + 148
0x8167d0a dispatch_command(enum_server_command, THD*, char*, unsigned int) + 1376
0x8167714 do_command(THD*) + 314
0x8166a58 handle_one_connection + 590
0x40054f60 _end + 934053120
0x401e7327 _end + 935700679

This bug was introduced in MySQL 5.0.3 with true VARCHAR columns. When padding a NULL true VARCHAR column with blanks, InnoDB does not set the field length bytes. I don't know if the padding is needed any more. I would tend to believe that NULL true VARCHAR columns could just be set to length zero, but this should be tested. See Bug #154.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/27639

I made a simpler and more correct patch, which is available at the following location:
http://lists.mysql.com/internals/27673

Marko,

please update this comment:

                                /* MySQL pads all non-BLOB and non-TEXT
                                string types with space ' ' */

Does this patch also fix Bug #8552?

Regards,

Heikki

Heikki,

I fixed that comment in <http://lists.mysql.com/internals/27674>.

Bug #8552 is still alive and kicking:

checksum table TMP_OSQLJVER1;
-- 2286516652
alter table TMP_OSQLJVER1 row_format=redundant;
checksum table TMP_OSQLJVER1;
-- 4133951980
alter table TMP_OSQLJVER1 row_format=default;
checksum table TMP_OSQLJVER1;
-- 4133951980
alter table TMP_OSQLJVER1 row_format=fixed;
checksum table TMP_OSQLJVER1;
-- 4133951980
alter table TMP_OSQLJVER1 row_format=compact;
checksum table TMP_OSQLJVER1;
-- 4133951980
alter table TMP_OSQLJVER1 row_format=default engine=myisam;
checksum table TMP_OSQLJVER1;
-- 425803963
alter table TMP_OSQLJVER1 row_format=fixed engine=myisam;
checksum table TMP_OSQLJVER1;
-- 4133951980
checksum table TMP_OSQLJVER1;
-- 4133951980

Thank you for your bug report. This issue has been committed to our
source repository of that product and will be incorporated into the
next release.

If necessary, you can access the source repository and build the latest
available version, including the bugfix, yourself. More information 
about accessing the source trees is available at
    http://www.mysql.com/doc/en/Installing_source_tree.html

Additional info:

The fix should be included in the 5.0.11 release.