Bug #53354 crash when creating partitioned table with multiple columns in the partition key
Submitted: 2 May 2010 7:38 Modified: 28 Jan 2011 12:12
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S1 (Critical)
Version:mysql-5.1-telco-7.0 OS:Any (MS Windows, Linux)
Assigned to: Maitrayi Sabaratnam CPU Architecture:Any
Tags: mysql-5.1.44 ndb-7.1.3, ndb-7.0.14

[2 May 2010 7:38] Shane Bester
Description:
more dabbling in ndb, i found some partition crashes with multiple columns in the partition definition:

mysqld.exe!ha_ndbcluster::set_up_partition_info()[ha_ndbcluster.cc:12299]
mysqld.exe!ha_ndbcluster::create()[ha_ndbcluster.cc:7156]
mysqld.exe!ha_create_table()[handler.cc:3623]
mysqld.exe!rea_create_table()[unireg.cc:474]
mysqld.exe!mysql_create_table_no_lock()[sql_table.cc:3961]
mysqld.exe!mysql_create_table()[sql_table.cc:4061]
mysqld.exe!mysql_execute_command()[sql_parse.cc:2744]
mysqld.exe!mysql_parse()[sql_parse.cc:5981]
mysqld.exe!dispatch_command()[sql_parse.cc:1238]
mysqld.exe!do_command()[sql_parse.cc:877]
mysqld.exe!handle_one_connection()[sql_connect.cc:1133]
mysqld.exe!pstart()[my_winthread.c:105]
mysqld.exe!_callthreadstart()[thread.c:295]
mysqld.exe!_threadstart()[thread.c:275]
kernel32.dll!BaseThreadStart()

How to repeat:
#setup single node cluster then run:

drop table if exists `t1`;
create table `t1` (`a` int primary key) engine=ndbcluster
partition by key(`a`,`a`) partitions 1;
[2 May 2010 14:10] Sveta Smirnova
Thank you for the report.

Verified as described.
[3 May 2010 8:35] Hartmut Holzgraefe
Linux backtrace for ndb-7.0.14:

#0  0x006f8422 in __kernel_vsyscall ()
#1  0x007d0e93 in __pthread_kill (threadid=3051752304, signo=11)
    at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:64
#2  0x0867b6f3 in my_write_core (sig=11) at stacktrace.c:332
#3  0x082bacf9 in handle_segfault (sig=11) at mysqld.cc:2614
#4  <signal handler called>
#5  0x0850dde9 in ha_ndbcluster::set_up_partition_info (this=0x8c94568, 
    part_info=0x8c5a638, table=0xb5e5e148, tab_par=0xb5e5e034)
    at ha_ndbcluster.cc:12299
#6  0x084fb2d0 in ha_ndbcluster::create (this=0x8c94568, 
    name=0xb5e5efcb "./test/t1", form=0xb5e5e148, create_info=0xb5e5f368)
    at ha_ndbcluster.cc:7156
#7  0x084029b4 in handler::ha_create (this=0x8c94568, 
    name=0xb5e5efcb "./test/t1", form=0xb5e5e148, info=0xb5e5f368)
    at handler.cc:3415
#8  0x0840310b in ha_create_table (thd=0x8c5d058, path=0xb5e5efcb "./test/t1", 
    db=0x8c5a308 "test", table_name=0x8c5a108 "t1", create_info=0xb5e5f368, 
    update_create_info=false) at handler.cc:3622
#9  0x083afe8d in rea_create_table (thd=0x8c5d058, 
    path=0xb5e5efcb "./test/t1", db=0x8c5a308 "test", 
    table_name=0x8c5a108 "t1", create_info=0xb5e5f368, create_fields=..., 
    keys=1, key_info=0x8c5bb88, file=0x8c5a798) at unireg.cc:473
#10 0x084288d3 in mysql_create_table_no_lock (thd=0x8c5d058, 
---Type <return> to continue, or q <return> to quit---
    db=0x8c5a308 "test", table_name=0x8c5a108 "t1", create_info=0xb5e5f368, 
    alter_info=0xb5e5f658, internal_tmp_table=false, select_field_count=0)
    at sql_table.cc:3959
#11 0x08428de4 in mysql_create_table (thd=0x8c5d058, db=0x8c5a308 "test", 
    table_name=0x8c5a108 "t1", create_info=0xb5e5f368, alter_info=0xb5e5f658, 
    internal_tmp_table=false, select_field_count=0) at sql_table.cc:4061
#12 0x082ce8a7 in mysql_execute_command (thd=0x8c5d058) at sql_parse.cc:2744
#13 0x082d8dbe in mysql_parse (thd=0x8c5d058, 
    inBuf=0x8c59ff0 "create table `t1` (`a` int primary key) engine=ndbcluster\npartition by key(`a`,`a`) partitions 1", length=96, found_semicolon=0xb5e60058)
    at sql_parse.cc:5977
#14 0x082cad2b in dispatch_command (command=COM_QUERY, thd=0x8c5d058, 
    packet=0x8c7d961 "create table `t1` (`a` int primary key) engine=ndbcluster\npartition by key(`a`,`a`) partitions 1", packet_length=96)
    at sql_parse.cc:1236
#15 0x082c9e1f in do_command (thd=0x8c5d058) at sql_parse.cc:877
#16 0x082c81c6 in handle_one_connection (arg=0x8c5d058) at sql_connect.cc:1133
#17 0x007cb80e in start_thread (arg=0xb5e60b70) at pthread_create.c:300
#18 0x004e88de in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
[3 May 2010 9:13] Hartmut Holzgraefe
actually only happens when the same column is listed more than once 
in the PARTITION BY KEY list.

This works: 
  create table `t1` (`a` int, b int, primary key (a,b)) 
  engine=ndbcluster partition by key(`a`,`b`) partitions 1;

This again doesn't:
  create table `t1` (`a` int, b int, primary key (a,b)) 
  engine=ndbcluster partition by key(`a`,`b`,`a`) partitions 1;

Crash is on dereferencing "fields[i]->field_index" in:

    [...]
    Field **fields= part_info->part_field_array;
    [...]
    for (i= 0; i < part_info->part_field_list.elements; i++)
    {
      NDBCOL *col= tab->getColumn(fields[i]->field_index);
      [...]

shouldn't the loop fetch from part_field_list, not part_field_array, there?
[4 May 2010 15:05] Maitrayi Sabaratnam
Validating the fields[i] pointer before accessing will solve the problem.

=== modified file 'sql/ha_ndbcluster.cc'
--- sql/ha_ndbcluster.cc        2010-05-04 13:16:50 +0000
+++ sql/ha_ndbcluster.cc        2010-05-04 14:42:28 +0000
@@ -12557,13 +12557,11 @@
   if (part_info->part_type == HASH_PARTITION &&
       part_info->list_of_part_fields == TRUE)
   {
-    Field **fields= part_info->part_field_array;
-
     ftype= NDBTAB::HashMapPartition;

-    for (i= 0; i < part_info->part_field_list.elements; i++)
+    for (Field **fields= part_info->part_field_array;  *(fields); ++fields)
     {
-      NDBCOL *col= tab->getColumn(fields[i]->field_index);
+      NDBCOL *col= tab->getColumn((*fields)->field_index);
       DBUG_PRINT("info",("setting dist key on %s", col->getName()));
       col->setPartitionKey(TRUE);
     }

mysql> create table `t11` (`a` int, b int, primary key (a,b)) engine=ndbcluster partition by key(`a`,`b`,`a`) partitions 1;
Query OK, 0 rows affected (0.41 sec)

mysql> create table `t10` (`a` int primary key ) engine=ndbcluster partition by key(`a`,`a`) partitions 1;
Query OK, 0 rows affected (0.18 sec)
[4 May 2010 15:08] Jonas Oreland
Maitrayi,

This form should give an error, cause it doesnt make sense
to put same key-column in list more than once

/Jonas
[10 May 2010 14:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/107872

3531 Maitrayi Sabaratnam	2010-05-10
      Bug#53354 crash when creating partitioned table with multiple columns in the partition key
[10 May 2010 17:36] MySQL Verification Team
There's a typo in the new error message :)

s/partion/partition/
[12 Oct 2010 12:34] MySQL Verification Team
Patch pending for months.. very strange.

Version: '5.1.47-ndb-7.1.8-cluster-gpl'  socket: ''  port: 3306  MySQL Cluster Server (GPL)

mysqld.exe!ha_ndbcluster::set_up_partition_info()[ha_ndbcluster.cc:12865]
mysqld.exe!ha_ndbcluster::create()[ha_ndbcluster.cc:7677]
mysqld.exe!ha_create_table()[handler.cc:3841]
mysqld.exe!rea_create_table()[unireg.cc:474]
mysqld.exe!mysql_create_table_no_lock()[sql_table.cc:3978]
mysqld.exe!mysql_create_table()[sql_table.cc:4078]
mysqld.exe!mysql_execute_command()[sql_parse.cc:2759]
mysqld.exe!mysql_parse()[sql_parse.cc:5996]
mysqld.exe!dispatch_command()[sql_parse.cc:1238]
mysqld.exe!do_command()[sql_parse.cc:877]
mysqld.exe!handle_one_connection()[sql_connect.cc:1143]
mysqld.exe!pstart()[my_winthread.c:105]
mysqld.exe!_callthreadstart()[thread.c:295]
mysqld.exe!_threadstart()[thread.c:275]
kernel32.dll!BaseThreadStart()
[13 Oct 2010 10:07] Martin Skold
+  { 4330, DMEC, AE, "Duplicate key is not allowed in hash partion" },
Should be:
{ 4330, DMEC, AE, "Duplicate key is not allowed in hash partition" },
Fix this and it is approved.
[26 Oct 2010 8:13] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/121896

3531 Maitrayi Sabaratnam	2010-10-26
      Bug#53354 crash when creating partitioned table with multiple columns in the partition key
[29 Oct 2010 8:46] Maitrayi Sabaratnam
It was pointet out that it would be better to fix this in the server, since this will apply to other engines as well. The following fix was accepted by the server team. This will be added to cluster code, quoted with #ifndef MCP_BUG53354..#endif There will be a new bug filed on the server for this issue. 

I'm writing a test case now, before pushing.

=== modified file 'sql/sql_partition.cc'
--- sql/sql_partition.cc        2010-10-12 14:53:28 +0000
+++ sql/sql_partition.cc        2010-10-28 13:28:50 +0000
@@ -764,6 +764,9 @@
   bool result;
   char *field_name;
   bool is_list_empty= TRUE;
+  int fields_handled = 0;
+  char* field_name_array[MAX_KEY];
+
   DBUG_ENTER("handle_list_of_fields");

   while ((field_name= it++))
@@ -779,6 +782,24 @@
       result= TRUE;
       goto end;
     }
+
+#ifndef MCP_BUG53354
+    /* Check for duplicate fields in the list.
+     * Assuming that there are not many fields in the partition key list.
+     * If there were, better algorithm than the for-loop is recommended.
+     */
+
+    field_name_array[fields_handled] = field_name;
+    for (int i = 0; i < fields_handled; ++i)
+    {
+      if (strcmp(field_name_array[i], field_name) == 0)
+      {
+         my_error(ER_FIELD_NOT_FOUND_PART_ERROR, MYF(0));
+         DBUG_RETURN(TRUE);
+      }
+    }
+    fields_handled++;
+#endif
   }
   if (is_list_empty)
   {
[1 Nov 2010 9:13] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/122402

3901 Maitrayi Sabaratnam	2010-11-01
      Bug#53354 crash when creating partitioned table with multiple columns in the partition key
[2 Nov 2010 8:04] Bugs System
Pushed into mysql-5.1-telco-7.0 5.1.51-ndb-7.0.20 (revid:maitrayi.sabaratnam@oracle.com-20101102075843-lhb4yl8a4zjm8e41) (version source revid:maitrayi.sabaratnam@oracle.com-20101102075843-lhb4yl8a4zjm8e41) (merge vers: 5.1.51-ndb-7.0.20) (pib:21)
[2 Nov 2010 13:14] Maitrayi Sabaratnam
This bug is fixed in cluster. A new bug (Bug #57924) is created on mysql server 5.1 to handle this issue.
[4 Nov 2010 13:46] Jon Stephens
Mai confirms on IRC this was pushed to 7.0+. Documented in the NDB-7.0.20 and 7.1.9 changelogs as follows:

        Trying to use the same column more than once in the partitioning
        key when partitioning a table by KEY caused mysqld to crash.
        Such duplication of key columns is now expressly disallowed, and
        fails with an appropriate error.

Closed.
[21 Jan 2011 9:10] Magnus BlÄudd
Moving test case for this bug to ndb_partition_error.test, should not have been added to ndb_basic.test
[26 Jan 2011 20:15] Bugs System
Pushed into mysql-trunk 5.6.2 (revid:mattias.jonsson@oracle.com-20110126201331-ab82uv7s5qmdufs5) (version source revid:mattias.jonsson@oracle.com-20110126201331-ab82uv7s5qmdufs5) (merge vers: 5.6.2) (pib:24)
[26 Jan 2011 20:16] Bugs System
Pushed into mysql-5.5 5.5.10 (revid:mattias.jonsson@oracle.com-20110126183353-8fngni1uuyybmz9u) (version source revid:mattias.jonsson@oracle.com-20110126183353-8fngni1uuyybmz9u) (merge vers: 5.5.10) (pib:24)
[26 Jan 2011 20:16] Bugs System
Pushed into mysql-5.1 5.1.56 (revid:mattias.jonsson@oracle.com-20110126155021-evjpfdciphnd50sy) (version source revid:mattias.jonsson@oracle.com-20110126155021-evjpfdciphnd50sy) (merge vers: 5.1.56) (pib:24)
[27 Jan 2011 16:30] Jon Stephens
Also documented fix in the 5.1.56, 5.5.10, and 5.6.2 changelogs; see above for changelog entry. 

Closed.
[27 Jan 2011 16:47] Jon Stephens
Disregard my previous comment.

Fix for both bugs documented additionally in the 5.1.56 changelog only; see developer comments in BUG#57924.

Closed.
[28 Jan 2011 12:12] Jon Stephens
Fix is already documented -- returning to Closed state.