Bug #47542 Online alter table prevents upgrade from 6.3 to 7.0
Submitted: 23 Sep 2009 9:04 Modified: 28 Sep 2009 6:47
Reporter: Tino Rachui Email Updates:
Status: Closed Impact on me:
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:7.0.7 OS:Any
Assigned to: Jonas Oreland CPU Architecture:Any
Tags: cluster online upgrade

[23 Sep 2009 9:04] Tino Rachui
I have a MySQL Cluster 6.3.20 which I try to online upgrade to version 7.0.7. Before I do this though (when I'm still completely running a 6.3.20 Cluster) I have changed the schema of my database (added a couple of tables, altered a few). Migrating the management node still works but when I try to start the first data node it always crashes. The error message I get:
"Node 2: Forced node shutdown completed. Occured during startphase 5. 
Caused by error 2341: 'Internal program error (failed ndbrequire)(Internal error, 
programming error or missing error message, please report a bug). 
Temporary error, restart node'."

How to repeat:
Run a MySQL Cluster 6.3.20 (mine has one management node, two data nodes, two SQL nodes). Create a database. Modify the database later (create additional tables, alter existing tables). Try to do an online upgrade to 7.0.7 afterwards.
[23 Sep 2009 9:05] Tino Rachui
Log files

Attachment: ndb_error_report_20090910125612.tar.bz2 (application/x-bzip, text), 73.27 KiB.

[23 Sep 2009 9:07] Jonas Oreland
also attach/upload original schema, and modifications that you performed
to make it always fail
[23 Sep 2009 9:12] Tino Rachui
Initial schema of the database

Attachment: vda-schema-create.sql (text/x-sql), 21.07 KiB.

[23 Sep 2009 9:13] Tino Rachui
Changes to be applied to the database before starting migration

Attachment: vda-migrate-schema-extend.sql (text/x-sql), 6.12 KiB.

[23 Sep 2009 10:07] Jonas Oreland
booooooom tjackalak!

I manage to reproduce.
(but I had to modify the migrate script which contained errors)
<   ADD `creation_date` DATETIME NULL,
> --  ADD `creation_date` DATETIME NULL,
<   ADD `template_id_v31` BIGINT NULL,
> --  ADD `template_id_v31` BIGINT NULL,
< UPDATE `vdadb`.`t_pool` t0, `vdadb`.`t_desktop` t1, `vdadb`.`t_template` t2 SET t0.template_id_v31 = t2.id WHERE t0.template_id = t1.id AND t1.external_id = t2.external_id AND t1.desktop_provider_id = t2.desktop_provider_id;
> UPDATE `vdadb`.`t_pool` t0, `vdadb`.`t_desktop` t1, `vdadb`.`t_template` t2 SET t0.template_id_v31 = t2.id WHERE t0.template_id_v31 = t1.id AND t1.external_id = t2.external_id AND t1.desktop_provider_id = t2.desktop_provider_id;
[23 Sep 2009 11:00] Tino Rachui
Initial db schema (this time really the initial one)

Attachment: vda-schema-create.sql (text/x-sql), 17.80 KiB.

[23 Sep 2009 11:02] Tino Rachui
@Jonas: Sorry I've attached the wrong initial db schema. Find the "really" initial one attached now. With this version no adaptations to the migration script should be necessary.
[23 Sep 2009 11:58] Jonas Oreland
If an online alter table has been performed in 6.3, then subsequent
online upgrade to 7.0 will fail, as the state of the table object
in 6.3 was not converted to 7.0 format.

(but this conversion was only needed for tables that had been online altered)
[23 Sep 2009 12:01] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:


3033 Jonas Oreland	2009-09-23
      ndb - bug#47542 - convert ALTER_TABLE_COMMITTED from 6.3 to SF_IN_USE in 7.0
        during online upgrade, or else object will not be recreated,
        causing mysteriuous subsequent failures
[23 Sep 2009 12:32] Tino Rachui
@Jonas: Thanks for the quick investigation and even quicker fix. Can we expect this fix to be part of 7.0.8? And is there a time line for 7.0.8 already?
[23 Sep 2009 12:36] Jonas Oreland
expect in 7.0.8: yes
time line for 7.0.8: source release end of this month, binary some time after

and thx self for good testcase

[24 Sep 2009 12:21] Jonas Oreland
pushed to 7.0.8 and 7.1
[24 Sep 2009 13:22] Jon Stephens
Documented bugfix in the NDB-7.0.8 changelog as follows:

        After an NDB table had an ALTER ONLINE TABLE operation performed on 
        it in a MySQL Cluster running a MySQL Cluster NDB 6.3.x release, it 
        could not be upgraded online to a MySQL Cluster NDB 7.0.x release. 
        This issue was detected using MySQL Cluster NDB 6.3.20, but is likely 
        to effect any MySQL Cluster NDB 6.3.x release supporting online DDL

Also noted in Cluster Upgrades/Downgrades section of Manual.

[28 Sep 2009 6:47] Tino Rachui
Fix works for me!
[30 Sep 2009 8:14] Bugs System
Pushed into 5.1.37-ndb-7.0.9 (revid:jonas@mysql.com-20090930075942-1q6asjcp0gaeynmj) (version source revid:jonas@mysql.com-20090923120124-77yqh89segwchgy6) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[30 Sep 2009 8:15] Bugs System
Pushed into 5.1.35-ndb-7.1.0 (revid:jonas@mysql.com-20090930080049-1c8a8cio9qgvhq35) (version source revid:jonas@mysql.com-20090923125213-cofd25jcqcm2cgn3) (merge vers: 5.1.35-ndb-7.1.0) (pib:11)