MySQL Bugs: #47542: Online alter table prevents upgrade from 6.3 to 7.0

Bug #47542	Online alter table prevents upgrade from 6.3 to 7.0
Submitted:	23 Sep 2009 9:04	Modified:	28 Sep 2009 6:47
Reporter:	Tino Rachui	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S2 (Serious)
Version:	7.0.7	OS:	Any
Assigned to:	Jonas Oreland	CPU Architecture:	Any
Tags:	cluster online upgrade

Description:
I have a MySQL Cluster 6.3.20 which I try to online upgrade to version 7.0.7. Before I do this though (when I'm still completely running a 6.3.20 Cluster) I have changed the schema of my database (added a couple of tables, altered a few). Migrating the management node still works but when I try to start the first data node it always crashes. The error message I get:
"Node 2: Forced node shutdown completed. Occured during startphase 5. 
Caused by error 2341: 'Internal program error (failed ndbrequire)(Internal error, 
programming error or missing error message, please report a bug). 
Temporary error, restart node'."

How to repeat:
Run a MySQL Cluster 6.3.20 (mine has one management node, two data nodes, two SQL nodes). Create a database. Modify the database later (create additional tables, alter existing tables). Try to do an online upgrade to 7.0.7 afterwards.

Log files

Attachment: ndb_error_report_20090910125612.tar.bz2 (application/x-bzip, text), 73.27 KiB.

also attach/upload original schema, and modifications that you performed
to make it always fail

Initial schema of the database

Attachment: vda-schema-create.sql (text/x-sql), 21.07 KiB.

Changes to be applied to the database before starting migration

Attachment: vda-migrate-schema-extend.sql (text/x-sql), 6.12 KiB.

booooooom tjackalak!

I manage to reproduce.
(but I had to modify the migrate script which contained errors)
89c89
<   ADD `creation_date` DATETIME NULL,
---
> --  ADD `creation_date` DATETIME NULL,
106c106
<   ADD `template_id_v31` BIGINT NULL,
---
> --  ADD `template_id_v31` BIGINT NULL,
141c141
< UPDATE `vdadb`.`t_pool` t0, `vdadb`.`t_desktop` t1, `vdadb`.`t_template` t2 SET t0.template_id_v31 = t2.id WHERE t0.template_id = t1.id AND t1.external_id = t2.external_id AND t1.desktop_provider_id = t2.desktop_provider_id;
---
> UPDATE `vdadb`.`t_pool` t0, `vdadb`.`t_desktop` t1, `vdadb`.`t_template` t2 SET t0.template_id_v31 = t2.id WHERE t0.template_id_v31 = t1.id AND t1.external_id = t2.external_id AND t1.desktop_provider_id = t2.desktop_provider_id;

Initial db schema (this time really the initial one)

Attachment: vda-schema-create.sql (text/x-sql), 17.80 KiB.

@Jonas: Sorry I've attached the wrong initial db schema. Find the "really" initial one attached now. With this version no adaptations to the migration script should be necessary.

If an online alter table has been performed in 6.3, then subsequent
online upgrade to 7.0 will fail, as the state of the table object
in 6.3 was not converted to 7.0 format.

(but this conversion was only needed for tables that had been online altered)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/84337

3033 Jonas Oreland	2009-09-23
      ndb - bug#47542 - convert ALTER_TABLE_COMMITTED from 6.3 to SF_IN_USE in 7.0
        during online upgrade, or else object will not be recreated,
        causing mysteriuous subsequent failures

@Jonas: Thanks for the quick investigation and even quicker fix. Can we expect this fix to be part of 7.0.8? And is there a time line for 7.0.8 already?

expect in 7.0.8: yes
time line for 7.0.8: source release end of this month, binary some time after

and thx self for good testcase

/Jonas

pushed to 7.0.8 and 7.1

Documented bugfix in the NDB-7.0.8 changelog as follows:

        After an NDB table had an ALTER ONLINE TABLE operation performed on 
        it in a MySQL Cluster running a MySQL Cluster NDB 6.3.x release, it 
        could not be upgraded online to a MySQL Cluster NDB 7.0.x release. 
        This issue was detected using MySQL Cluster NDB 6.3.20, but is likely 
        to effect any MySQL Cluster NDB 6.3.x release supporting online DDL
        operations.

Also noted in Cluster Upgrades/Downgrades section of Manual.

Closed.

Fix works for me!

Pushed into 5.1.37-ndb-7.0.9 (revid:jonas@mysql.com-20090930075942-1q6asjcp0gaeynmj) (version source revid:jonas@mysql.com-20090923120124-77yqh89segwchgy6) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)

Pushed into 5.1.35-ndb-7.1.0 (revid:jonas@mysql.com-20090930080049-1c8a8cio9qgvhq35) (version source revid:jonas@mysql.com-20090923125213-cofd25jcqcm2cgn3) (merge vers: 5.1.35-ndb-7.1.0) (pib:11)