MySQL Bugs: #19148: Backup causes cluster down if _really

Bug #19148	Backup causes cluster down if _really_ early abort happens
Submitted:	17 Apr 2006 20:03	Modified:	2 Nov 2006 5:08
Reporter:	Serge Kozlov	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S2 (Serious)
Version:	5.1	OS:	Linux (Linux FC4)
Assigned to:	Stewart Smith	CPU Architecture:	Any

Description:
Add to config.ini BackupMemory = 1M and start backup in ndb_mgm.

 
space/run/ndb_2_error.log
=================
Current byte-offset of file-pointer is: 568

Time: Monday 17 April 2006 - 21:55:04
Status: Temporary error, restart node
Message: Assertion (Internal error, programming error or missing error message,
please report a bug)
Error: 2301
Error data: ArrayPool<T>::getPtr
Error object: ../../../../../storage/ndb/src/kernel/vm/ArrayPool.hpp line: 425 (
block: BACKUP)
Program: /home/ndbdev/skozlov/builds/libexec/ndbd
Pid: 18020
Trace: /space/run/ndb_2_trace.log.1
Version: Version 5.1.9 (beta)
***EOM***

How to repeat:
1. Add to config.ini BackupMemory: 1M. See below the example:

[ndbd default]
NoOfReplicas: 1
IndexMemory: 50M
DataMemory: 100M
DataDir: /space/run
FileSystemPath: /space/run
BackupMemory: 1M
#Note: you had 2 replicas but only 1 data node

[mgm default]
ArbitrationRank: 1
DataDir: /space/run

[ndb_mgmd]
Id: 1
HostName: ndb16

[ndbd]
Id: 2
HostName: ndb16

[ndbd]
Id: 3
HostName: ndb16

[mysqld]
Id: 15

2. Start cluster.
3. Run ndb_mgm
4. Start backup via 'START BACKUP' command.
5. Following error appears and both ndbd nodes stopped:

Start of backup failed
Node 2: Forced node shutdown completed. Initiated by signal 0. Caused by error 2
301: 'Assertion(Internal error, programming error or missing error message, plea
se report a bug). Temporary error, restart node'.
*  3001: Could not start backup
*        Backup abortet due to node failure: Permanent error: Internal error
ndb_mgm> Node 3: Forced node shutdown completed. Initiated by signal 0. Caused b
y error 2305: 'Arbitrator shutdown, please investigate error(s) on other node(s)
(Arbitration error). Temporary error, restart node'.
exit

Cluster handles wrong configurations by shutting down nodes

Part of setting up cluster for an application is to test out which applicaiton that works.

This is not something that will change anytime soon.

However you should get an appropriate error message when the node goes down.

I'm therefore relableing the bug and setting different prio

added wrong error message to 17297

Have patch for 5.1 that works (with additional test case in autotest).

Now just checking with 5.0.

Not going to check 4.1 as only really caused by incorrect configuration (that can be fixed with online upgrade).

doesn't affect 5.0 (from quick test)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/13845

ChangeSet@1.2314, 2006-10-18 18:19:16+10:00, stewart@willster.(none) +4 -0
  Bug #19148  	Backup causes cluster down if _really_ early abort happens
  
  Early abort, failing to allocate buffers, would go down the abort track and
   end up trying to sendDropTriggers (as the standard stop backup does). 
  However, it is possible to not have any tables yet defined in the backup 
  (due to allocating buffers is before this). Simple check in sendDropTrig 
  (and continue to next step). Files may be open, so we continue to close them.
  
  Also updated ERROR_Codes (and added one to test this bug).

approved by Jonas, pushed to 5.1-ndb

pushed into 5.1.13

Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Documented fix for 5.1.13.