Description:
We are using:
revid jonas@mysql.com-20090131154212-kunoo34f0hqmiz4z dated
2009-02-02 from repo mysql-5.1-telco-6.4 compiled with Solaris Workshop 12
as follows:
CC=cc CXX=CC CFLAGS="-xO5 -fast -g -mt -m64" CXXFLAGS="-xO5 -fast -g -mt
-m64" ./configure --prefix=<somepath> --with-plugins=all --without-docs
--without-man
When trying to check the correction for bugid 42450
the following problem occured:
ndb_mgm> Node 3: Forced node shutdown completed. Occured during startphase 5. Caused by error 2310: 'Error while reading the REDO log(Ndbd file system inconsistency error, please report a bug). Ndbd file system error, restart node initial'.
In ndb_3_out.log are messages like:
...
2009-02-02 17:59:34 [ndbd] WARNING -- Ndb kernel thread 6 is stuck in: Job Hand
ling elapsed=7920
2009-02-02 17:59:34 [ndbd] INFO -- Watchdog: User time: 241 System time: 94
4
2009-02-02 17:59:34 [ndbd] WARNING -- Ndb kernel thread 6 is stuck in: Job Hand
ling elapsed=8030
2009-02-02 17:59:34 [ndbd] INFO -- Watchdog: User time: 241 System time: 94
5
2009-02-02 17:59:35 [ndbd] WARNING -- Ndb kernel thread 6 is stuck in: Job Hand
ling elapsed=8140
2009-02-02 17:59:35 [ndbd] INFO -- Watchdog: User time: 242 System time: 94
5
...
saving 102fb0000 at 1005e2d10 (0)saving 102ee8000 at 1005df010 (0)
saving 103140000 at 1005ea710 (0)saving 103078000 at 1005e6a10 (0)
saving 102e20000 at 1005db310 (0)
restartCreateObj(1) file: 1
restartCreateObj(2) file: 1
restartCreateObj(3) file: 1
restartCreateObj(4) file: 1
restartCreateObj(5) file: 1
restartCreateObj(6) file: 1
restartCreateObj(8) file: 1
restartCreateObj(9) file: 1
restartCreateObj(10) file: 1
restartCreateObj(11) file: 1
restartCreateObj(12) file: 1
restartCreateObj(16) file: 1
restartCreateObj(17) file: 1
restartCreateObj(18) file: 1
restartCreateObj(19) file: 1
restartCreateObj(20) file: 1
restartCreateObj(21) file: 1
restartCreateObj(22) file: 1
restartCreateObj(23) file: 1
restartCreateObj(24) file: 1
restartCreateObj(26) file: 1
restartCreateObj(30) file: 1
restartCreateObj(32) file: 1
tab: 2 frag: 0 replicaP->nextLcp: 0
scanning idx: 1 lcpId: 578 774900
scanning idx: 0 lcpId: 0
- scanning idx: 2 lcpId: 0
Didnt find any LCP for node: 3 tab: 2 frag: 0
tab: 2 frag: 1 replicaP->nextLcp: 0
scanning idx: 1 lcpId: 578 774900
scanning idx: 0 lcpId: 0
...
Didnt find any LCP for node: 3 tab: 32 frag: 5
tab: 32 frag: 6 replicaP->nextLcp: 0
scanning idx: 1 lcpId: 578 774900
scanning idx: 0 lcpId: 577 774900
- scanning idx: 2 lcpId: 0
Didnt find any LCP for node: 3 tab: 32 frag: 6
tab: 32 frag: 7 replicaP->nextLcp: 0
scanning idx: 1 lcpId: 578 774900
scanning idx: 0 lcpId: 577 774900
- scanning idx: 2 lcpId: 0
Didnt find any LCP for node: 3 tab: 32 frag: 7
2009-02-02 17:59:50 [ndbd] WARNING -- Ndb kernel thread 2 is stuck in: Print Jo
b Buffers at crash elapsed=109
2009-02-02 17:59:50 [ndbd] INFO -- Watchdog: User time: 2518 System time: 1
349
2009-02-02 17:59:50 [ndbd] WARNING -- Ndb kernel thread 2 is stuck in: Print Jo
b Buffers at crash elapsed=219
2009-02-02 17:59:50 [ndbd] INFO -- Watchdog: User time: 2518 System time: 1
349
2009-02-02 17:59:50 [ndbd] WARNING -- Ndb kernel thread 2 is stuck in: Print Jo
b Buffers at crash elapsed=329
...
2009-02-02 17:59:52 [ndbd] WARNING -- Ndb kernel thread 2 is stuck in: Print Jo
b Buffers at crash elapsed=549
2009-02-02 17:59:52 [ndbd] INFO -- Watchdog: User time: 2551 System time: 1
358
2009-02-02 17:59:52 [ndbd] INFO -- Error while reading REDO log. from 16262
D=8, F=4 Mb=142 FP=4575 W1=3386 W2=7 : end of log wo/ having found last GCI gci:
784127
2009-02-02 17:59:52 [ndbd] INFO -- DBLQH (Line: 16369) 0x0000000a
2009-02-02 17:59:52 [ndbd] INFO -- Error handler startup shutting down syste
m
2009-02-02 17:59:52 [ndbd] INFO -- Error handler shutdown completed - exitin
g
2009-02-02 17:59:52 [ndbd] INFO -- Angel received ndbd startup failure count
1.
2009-02-02 17:59:56 [ndbd] ALERT -- Node 3: Forced node shutdown completed. O
ccured during startphase 5. Caused by error 2310: 'Error while reading the REDO
log(Ndbd file system inconsistency error, please report a bug). Ndbd file system
error, restart node initial'.
How to repeat:
On Node A:
- running an "insert" Process (insert into ....)
- running a "delete" Process (delete from ...)
(Both processes dealing with the same table)
On Node B:
- restart ndbmtd
- during start phase, kill -9 "ndbmtd"
