Bug #18798 mysqld cores on update in ha_ndbcluster call if cluster has failed before
Submitted: 5 Apr 2006 13:01 Modified: 13 May 2006 14:29
Reporter: Jonathan Miller Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:5.1.9 OS:Linux (Linux 32 Bit OS)
Assigned to: Martin Skold CPU Architecture:Any

[5 Apr 2006 13:01] Jonathan Miller
Description:
Tue Apr  4 22:17:42 2006
tid:-1213297744 mysql/dbc_common.c:102
ROLLBACK failed. mysql reports: 1297 Got temporary error 4025 'Node failure caused abort of transaction' from NDBCLUSTER
Tue Apr  4 22:17:42 2006
tid:-1213297744 db_threadpool.c:118
process_transaction() error on new-order
Tue Apr  4 22:17:42 2006
tid:-1212765264 mysql/dbc_common.c:102
ROLLBACK failed. mysql reports: 1297 Got temporary error 4025 'Node failure caused abort of transaction' from NDBCLUSTER
Tue Apr  4 22:17:42 2006
tid:-1212765264 db_threadpool.c:118
process_transaction() error on payment
Tue Apr  4 22:17:43 2006
tid:-1210901584 mysql/dbc_common.c:120
NEW_ORDER_9: UPDATE stock
SET s_quantity = s_quantity - 2
WHERE s_i_id = 56318
  AND s_w_id = 2
mysql reports: 2013 Lost connection to MySQL server during query
Tue Apr  4 22:17:43 2006
tid:-1210901584 nonsp/dbc_new_order.c:256
NEW_ORDER_9 query: UPDATE stock
SET s_quantity = s_quantity - 2
WHERE s_i_id = 56318
 AND s_w_id = 2
Tue Apr  4 22:17:43 2006
tid:-1210901584 nonsp/dbc_new_order.c:29
NEW_ORDER FINISHED WITH RC 17

0x81cfd98 handle_segfault + 438
0x112420 (?)
(nil)
0x8328184 _ZN13ha_ndbcluster16records_in_rangeEjP12st_key_rangeS1_ + 536
0x828e89b _Z16check_quick_keysP5PARAMjP7SEL_ARGPcjS3_j + 1691
0x828e4fa _Z16check_quick_keysP5PARAMjP7SEL_ARGPcjS3_j + 762
0x828ea6e _Z18check_quick_selectP5PARAMjP7SEL_ARG + 330
0x828ec39 _Z20get_key_scans_paramsP5PARAMP8SEL_TREEbd + 289
0x8294514 _ZN10SQL_SELECT17test_quick_selectEP3THD6BitmapILj64EEymb + 1820
0x82557dd _ZN10SQL_SELECT11check_quickEP3THDbm + 73
0x8251f30 _Z12mysql_updateP3THDP13st_table_listR4ListI4ItemES6_PS4_jP8st_orderm15enum_duplicatesb + 1366
0x81eaf5c _Z21mysql_execute_commandP3THD + 9248
0x81f0aef _Z11mysql_parseP3THDPcj + 535
0x81f134c _Z16dispatch_command19enum_server_commandP3THDPcj + 1862
0x81f248e _Z10do_commandP3THD + 260
0x81f2845 handle_one_connection + 725
0xa25b80 (?)
0x97d9ce (?)

How to repeat:
Run DBT2 with Cluster Disk Data
[5 Apr 2006 13:01] Jonathan Miller
060403 21:39:30 [ERROR] NDB: Could not get apply status share
060404 21:26:37 [ERROR] Got error 4028 when reading table './dbt2/stock'
060404 21:26:37 [ERROR] Got error 4028 when reading table './dbt2/stock'
060404 21:26:37 [ERROR] Got error 270 when reading table './dbt2/stock'
060404 21:26:37 [ERROR] Got error 270 when reading table './dbt2/stock'
060404 21:26:37 [ERROR] Got error 270 when reading table './dbt2/orders'
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=20
max_connections=100
threads_connected=20
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x9830a18
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xb6099d1c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x81cfd98
0x112420
(nil)
0x8328184
0x828e89b
0x828e4fa
0x828ea6e
0x828ec39
0x8294514
0x82557dd
0x8251f30
0x81eaf5c
0x81f0aef
0x81f134c
0x81f248e
0x81f2845
0xa25b80
0x97d9ce
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x983bfb0 = UPDATE stock
SET s_quantity = s_quantity - 9
WHERE s_i_id = 6240
  AND s_w_id = 2
thd->thread_id=243
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
mysqld: my_new.cc:51: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
Fatal signal 6 while backtracing
[23 Apr 2006 6:02] Jonas Oreland
this could also be the mem corruption.
removing stewart as he's on UC
[23 Apr 2006 6:04] Jonas Oreland
Another thing,
this is to my knowledge not repeatable. so the show stopper flag is a bit weird.

And I just completed my 15 hours dbt2 run wo/ problem...
[5 May 2006 12:28] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/6010
[5 May 2006 13:17] Tomas Ulin
approved
[13 May 2006 14:29] Jon Stephens
Thank you for your bug report. This issue has already been fixed
in the latest released version of that product, which you can download at 
http://www.mysql.com/downloads/

Additional info:

Documented bugfix in 5.1.10 changelog; closed.