Bug #52832 Random lock up - Lock wait timeout exceeded - NDBCluster
Submitted: 14 Apr 2010 18:46 Modified: 6 Aug 2010 8:51
Reporter: Dean Elzey Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S1 (Critical)
Version:5.1.35-ndb-7.0.7 OS:Linux (CentOS 5.3)
Assigned to: CPU Architecture:Any
Tags: lock wait timeout exceeded, ndbcluster

[14 Apr 2010 18:46] Dean Elzey
Description:
Server version: 5.1.35-ndb-7.0.7-cluster-gpl-log MySQL Cluster Server (GPL)

Database server seems to sit at a "Lock wait timeout exceeded" error, restarting the service helps however I have to killall -9 the service as just normal stop will not work.

Database always locks while accessing the same table, `pl_user_sessions` at different times.

Here is the DDL and sample Data:

CREATE TABLE IF NOT EXISTS `pl_user_sessions` (
  `session_id` varchar(127) NOT NULL,
  `last_activity` int(10) unsigned NOT NULL,
  `data` text NOT NULL,
  PRIMARY KEY (`session_id`)
) ENGINE=ndbcluster DEFAULT CHARSET=utf8 PACK_KEYS=0 ROW_FORMAT=COMPACT;

INSERT INTO `pl_user_sessions` (`session_id`, `last_activity`, `data`) VALUES
('oq9lqb7sg3d112higqcv45diq7', 1271247851, 'c2Vzc2lvbl9pZHxzOjI2OiJvcTlscWI3c2czZDExMmhpZ3FjdjQ1ZGlxNyI7dG90YWxfaGl0c3xpOjc2Mztfa2ZfZmxhc2hffGE6MDp7fXVzZXJfYWdlbnR8czoxMDk6Ik1vemlsbGEvNS4wIChXaW5kb3dzOyBVOyBXaW5kb3dzIE5UIDYuMDsgZW4tVVM7IHJ2OjEuOS4xLjkpIEdlY2tvLzIwMTAwMzE1IEZpcmVmb3gvMy41LjkgKC5ORVQgQ0xSIDMuNS4zMDcyOSkiO2lwX2FkZHJlc3N8czo5OiIxMjcuMC4wLjEiO2xhc3RfYWN0aXZpdHl8aToxMjcxMjQ3ODUxO3FhX2RhdGF8YTozOntzOjEzOiJsYXN0X2ludGVydmFsIjtzOjE2OiIwNC0xMi0yMDEwIDEyOjE3IjtzOjEwOiJsYXN0X2xvZ2luIjtzOjE2OiIwNC0xMy0yMDEwIDA4OjM0IjtzOjEwOiJzdGF0X2RyaXZlIjtpOjI7fWF1dGhfdXNlcnxPOjEwOiJVc2VyX01vZGVsIjo2OntzOjE0OiIAKgBvYmplY3RfbmFtZSI7czo0OiJ1c2VyIjtzOjk6IgAqAG9iamVjdCI7YTo0Nzp7czoyOiJpZCI7aToyO3M6NToiZW1haWwiO3M6MjA6Imh3ZHVtbXlAaGVhdHdhdmUuY29tIjtzOjg6InVzZXJuYW1lIjtzOjY6ImRlbHpleSI7czo4OiJwYXNzd29yZCI7czo1MDoiZmQ3NjBjZWI3ZTlmNzdkYTczNTYwZDA2ZWY2ZGY0MzE2ZjNhYzQ3NzM2Y2NiMDMzMWUiO3M6NjoibG9naW5zIjtpOjM5O3M6MTA6Imxhc3RfbG9naW4iO2k6MTI3MTE2NTY2MTtzOjEzOiJsYXN0X2ludGVydmFsIjtpOjEyNzEwOTI2Njg7czoxMDoibGFzdF9wYXJ0eSI7aToxMjU4OTkxNzI1O3M6MTE6ImZhY2Vib29rX2lkIjtOO3M6NDoibmFtZSI7czo3OiJIVyBEZWFuIjtzOjc6InJvbGVfaWQiO2k6MztzOjY6ImdlbmRlciI7aToxO3M6MTE6ImhvbWVfcmVnaW9uIjtpOjM7czoxMjoicGFwZXJkb2xsX2lkIjtpOjM7czo5OiJzdGF0X2Nhc2giO2k6OTU4MztzOjk6InN0YXRfY3JlZCI7ZDoxMDUuNTtzOjEzOiJzdGF0X2NyZWRfbWF4IjtkOjEwNS41O3M6MTA6InN0YXRfanVpY2UiO2Q6MTA1O3M6MTQ6InN0YXRfanVpY2VfbWF4IjtkOjEwNTtzOjk6InN0YXRfZmFtZSI7ZDoxMTk3O3M6MTQ6InN0YXRfZmFtZV9iYXNlIjtkOjcyODtzOjEzOiJzdGF0X2ZhbWVfbWF4IjtkOjE3MzM7czoxMDoic3RhdF92aWRlbyI7ZDowO3M6MTQ6InN0YXRfcmVjb3JkaW5nIjtkOjA7czoxNjoic3RhdF9wZXJmb3JtYW5jZSI7ZDowO3M6MTE6InN0YXRfc29jaWFsIjtkOjA7czoxMDoic3RhdF9kcml2ZSI7aToyO3M6MTA6InN0YXRfbGV2ZWwiO2k6MjtzOjk6InN0YXRfcmFuayI7aToxO3M6MTA6InN0YXRfZmF2b3IiO2k6MTkwMDtzOjk6InN0YXRfYmFuayI7aTowO3M6MTA6ImJhbmRfbGV2ZWwiO2k6MDtzOjEyOiJiYW5kX21hbmFnZXIiO2k6MztzOjEzOiJiYW5kX2h5cGVfbWFuIjtpOjEyO3M6MTQ6ImJhbmRfZnJvbnRfbWFuIjtpOjk7czoxODoiYmFuZF9iYWNrdXBfdm9jYWxzIjtpOjQ7czoxNToiYmFuZF9taXhfbWFzdGVyIjtpOjU7czoxMDoiZ2Vhcl92aWRlbyI7ZDowO3M6MTQ6ImdlYXJfcmVjb3JkaW5nIjtkOjA7czoxNjoiZ2Vhcl9wZXJmb3JtYW5jZSI7ZDowO3M6MTE6ImdlYXJfc29jaWFsIjtkOjA7czoxMDoic3RhdF9jaGlwcyI7aTowO3M6MTQ6Imxhc3Rfc3RhdF9jYWxjIjtpOjEyNzEyNDc2MDA7czoxNjoiYXZhaWxhYmxlX3BvaW50cyI7aTo1O3M6MTA6InBsYXlfYXVkaW8iO2k6MDtzOjY6InN0YXR1cyI7czo1OiJSRUFEWSI7czoxMjoiYWZmaWxpYXRlX2lkIjtpOjE7fXM6MTA6IgAqAGNoYW5nZWQiO2E6MDp7fXM6OToiACoAbG9hZGVkIjtiOjE7czo4OiIAKgBzYXZlZCI7YjoxO3M6MTA6IgAqAHNvcnRpbmciO2E6MTp7czoyOiJpZCI7czozOiJhc2MiO319');

The logs from running nbd_error_reporter are attached

How to repeat:
Kill mysqld services, wait less than 24 hours

Suggested fix:
unknown
[6 Jul 2010 8:43] Andrew Hutchings
Hello Dean,

Do you have any transactions on this table that run for long periods of time (more than a couple of seconds), even if they are just contain selects?
[6 Aug 2010 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".