Bug #71127 Checkpoint routine can lead to deadlock
Submitted: 11 Dec 2013 11:51 Modified: 19 Dec 2013 11:32
Reporter: Mats Kindahl Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Fabric Severity:S3 (Non-critical)
Version:1.4.0 OS:Any
Assigned to: CPU Architecture:Any

[11 Dec 2013 11:51] Mats Kindahl
Description:
The checkpoint routine is causing deadlock issues:

<Fault 1: '<class \'mysql.fabric.errors.DatabaseError\'>:(\'Command (INSERT
INTO checkpoints(proc_uuid, lockable_objects, job_uuid, sequence, action_fqn,
param_args, param_kwargs) SELECT %s, %s, %s, coalesce(MAX(sequence), 0) + 1,
%s, %s, %s FROM checkpoints WHERE proc_uuid = %s,
(\\\'3693045d-3a2f-4b78-b2ab-1e388c62774e\\\',
"c__builtin__\\\\nset\\\\np0\\\\n((lp1\\\\nS\\\'lock\\\'\\\\np2\\\\natp3\\\\nR
p4\\\\n.", \\\'009cac0b-0437-4ce5-8897-38eaa39d7f1a\\\',
\\\'mysql.fabric.services.server._lookup_servers\\\',
"(S\\\'group_id\\\'\\\\np0\\\\nNNtp1\\\\n.", \\\'(dp0\\\\n.\\\',
\\\'3693045d-3a2f-4b78-b2ab-1e388c62774e\\\')) failed: 1213 (40001): Deadlock
found when trying to get lock; try restarting transaction\', 1213)'> <class
'xmlrpclib.Fault'>

How to repeat:
Run the following program to repeat the problem:

import xmlrpclib
import time
import random
import threading
import traceback

def run(url, thd):
    client = xmlrpclib.ServerProxy(url)

    while True:
        try:
            client.group.lookup_servers("group_id")
        except Exception as error:
            print thd, "accessing error", error, type(error)
            traceback.print_exc()
        sleep = random.random() * 2
        time.sleep(sleep)

if __name__ == "__main__":
    url = "http://localhost:8080"

    client = xmlrpclib.ServerProxy(url)
    client.group.create("group_id")

    for nt in range(0, 25):
        name = "run-%s" % (nt, )
        thread = threading.Thread(
            target=run,
            name=name,
            args=(url, name)
        )
        thread.start()

Suggested fix:
Simplify the queries.
[19 Dec 2013 11:35] Mats Kindahl
Thank you for your bug report. This issue has already been fixed in the latest released version of that product, which you can download at

  http://www.mysql.com/downloads/