MySQL Bugs: #25537: Running UPDATEs reveals memory leak

Bug #25537	Running UPDATEs reveals memory leak
Submitted:	10 Jan 2007 23:42	Modified:	8 Feb 2007 16:55
Reporter:	Hakan Küçükyılmaz	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S2 (Serious)
Version:	5.2.1-falcon-alpha	OS:	Linux (Linux/Windows)
Assigned to:	Kevin Lewis	CPU Architecture:	Any

Description:
Running UPDATES in a tight loop inside a stored procedure reveals memory leak.

How to repeat:
Use following script.

SET @@storage_engine = Falcon;
#
# Adapted from falcon_bug_23818_A
#

DROP TABLE IF EXISTS t1;
DROP PROCEDURE IF EXISTS p1;

CREATE TABLE t1 (a int, b varchar(500));
INSERT INTO t1 VALUES (0,null),(0,null),(0,null);
CREATE INDEX i1 ON t1 (b);

DELIMITER //
CREATE PROCEDURE p1 ()
BEGIN
  DECLARE a int DEFAULT 32710;
  DECLARE b int DEFAULT 0;
  DECLARE c int DEFAULT 0;
  DECLARE v int DEFAULT 0;
  DECLARE CONTINUE HANDLER FOR SQLEXCEPTION BEGIN END;
  while v < 1000000 do
    START TRANSACTION;
    SELECT a + 1 into a;
    SELECT rand(a) * DEGREES(a) into b;
    SELECT FLOOR(MOD(b,255)) into c;
    UPDATE t1 SET b = repeat(hex(c), rand(c) * 250);
    COMMIT;
    SELECT v;
    SET v = v + 1;
  end while;
END//

DELIMITER ;

Run p1() in 5 connections and look for memory usage.

This is definitely a concurrency memory leak.  The Transaction, Savepoint, and DeferredIndex objects are not getting released when several of these stored procedures are running at the same time. Once the other SPs are done, the one remaining can run over and over without increasing memory.

Pushed changeset to fix this memory leak in Falcon.

The main changes are;

1) Set Transaction::hasUpdates only when the transaction really receives changes that may need to be committed.
  A. When a record object is added to the transaction.
  B. When an index is created.
  C. When Database::deleteSection is called for a transaction.
This flag lets Transaction::commit call SRLCommit::append() to add a commit record to the serial log.  The gopher thread will see this and call Transaction::writeComplete(void) when it is done with the transaction.
hasUpdates in NOT set when a single node is added to a DeferredIndex.  We wait until the associated record is added to the transaction.

2) To make sure that writeComplete can find the transaction pointer, it is sent into SRLCommit::append() to be stored in the associated SerialLogTransaction.

3) Back in Transaction::commit(), if hasUpdates is false, instead of just setting WritePending to false, call writeComplete() to do that, which has the added benefit of droppong all deferredIndexes.  The index might contain an index entry after an update conflict since the indexes are updated first.

pushed into 5.2.2.

it should be 5.2.3.

A note has been added to the 5.2.3 changelog.