Bug #26843 memory leak in falcon
Submitted: 5 Mar 2007 15:23 Modified: 20 Apr 2007 14:51
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S2 (Serious)
Version:5.2.4 OS:Linux (suse9.3 x86)
Assigned to: CPU Architecture:Any
Tags: falcon, memory leak

[5 Mar 2007 15:23] Shane Bester
Description:
hi, i noticed continuous memory consumption after doing some inserts into a falcon table.

mysqld is start with --skip-name-resolve --skip-grant-tables options only.

memory starts at <30M then after 10 minutes of running the testcase is at >1GB

eventually a crash happens.
this is on BK sources from 2007-03-02

How to repeat:
in connection1
----------------
delimiter //
DROP FUNCTION IF EXISTS random_string//

CREATE FUNCTION random_string(maxlen INT) RETURNS TEXT
BEGIN
        DECLARE valid_chars VARCHAR(255) DEFAULT "abcdefghijklmnopqrstuvwxyz";
        DECLARE tmpstr TEXT DEFAULT "";
        DECLARE strlen INT UNSIGNED DEFAULT 0;
        DECLARE static_rand INT UNSIGNED DEFAULT 0;
        DECLARE offset INT UNSIGNED DEFAULT 0;
        DECLARE counter INT UNSIGNED DEFAULT 0;
        SET strlen=FLOOR((RAND()*maxlen));
        REPEAT 
                SET counter = counter + 1;
                SET offset=FLOOR(1 + (RAND()* 26));
                SET tmpstr = CONCAT(tmpstr,SUBSTRING(valid_chars,offset,1));
        UNTIL counter > strlen END REPEAT;
RETURN tmpstr;
END; //

delimiter ;

drop procedure if exists p1;
drop table if exists t1;
create table t1(a int,b varchar(96),key(b),key(a))default charset=utf8 collate=utf8_unicode_ci engine=falcon;
delimiter //
create procedure p1 () 
begin 
        declare v int default 0; 
        while v < 1000000
        do
                if v mod 100 = 0 then 
                        select v; 
                end if; 
                insert into t1 values (v,random_string(255));
                set v = v + 1;
         end while;
end //

delimiter ;
call p1();

in connection 2
-------------------
call p1();

Suggested fix:
don't leak memory
[5 Mar 2007 15:39] Ann Harrison
At the moment, all uncommitted records are stored in memory.  We're working on a change that will flush records to the serial log after some limit.  It should be
available in one of the next two updates.
[5 Mar 2007 16:04] MySQL Verification Team
isn't autocommit=1 default?
[5 Mar 2007 17:03] Ann Harrison
Sorry - I missedd the autocommit state.  That's another known problem,
and being worked on.  Thanks for pointing it out.
[6 Mar 2007 12:23] MySQL Verification Team
Thank you for the bug report.
[7 Mar 2007 0:03] Hakan Küçükyılmaz
Still showing memory leak with latest change sets:

ChangeSet@1.2467, 2007-03-06 16:47:02-05:00, jas@fluffy.netfrastructure.com +2 -0
  Fix a number of record leaks.

ChangeSet@1.2466, 2007-03-06 16:02:57-05:00, jas@rowvwade. +3 -0
  Fix leak of StorageIndex objects in StorageTableShare.
[16 Apr 2007 12:00] MySQL Verification Team
I can no longer repeat this leak on today's mysql-6.0.0-falcon BK build.  Memory after 15 minutes of running testcase was at 60M instead of >1GB. Looks fixed.
[16 Apr 2007 14:48] Hakan Küçükyılmaz
Probably related to Kevin's last memory leak fix.

Great!
[16 Apr 2007 15:08] Kevin Lewis
Well I ran this test today with break points on calls to chill and the recent fix for BLOB memory leaks, and this test does not hit those.  So it is not something I did.  But I can confirm that the memory is pretty stable.  I think it is related to a fix for recordVersion reference counting that Jim made a couple weeks ago.
[16 Apr 2007 16:50] Hakan Küçükyılmaz
Documenting not Duplicate! Sorry for confusion.
[20 Apr 2007 14:51] MC Brown
A note has been added to the 5.2.4 changelog.
[10 Jul 2007 19:08] MC Brown
This bug report entry has been moved to the 6.0.0 Falcon changelog.