MySQL Bugs: #9819: Cursors: Mysql Server Crash while fetching from table with 5 million records.

Bug #9819	Cursors: Mysql Server Crash while fetching from table with 5 million records.
Submitted:	11 Apr 2005 15:44	Modified:	23 Sep 2005 16:48
Reporter:	Disha	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S2 (Serious)
Version:	5.0.3 Beta	OS:	Any (ALL)
Assigned to:	Konstantin Osipov	CPU Architecture:	Any

Description:
When we fetch data from a cursor declared on a table having 5 million rows, the MySQL server crashes and results in lost connection.

How to repeat:
1. Connect to the MySQL server as 'root'
2. Create table tb1 by executing the following query:

create table tb1 (
f1 char(100), 
f2 char binary, 
f3 char ascii, 
f12 binary, 
f13 tinyint, 
f14 tinyint unsigned, 
f15 tinyint zerofill, 
f16 tinyint unsigned zerofill, 
f17 smallint, 
f18 smallint unsigned,  
f19 smallint zerofill, 
f20 smallint unsigned zerofill, 
f21 mediumint, 
f22 mediumint unsigned, 
f23 mediumint zerofill, 
f24 mediumint unsigned zerofill, 
f25 int, 
f26 int unsigned, 
f27 int zerofill, 
f28 int unsigned zerofill, 
f29 bigint, 
f30 bigint unsigned, 
f31 bigint zerofill, 
f32 bigint unsigned zerofill, 
f33 decimal not null DEFAULT 9.9, 
f34 decimal unsigned not null DEFAULT 9.9, 
f35 decimal zerofill not null DEFAULT 9.9, 
f36 decimal unsigned zerofill not null DEFAULT 9.9, 
f37 decimal (0) not null DEFAULT 9.9, 
f38 decimal (64) not null DEFAULT 9.9, 
f39 decimal (0) unsigned not null DEFAULT 9.9, 
f40 decimal (64) unsigned not null DEFAULT 9.9, 
f41 decimal (0) zerofill not null DEFAULT 9.9, 
f42 decimal (64) zerofill not null DEFAULT 9.9, 
f43 decimal (0) unsigned zerofill not null DEFAULT 9.9, 
f44 decimal (64) unsigned zerofill not null DEFAULT 9.9, 
f45 decimal (0,0) not null DEFAULT 9.9, 
f46 decimal (63,63) not null DEFAULT 9.9, 
f47 decimal (0,0) unsigned not null DEFAULT 9.9, 
f48 decimal (63,63) unsigned not null DEFAULT 9.9, 
f49 decimal (0,0) zerofill not null DEFAULT 9.9, 
f50 decimal (63,63) zerofill not null DEFAULT 9.9, 
f51 decimal (0,0) unsigned zerofill not null DEFAULT 9.9, 
f52 decimal (63,63) unsigned zerofill not null DEFAULT 9.9, 
f53 numeric not null DEFAULT 99, 
f54 numeric unsigned not null DEFAULT 99, 
f55 numeric zerofill not null DEFAULT 99, 
f56 numeric unsigned zerofill not null DEFAULT 99, 
f57 numeric (0) not null DEFAULT 99, 
f58 numeric (64) not null DEFAULT 99
) //

2. Populate the table with 5 million rows

3. Create a second table by executing the following query:

Create table res_t2 (f1 char(100) , f2 mediumint , f3 int , f4 real , f5 numeric)//

4. Create a procedure as follows:

Create procedure sp1( )
BEGIN
	DECLARE done integer default 0;
 	DECLARE var1 char(100) ;
	DECLARE var2 mediumint;
	DECLARE var3 int ;
	DECLARE var4 real ;
 	DECLARE var5 numeric ;
	DECLARE cur1 CURSOR FOR SELECT f1,f13,f25,f33,f53  FROM tb1 ; 
	DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done = 1; 
	Open cur1;
	While done <> 1 do
		FETCH cur1 INTO var1, var2, var3, var4, var5;
		Insert into res_t2 values (var1, var2, var3, var4, var5);
	End While;
	Close cur1;
END //			

5. Call sp1()//

EXPECTED RESULTS
The execution of sp1 should complete without any errors and table res_t2 should be populated with the records. This works fine with 10000 and 500000 records.

ACTUAL RESULT
The execution of sp1 terminates, MySQL server crashes and results in a lost connection.

I don't got the server crash, just ran out the memory. I will test however
the behavior of optimized server:

c:\mysql\bin>mysql -uroot test
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 5.0.5-beta-debug

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> Create table res_t2 (f1 char(100) , f2 mediumint , f3 int , f4 real , f5
    -> numeric)//
    -> \c
mysql> delimiter //
mysql> Create table res_t2 (f1 char(100) , f2 mediumint , f3 int , f4 real , f5
    -> numeric)//
Query OK, 0 rows affected (0.11 sec)

mysql> Call sp1()//
ERROR 5 (HY000): Out of memory at line 195, 'C:\mysql-5.0.5-beta\mysys\my_alloc.c'
mysql> select count(*) from res_t2;
    -> //
+----------+
| count(*) |
+----------+
|  2039723 |
+----------+
1 row in set (0.37 sec)

mysql>

c:\mysql\bin>mysqld-debug --standalone --console
050411 19:49:10  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
050411 19:49:10  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 0 502216.
InnoDB: Doing recovery: scanned up to log sequence number 0 502216
InnoDB: Last MySQL binlog file position 0 0, file name
050411 19:49:10  InnoDB: Started; log sequence number 0 502216
050411 19:49:10  InnoDB: Starting recovery for XA transactions...
050411 19:49:10  InnoDB: 0 transactions in prepared state after recovery
050411 19:49:11 [Note] mysqld-debug: ready for connections.
Version: '5.0.5-beta-debug'  socket: ''  port: 3306  Source distribution
050411 22:53:40 [ERROR] Out of memory; check if mysqld or some other process uses all available memory; if not, you may h
ave to use 'ulimit' to allow mysqld to use more memory or you can add more swap space
050411 22:53:41 [ERROR] Out of memory; check if mysqld or some other process uses all available memory; if not, you may h
ave to use 'ulimit' to allow mysqld to use more memory or you can add more swap space

c:\mysql\bin>mysql -uroot test
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2 to server version: 5.0.5-beta-nt

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> delimiter //
mysql> Create table res_t2 (f1 char(100) , f2 mediumint , f3 int , f4 real , f5
    -> numeric)//
Query OK, 0 rows affected (0.14 sec)

mysql> Call sp1()//
ERROR 5 (HY000): Out of memory (Needed 2310412 bytes)
mysql> select count(*) from res_t2;
    -> //
+----------+
| count(*) |
+----------+
|  2183282 |
+----------+
1 row in set (1.26 sec)

c:\mysql\bin>mysqld-nt --standalone --console
050411 23:48:28  InnoDB: Started; log sequence number 0 502216
050411 23:48:28  InnoDB: Starting recovery for XA transactions...
050411 23:48:28  InnoDB: 0 transactions in prepared state after recovery
050411 23:48:29 [Note] mysqld-nt: ready for connections.
Version: '5.0.5-beta-nt'  socket: ''  port: 3306  Source distribution
050411 23:56:51 [ERROR] Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use m
ore memory or you can add more swap space
050411 23:56:54 [ERROR] Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use m
ore memory or you can add more swap space

This bug should be taken care of when we replace the current implementation of cursors in stored
procedures with the one used in the client-server protocol (among some other bugs)

When will this be done?  I'm sure there's lots waiting for a fix like we are

will it be in 5.0.7?

This bug will be fixed after Bug#10760 (dependency)

The test case provided by Disha is invalid syntax for the latest MySQL.
Using the test case provided by Miguel.

I was able to verify that the bug does not show up when using the proposed patch and Miguel's
test case. A reduced test case is added to sp-big.test

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/30173

Fixed in 5.0.13

Noted in 5.0.13 changelog.