MySQL Bugs: #34790: 'create server' doesn't handle out of memory scenario well enough

Bug #34790	'create server' doesn't handle out of memory scenario well enough
Submitted:	24 Feb 2008 14:49	Modified:	15 Apr 2008 1:41
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Federated storage engine	Severity:	S3 (Non-critical)
Version:	5.1.23	OS:	Any
Assigned to:	Sergey Vojtovich	CPU Architecture:	Any

Description:
while running the testcase for bug #34789 , mysqld crashed because it was out of memory:

Version: '5.1.23-rc-community'  socket: ''  port: 3306  MySQL Community Server (GPL)
080224 16:35:07 [ERROR] mysqld-release: Out of memory (Needed 906688 bytes)
080224 16:35:17 [ERROR] mysqld-release: Out of memory (Needed 906688 bytes)
080224 16:35:18 [ERROR] mysqld-release: Out of memory (Needed 906688 bytes)
080224 16:35:18 [ERROR] mysqld-release: Out of memory (Needed 906688 bytes)
080224 16:35:18 [ERROR] mysqld-release: Out of memory (Needed 906688 bytes)

mysqld.exe!my_utf8_uni
mysqld.exe!my_well_formed_len_mb
mysqld.exe!well_formed_copy_nchars
mysqld.exe!Field_string::store
mysqld.exe!insert_server_record
mysqld.exe!insert_server
mysqld.exe!create_server
mysqld.exe!mysql_execute_command
mysqld.exe!sp_instr_stmt::exec_core
mysqld.exe!sp_lex_keeper::reset_lex_and_exec_core
mysqld.exe!sp_instr_stmt::execute
mysqld.exe!sp_head::execute
mysqld.exe!sp_head::execute_procedure
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_threadstart

this bug is different from #34789 .  The server should return an error message such as 'out of memory', instead of crashing.

How to repeat:
I recommend a 32-bit binary to ease of repeating.
start mysqld with --key_buffer_size=1800M (to get a head start on the 2G limit).

now run:

delimiter ;
drop procedure if exists `p`;
delimiter //
create procedure `p`(`a` int)
begin
declare `c` int default 0;
repeat
   if `c` mod 25000 = 0 then
      select `c` as `iteration`;
   end if;
   drop server if exists ss;
   create server ss foreign data wrapper mysql options (user '', host '', database '');
   set `c` = `c`+1;
until `c` > `a`
end repeat;
end//
delimiter ;
call `p`(10000000);

Suggested fix:
handle the memory allocation failures gracefully.
will be easier to fix this bug before bug #34789

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/44221

ChangeSet@1.2569, 2008-03-19 16:54:14+04:00, svoj@mysql.com +1 -0
  BUG#34790 - 'create server' doesn't handle out of memory scenario
              well enough
  
  CREATE SERVER may cause server crash if there is not enough memory
  to execute this operation.
  
  Fixed that create_server() and prepare_server_struct_for_insert()
  didn't check return value of functions that allocate memory.
  
  As this is out of memory issue fix, not test case available.

I opened bug #35437 which is related to this.

Reviewers note: the same problem in ALTER SERVER is subject of BUG#35437 - potential crash with ALTER SERVER statement when out of memory.

http://lists.mysql.com/commits/44221 is Ok to push.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/44268

ChangeSet@1.2569, 2008-03-20 11:40:26+04:00, svoj@mysql.com +1 -0
  BUG#34790 - 'create server' doesn't handle out of memory scenario
              well enough
  
  CREATE SERVER may cause server crash if there is not enough memory
  to execute this operation.
  
  Fixed that create_server() and prepare_server_struct_for_insert()
  didn't check return value of functions that allocate memory.
  
  As this is out of memory issue fix, not test case available.

Pushed into 5.1.24-rc

Pushed into 6.0.5-alpha

Pushed into 5.1.23-ndb-6.3.11.

Noted in 5.1.24, 6.0.5 changelogs.

CREATE SERVER and ALTER SERVER could crash the server if
out-of-memory conditions occurred.