Bug #43572 Handle failures from hash_init
Submitted: 11 Mar 2009 18:11 Modified: 8 Aug 2009 1:10
Reporter: Mark Callaghan Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:5.0, 5.1, 6.0 bzr OS:Any
Assigned to: V Venkateswaran CPU Architecture:Any
Tags: error, hash_init, IGNORE, return

[11 Mar 2009 18:11] Mark Callaghan
Description:
About half of the calls to hash_init ignore the return value. This call can fail and the callers must deal with that. Even NDB ignores errors in some cases for this. Without checks it is very risky to have a server get close to out of memory.

How to repeat:
read the code

Suggested fix:
Check the return value from hash_init
[12 Mar 2009 7:09] Sveta Smirnova
Thank you for the report.

Verified as described.
[27 Apr 2009 8:25] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72828

2872 Narayanan V	2009-04-27
      Bug#43572 Handle failures from hash_init
      
      calls to hash_init ignore the return value. 
      This call can fail and the callers must deal
      with that.
      
      The current patch fixes instances of hash_init
      usages which ignore the return value.
[4 Jun 2009 5:04] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75576

2764 Narayanan V	2009-06-04
      Bug#43572 Handle failures from hash_init
      
      Failure to allocate memory for the hash->array element,
      caused hash_init to return without allocating the other
      members of the hash. Thus although the dynamic array
      could be initialized at a later point in the code, the
      incompletely allocated hash caused fatal failures.
      
      This patch moves the initialization of the other members
      of the hash above the array allocation, so that the usage
      of this hash will not result in fatal failures.
     @ mysys/hash.c
        Bug#43572 Handle failures from hash_init
        
        Move the initialization of the other members
        of the hash above the array allocation, so that
        the usage of this hash will not result in fatal
        failures.
[6 Jun 2009 5:10] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75768

2771 Narayanan V	2009-06-06
      Bug#43572 Handle failures from hash_init
      
      Failure to allocate memory for the hash->array element,
      caused hash_init to return without allocating the other
      members of the hash. Thus although the dynamic array
      could be initialized at a later point in the code, the
      incompletely allocated hash caused fatal failures.
      
      This patch moves the initialization of the other members
      of the hash above the array allocation, so that the usage
      of this hash will not result in fatal failures.
     @ include/hash.h
        Bug#43572 Handle failures from hash_init
        
        hash_inited is used to verify that the hash is
        valid. After the change induced by the current
        patch hash->array.buffer !=0 is not a valid check
        for this condition, since, the dynamic array can
        be allocated even at a later time. Bootstrap SQL
        script is setting some variables, which are
        actually not set due to this hash_inited issue.
        Thus we get empty grant tables.
        
        A better way to check if the hash is valid is
        to verify that hash->blength is greater than 0.
     @ mysys/hash.c
        Bug#43572 Handle failures from hash_init
        
        Move the initialization of the other members
        of the hash above the array allocation, so that
        the usage of this hash will not result in fatal
        failures.
[17 Jun 2009 9:47] Ingo Strüwing
Approved. I have been told that a new bug for the my_hash_insert() problem will be reported and tests will be added during up-merge to 6.0.
[19 Jun 2009 10:47] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/76661

2780 V Narayanan	2009-06-19
      Bug#43572 Handle failures from hash_init
            
      Failure to allocate memory for the hash->array element,
      caused hash_init to return without initializing the other
      members of the hash. Thus although the dynamic array
      buffer may be allocated at a later point in the code, the
      incompletely initialized hash caused fatal failures.
      
      This patch moves the initialization of the other members
      of the hash above the array allocation, so that the usage
      of this hash will not result in fatal failures.
     @ include/hash.h
        Bug#43572 Handle failures from hash_init
        
        hash_inited is used to verify that the hash is
        valid. After the change induced by the current
        patch hash->array.buffer !=0 is not a valid check
        for this condition, since, the dynamic array can
        be allocated even at a later time. Bootstrap SQL
        script is setting some variables, which are
        actually not set due to this hash_inited issue.
        Thus we get empty grant tables.
        
        A better way to check if the hash is valid is
        to verify that hash->blength is greater than 0.
     @ mysys/hash.c
        Bug#43572 Handle failures from hash_init
        
        Move the initialization of the other members
        of the hash above the array allocation, so that
        the usage of this hash will not result in fatal
        failures.
[19 Jun 2009 12:01] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/76676

2780 V Narayanan	2009-06-19
      Bug#43572 Handle failures from hash_init
            
      Failure to allocate memory for the hash->array element,
      caused hash_init to return without initializing the other
      members of the hash. Thus although the dynamic array
      buffer may be allocated at a later point in the code, the
      incompletely initialized hash caused fatal failures.
      
      This patch moves the initialization of the other members
      of the hash above the array allocation, so that the usage
      of this hash will not result in fatal failures.
     @ include/hash.h
        Bug#43572 Handle failures from hash_init
        
        hash_inited is used to verify that the hash is
        valid. After the change induced by the current
        patch hash->array.buffer !=0 is not a valid check
        for this condition, since, the dynamic array can
        be allocated even at a later time. Bootstrap SQL
        script is setting some variables, which are
        actually not set due to this hash_inited issue.
        Thus we get empty grant tables.
        
        A better way to check if the hash is valid is
        to verify that hash->blength is greater than 0.
     @ mysys/hash.c
        Bug#43572 Handle failures from hash_init
        
        Move the initialization of the other members
        of the hash above the array allocation, so that
        the usage of this hash will not result in fatal
        failures.
[19 Jun 2009 12:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/76678

2780 V Narayanan	2009-06-19
      Bug#43572 Handle failures from hash_init
            
      Failure to allocate memory for the hash->array element,
      caused hash_init to return without initializing the other
      members of the hash. Thus although the dynamic array
      buffer may be allocated at a later point in the code, the
      incompletely initialized hash caused fatal failures.
      
      This patch moves the initialization of the other members
      of the hash above the array allocation, so that the usage
      of this hash will not result in fatal failures.
     @ include/hash.h
        Bug#43572 Handle failures from hash_init
        
        hash_inited is used to verify that the hash is
        valid. After the change induced by the current
        patch hash->array.buffer !=0 is not a valid check
        for this condition, since, the dynamic array can
        be allocated even at a later time. Bootstrap SQL
        script is setting some variables, which are
        actually not set due to this hash_inited issue.
        Thus we get empty grant tables.
        
        A better way to check if the hash is valid is
        to verify that hash->blength is greater than 0.
     @ mysys/hash.c
        Bug#43572 Handle failures from hash_init
        
        Move the initialization of the other members
        of the hash above the array allocation, so that
        the usage of this hash will not result in fatal
        failures.
[22 Jun 2009 11:02] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/76803

2968 V Narayanan	2009-06-22
      Bug#43572  Handle failures from hash_init
      
      The merge from mysql-5.0 caused the growth_size parameter to the
      my_init_dynamic_array function to be ignored. This patche corrects
      the problem.
     @ mysys/hash.c
        Bug#43572  Handle failures from hash_init
        
        Replacing the last parameter to my_init_dynamic_array with
        growth_size.
[22 Jun 2009 11:10] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/76805

2968 V Narayanan	2009-06-22
      Bug#43572  Handle failures from hash_init
      
      The merge from http://lists.mysql.com/commits/76678 caused the 
      growth_size parameter to the my_init_dynamic_array function to
      be ignored. This patch corrects the problem.
     @ mysys/hash.c
        Bug#43572  Handle failures from hash_init
        
        Replacing the last parameter to my_init_dynamic_array with
        growth_size.
[24 Jun 2009 8:57] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/77002

3391 V Narayanan	2009-06-24
      Bug#43572 Handle failures from hash_init
      
      The bug needs to be tested with error injection. This
      patch adds error injection tests, for, testing failure
      of hash_init, caused by memory allocation failure in
      the dynamic array used in the hash.
     @ mysql-test/r/hash_error_injection.result
        Bug#43572 Handle failures from hash_init
        
        Result file for error injection testing in hash_init
     @ mysql-test/t/hash_error_injection.test
        Bug#43572 Handle failures from hash_init
        
        sets the debug flag to simulate a dynamic array allocation
        failure while initializing the hash. After setting the flag
        create hash indexes on memory tables and test the queries to
        ensure that, failure in the dynamic array allocation in hash_init
        does not causes these queries, that exercise the hash_init 
        function to fail.
     @ mysys/hash.c
        Bug#43572 Handle failures from hash_init
        
        Contains the changes needed to add error injection
        in the hash_init function. Test a debug flag to determine
        if the dynamic array needs to be reset, thus, simulating
        an memory allocation failure.
[7 Jul 2009 7:52] Bugs System
Pushed into 5.0.84 (revid:joro@sun.com-20090707074938-ksah1ibn0vs92cem) (version source revid:v.narayanan@sun.com-20090619115921-zw681o7tstw462mn) (merge vers: 5.0.84) (pib:11)
[8 Jul 2009 13:30] Bugs System
Pushed into 5.1.37 (revid:joro@sun.com-20090708131116-kyz8iotbum8w9yic) (version source revid:joro@sun.com-20090622115751-e2946ixgjf73narz) (merge vers: 5.1.37) (pib:11)
[9 Jul 2009 7:35] Bugs System
Pushed into 5.0.84 (revid:joro@sun.com-20090707074938-ksah1ibn0vs92cem) (version source revid:v.narayanan@sun.com-20090619115921-zw681o7tstw462mn) (merge vers: 5.0.84) (pib:11)
[9 Jul 2009 7:38] Bugs System
Pushed into 5.1.37 (revid:joro@sun.com-20090708131116-kyz8iotbum8w9yic) (version source revid:joro@sun.com-20090622115751-e2946ixgjf73narz) (merge vers: 5.1.37) (pib:11)
[9 Jul 2009 10:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/78269

2800 V Narayanan	2009-07-09
      Bug#43572 Handle failures from hash_init
      
      This patch is a follow up to http://lists.mysql.com/commits/76678.
      When an allocation failure occurs for the buffer in the dynamic
      array, an error condition was being set. The dynamic array is
      usable even if the memory allocation fails. Since in most cases
      the thread can continue to work without any problems the error
      condition should not be set here.
      
      This patch adds logic to remove the error condition from being set
      when the memory allocation for the buffer in dynamic array fails.
     @ mysys/array.c
        Bug#43572 Handle failures from hash_init
        
        Remove the MY_WME flag from the call to malloc in order to
        prevent the error status from being set.
[10 Jul 2009 11:20] Bugs System
Pushed into 5.4.4-alpha (revid:anozdrin@bk-internal.mysql.com-20090710111017-bnh2cau84ug1hvei) (version source revid:joro@sun.com-20090622120415-86a98nwfqojcuqe8) (merge vers: 5.4.4-alpha) (pib:11)
[12 Jul 2009 5:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/78461

2804 V Narayanan	2009-07-12
      Bug#43572 Handle failures from hash_init
      
      This patch is a follow up to http://lists.mysql.com/commits/76678.
      When an allocation failure occurs for the buffer in the dynamic
      array, an error condition was being set. The dynamic array is
      usable even if the memory allocation fails. Since in most cases
      the thread can continue to work without any problems the error
      condition should not be set here.
      
      This patch adds logic to remove the error condition from being set
      when the memory allocation for the buffer in dynamic array fails.
     @ mysys/array.c
        Bug#43572 Handle failures from hash_init
        
        Remove the MY_WME flag from the call to malloc in order to
        prevent the error status from being set in the init_dynamic_array
        method. Since this memory allocation failure is no longer
        fatal this method has been modified to return FALSE
        (indicate success) irrespective of array->buffer being
        allocated.
[4 Aug 2009 13:58] Bugs System
Pushed into 5.0.85 (revid:davi.arnaut@sun.com-20090804135315-6lfdnk4zjwk7kn7r) (version source revid:davi.arnaut@sun.com-20090804135315-6lfdnk4zjwk7kn7r) (merge vers: 5.0.85) (pib:11)
[4 Aug 2009 19:52] Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090804194615-h40sa098mx4z49qg) (version source revid:satya.bn@sun.com-20090727062228-uf93hg5izc1lpe3z) (merge vers: 5.4.4-alpha) (pib:11)
[4 Aug 2009 20:45] Bugs System
Pushed into 5.1.38 (revid:davi.arnaut@sun.com-20090804204317-ggodqkik7de6nfpz) (version source revid:davi.arnaut@sun.com-20090804204317-ggodqkik7de6nfpz) (merge vers: 5.1.38) (pib:11)
[8 Aug 2009 1:10] Paul DuBois
Noted in 5.0.85, 5.1.38, 5.4.4 changelogs.

The server did not always check the return value of calls to the
hash_init() function.
[12 Aug 2009 22:41] Paul DuBois
Noted in 5.4.2 changelog because next 5.4 version will be 5.4.2 and not 5.4.4.
[15 Aug 2009 1:57] Paul DuBois
Ignore previous comment about 5.4.2.
[26 Aug 2009 13:46] Bugs System
Pushed into 5.1.37-ndb-7.0.8 (revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[26 Aug 2009 13:46] Bugs System
Pushed into 5.1.37-ndb-6.3.27 (revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (version source revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (merge vers: 5.1.37-ndb-6.3.27) (pib:11)
[26 Aug 2009 13:48] Bugs System
Pushed into 5.1.37-ndb-6.2.19 (revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (version source revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (merge vers: 5.1.37-ndb-6.2.19) (pib:11)
[27 Aug 2009 16:32] Bugs System
Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090827163030-6o3kk6r2oua159hr) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[1 Oct 2009 5:58] Bugs System
Pushed into 5.1.39-ndb-6.3.28 (revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (version source revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (merge vers: 5.1.39-ndb-6.3.28) (pib:11)
[1 Oct 2009 7:25] Bugs System
Pushed into 5.1.39-ndb-7.0.9 (revid:jonas@mysql.com-20091001072547-kv17uu06hfjhgjay) (version source revid:jonas@mysql.com-20091001071652-irejtnumzbpsbgk2) (merge vers: 5.1.39-ndb-7.0.9) (pib:11)
[1 Oct 2009 13:25] Bugs System
Pushed into 5.1.39-ndb-7.1.0 (revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (version source revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (merge vers: 5.1.39-ndb-7.1.0) (pib:11)
[5 Oct 2009 10:50] Bugs System
Pushed into 5.1.39-ndb-6.2.19 (revid:jonas@mysql.com-20091005103850-dwij2dojwpvf5hi6) (version source revid:jonas@mysql.com-20090930185117-bhud4ek1y0hsj1nv) (merge vers: 5.1.39-ndb-6.2.19) (pib:11)
[8 Oct 2009 2:48] Paul DuBois
The 5.4 fix has been pushed to 5.4.2.