Bug #36705 key_buffer_size of >= 2G allocates all available virtual memory on 64-bit windo

Description:
This bug causes all available virtual memory to be exhausted:

E:\mysql-enterprise-gpl-5.0.56-winx64\bin>mysqld-nt --console --key_buffer_size=2048M

mysqld-nt: Out of memory (Needed 318145536 bytes)
mysqld-nt: Out of memory (Needed 2386092032 bytes)

<cut>

Just below 2G works:

E:\mysql-enterprise-gpl-5.0.56-winx64\bin>mysqld-nt --console --key_buffer_size=2047M
080514  4:58:48 [Note] mysqld-nt: ready for connections.
Version: '5.0.56-enterprise-gpl-nt'  socket: ''  port: 3306  MySQL Enterprise Server (GPL)

5.1.24 has no such problems like this.

How to repeat:
download and extract 64-bit no-install version of 5.0.x
mysqld-nt --console --skip-innodb --key_buffer_size=2048M

in the above description, --key_buffer_size=2048M caused over 9GB of virtual memory to be used up.

verified on 5.0.60-winx64

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46799

ChangeSet@1.2630, 2008-05-17 12:53:47+05:00, holyfoot@mysql.com +1 -0
  Bug #36705 key_buffer_size of >= 2G allocates all availabel virtual memory on 64-bit wondo.
  
  temporary variables of 'long' types were used to store ulong values,
  that causes init_key_cache to receive distorted parameters

Pushed into 5.0.64

Pushed into 5.1.25-rc

Noted in 5.0.64, 5.1.25 changelogs.

On WIndows 64-bit systems, temporary variables of long types were
used to store ulong values, causing key cache initialization to
receive distorted parameters. The effect was that setting
key_buffer_size to values of 2GB or more caused memory exhaustion to
due allocation of too much memory.

Setting report to Need Doc Info pending push into 6.0.x.

Pushed into 6.0.6-alpha

Noted in 6.0.6 changelog.

Pushed into 5.0.64  (revid:sp1r-holyfoot/hf@mysql.com/hfmain.(none)-20080517075347-24047) (pib:2)
(Retry automatic marking, to ensure nothing is missed. cm01)

Pushed into 5.1.25-rc  (revid:sp1r-gkodinov/kgeorge@magare.gmz-20080518084255-27675) (version source revid:sp1r-gkodinov/kgeorge@magare.gmz-20080518084255-27675) (pib:3)