Bug #53998 yassl code in Data::Process can allocate negative memory size (wraps to 2^32)
Submitted: 26 May 2010 14:59 Modified: 29 Jul 2011 23:09
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server: C API (client library) Severity:S1 (Critical)
Version:5.1.46 OS:Windows
Assigned to: CPU Architecture:Any

[26 May 2010 14:59] Shane Bester
Description:
random crashes in the yassl client libs happen here 
(see attachment for full stack):
                                                                          
libmysql.dll!_heap_alloc_dbg                                              
libmysql.dll!_nh_malloc_dbg                                               
libmysql.dll!malloc                                                       
libmysql.dll!operator new                                                 
libmysql.dll!yaSSL::input_buffer::input_buffer(unsigned int s=4294967286) 
libmysql.dll!yaSSL::Data::Process                                         
libmysql.dll!yaSSL::DoProcessReply                                        
libmysql.dll!yaSSL::processReply                                          
libmysql.dll!yaSSL::receiveData                                           
libmysql.dll!yaSSL_read                                                   
<cut>                                                                     

The problem is here:
-----
int dataSz = msgSz - ivExtra - digestSz - pad - padByte;   
...
// read data
    if (dataSz) {
..
     ssl.addData(data = NEW_YS input_buffer(dataSz));
-----

the problem is dataSz was negative since:
msgSz	48	int
padByte	1	int
pad	37	int
ivExtra	0	int
digestSz	20	int
dataSz	-10	int

How to repeat:
just establish many connections, perhaps with some network timeouts/errors in between so that a smaller than expected payload is delivered ?

Suggested fix:
reject such packets.
for now, changing:

if (dataSz) {

to

if (dataSz > 0) {

can be a good idea...
[26 May 2010 15:01] MySQL Verification Team
uncut client stack trace

Attachment: bug53998_full_stack.txt (text/plain), 2.06 KiB.

[27 May 2010 10:30] MySQL Verification Team
actually, i think the client side libs will also suffer due to the lock of locking (bug #34236). so this should be retested with libs build with -DMULTI_THREADED ...
[28 Apr 2011 16:23] Valeriy Kravchuk
In current mysql-5.1 code in question is different:

    // read data
    if (dataSz) {                               // could be compressed
        if (ssl.CompressionOn()) {
            input_buffer tmp;
            if (DeCompress(input, dataSz, tmp) == -1) {
                ssl.SetError(decompress_error);nk that we still have a bug here?
                return;
            }
            ssl.addData(NEW_YS input_buffer(tmp.get_size(),
                                            tmp.get_buffer(), tmp.get_size()));
        }
...

Do you think that we still have a bug here?
[28 May 2011 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[29 Jun 2011 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[30 Jul 2011 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".