MySQL Bugs: #29247: Double free in libmysqlclient

Bug #29247	Double free in libmysqlclient_r when mysqld restarted
Submitted:	20 Jun 2007 17:46	Modified:	11 Jul 2007 22:37
Reporter:	Andrew Agno	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: C API (client library)	Severity:	S3 (Non-critical)
Version:	5.0.37, 5.0.41, 5.0.46-BK	OS:	Linux (Ubuntu 7.04 64 bit)
Assigned to:	Alexey Botchkov	CPU Architecture:	Any
Tags:	FREE, my_no_flags_free, mysql_real_connect, reconnect, segfault, segv

Description:
I get a segfault when I run a fairly simple program that makes a query of the form "SELECT count(*) from <table>", repeatedly.  On the server is a loop:
while [ 1 ]; do /etc/init.d/mysqld restart; sleep 5; done

Running under GDB, with the mysqlclient library built from scratch with debugging enabled, I get:
======= Backtrace: =========
/lib/libc.so.6[0x2b0b01160b23]
/lib/libc.so.6(cfree+0x8c)[0x2b0b0116426c]
/tmp/mysql-5.0.37-install/lib/mysql/libmysqlclient_r.so.15(my_no_flags_free+0x84)[0x2b0b001acdab]
/tmp/mysql-5.0.37-install/lib/mysql/libmysqlclient_r.so.15(mysql_real_connect+0x23e)[0x2b0b001ea601]
/tmp/mysql-5.0.37-install/lib/mysql/libmysqlclient_r.so.15(mysql_reconnect+0x230)[0x2b0b001ec3b1]
/tmp/mysql-5.0.37-install/lib/mysql/libmysqlclient_r.so.15(cli_advanced_command+0x7b)[0x2b0b001e7d69]
/tmp/mysql-5.0.37-install/lib/mysql/libmysqlclient_r.so.15(mysql_send_query+0x1a3)[0x2b0b001ed263]
/tmp/mysql-5.0.37-install/lib/mysql/libmysqlclient_r.so.15(mysql_real_query+0xc0)[0x2b0b001ed32e]
/tmp/reconnect(__gxx_personality_v0+0x35b)[0x4011ab]
/tmp/reconnect[0x4014b8]
/lib/libc.so.6(__libc_start_main+0xf4)[0x2b0b0110e8e4]
/tmp/reconnect(__gxx_personality_v0+0x59)[0x400ea9]

Also,
(gdb) where
#0  0x00002b0b01121cab in raise () from /lib/libc.so.6
#1  0x00002b0b01123660 in abort () from /lib/libc.so.6
#2  0x00002b0b0115966b in ?? () from /lib/libc.so.6
#3  0x00002b0b01160b23 in ?? () from /lib/libc.so.6
#4  0x00002b0b0116426c in free () from /lib/libc.so.6
#5  0x00002b0b001acdab in my_no_flags_free (ptr=0x6188d0 "") at my_malloc.c:59
#6  0x00002b0b001ea601 in mysql_real_connect (mysql=0x7fffaab46280, host=0x618860 "127.0.0.1", user=0x602bd0 "user", passwd=0x602bf0 "password", db=0x6188b0 "db", port=3306,
    unix_socket=0x0, client_flag=2147525261) at client.c:1826
#7  0x00002b0b001ec3b1 in mysql_reconnect (mysql=0x7fffaab46960) at client.c:2458
#8  0x00002b0b001e7d69 in cli_advanced_command (mysql=0x7fffaab46960, command=COM_QUERY, header=0x0, header_length=0, arg=0x618908 "SELECT count(*) from nodes", arg_length=26, skip_check=1 '\001',
    stmt=0x0) at client.c:672
#9  0x00002b0b001ed263 in mysql_send_query (mysql=0x7fffaab46960, query=0x618908 "SELECT count(*) from nodes", length=26) at client.c:2763
#10 0x00002b0b001ed32e in mysql_real_query (mysql=0x7fffaab46960, query=0x618908 "SELECT count(*) from nodes", length=26) at client.c:2774
#11 0x00000000004011ab in query (mysql=0x7fffaab46960, sql=@0x7fffaab46940) at TestReconnect.cpp:47
#12 0x00000000004014b8 in main (argc=1, argv=0x7fffaab46f48) at TestReconnect.cpp:70
(gdb) up 5
#5  0x00002b0b001acdab in my_no_flags_free (ptr=0x6188d0 "") at my_malloc.c:59
59          free(ptr);
Current language:  auto; currently c
(gdb) print ptr
$2 = 0x6188d0 ""

How to repeat:
Run the following program:
#include <mysql/mysql.h>
#include <iostream>

using namespace std;

string host ( "127.0.0.1" );
string username( "user" );
string password( "password" );
string database( "db" );
uint16_t port( 3306 );

bool connect( MYSQL* mysql )
{
    mysql_init( mysql );

    if ( !mysql_real_connect( mysql, host.c_str(), username.c_str(), password.c_str(), database.c_str(),
        port, NULL, 0 ) )
    {
        cerr << "Could not connect to database" << endl;
        return false;
    }

    if ( mysql_options( mysql, MYSQL_OPT_RECONNECT, "1" ) != 0 )
    {
        cerr << "Could not set database reconnect option" << endl;
        return false;
    }

    if ( mysql_options( mysql, MYSQL_READ_DEFAULT_FILE, "my.cnf" ) != 0 )
    {
        cerr << "Could not read client option file " << endl;
        return false;
    }

    if ( mysql_select_db( mysql, database.c_str() ) != 0)
    {
        cerr << "Could not select database" << endl;
        return false;
    }

    return true;
}

MYSQL_RES* query( MYSQL* mysql, string sql )
{
    if( mysql_real_query( mysql, sql.c_str(), sql.size() ) != 0 )
    {
        return NULL;
    }

    return mysql_store_result( mysql );
}

int main( int argc, char* argv[] )
{
    MYSQL mysql;

    if( !connect( &mysql ) )
    {
        return 1;
    }

    string sql( "SELECT count(*) from table" );

    for( size_t i =0;; ++i )
    {
        cout << ".";
        if( i % 180 == 0 ) cout << endl;
        MYSQL_RES* result = query( &mysql, sql );
        if ( result )
        {
            mysql_free_result( result );
        }
    }
}

Compile like:
g++ -g -o reconnect TestReconnect.cpp -I /path/to/mysql-5.0.37/include -L /path/to/mysql-5.0.37/lib/mysql -lmysqlclient_r -lz -lpthread

Run like:
LD_LIBRARY_PATH=/path/to/mysql-5.0.37/lib/mysql:$LD_LIBRARY_PATH gdb ./reconnect

On the mysql server side, start the loop given in the Description section.  The client should crash, eventually.

Suggested fix:
Unknown

This bug also occurs with 5.0.41

Thank you for a bug report. Verified just as described with latest 5.0.46-debug on SuSE Linux 9.3. I've got, eventually, while running your test code and continuosly restarting myslqd:

....................
openxs@linux:~/dbs/5.0> *** glibc detected *** double free or corruption (top): 0x08061510 ***

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29500

ChangeSet@1.2496, 2007-06-25 16:40:29+05:00, holyfoot@mysql.com +1 -0
  Bug #29247 Double free in libmysqlclient_r when mysql restarted.
  
  If one sets MYSQL_READ_DEFAULTS_FILE and MYSQL_READ_DEFAULT_GROUP options
  after mysql_real_connect() called with that MYSQL instance,
  these options will affect next mysql_reconnect then.
  As we use a copy of the original MYSQL object inside mysql_reconnect,
  and mysql_real_connect frees options.my_cnf_file and _group strings,
  we will free these twice when we execute mysql_reconnect with the
  same MYSQL for the second time.
  
  I don't think we should ever read defaults files handling mysql_reconnect.
  So i just set them to 0 for the temporary MYSQL object there/

I should note that manual says we should call mysql_options() before
the mysql_real_connect().
http://dev.mysql.com/doc/refman/5.1/en/mysql-options.html

And the reported problem appears as Andrew does it in opposite order
in his example.

Yes, moving around the order of calls also fixes this.  We had the order reversed because it didn't work for some older version of mysqlclient (see:
http://dev.mysql.com/doc/refman/5.0/en/mysql-options.html) and the comment stating:
'
Even though the documentation says that you should call mysql_options before mysql_real_connect, when setting MYSQL_OPT_RECONNECT you MUST do so after the connection has been successfully established.

This is true as late as 5.0.16.
'

Pushed into 5.1.21-beta

Pushed into 5.0.46

Noted in 5.0.46, 5.1.21 changelogs.

Calling mysql_options() after mysql_real_connect() could cause 
clients to crash.

Also updated the mysql_options() pages to mention the problem
of mysql_real_connect() resetting auto-reconnect behavior prior
to MySQL 5.0.19/5.1.6.