MySQL Bugs: #78965: Heap buffer overflow in main.mysqlpump

Bug #78965	Heap buffer overflow in main.mysqlpump
Submitted:	26 Oct 2015 14:18	Modified:	1 Jul 2016 10:45
Reporter:	Richard Prohaska	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: mysqlpump Command-line Client	Severity:	S3 (Non-critical)
Version:	5.7.9	OS:	Ubuntu (14.04)
Assigned to:		CPU Architecture:	Any

Description:
When MySQL 5.7.9 is built with the address sanitizer, all of the mysqlpump mysql tests fail due to memory leaks detected by the address sanitizer.  The leaks are caused by missing object management in the dump client code.

With a MySQL 5.7.9 build without the address sanitizer, mysqlpump fails when run with valgrind's memcheck due to the same memory leaks.

How to repeat:
cmake -DWITH_ASAN=ON ...
mtr --do-tests=mysqlpump --force --retry=0

Suggested fix:
the dump client software needs to manage objects (delete them when no longer necessary).

Basically a duplicate of 

Bug#78224 memory leak in mysqlpump

ASan also reports not a memory leak but a heap-buffer-overflow on 5.7.9 main.mysqlpump test. The Valgrind bug 78224 does not mention a corresponding error there.

main.mysqlpump                           w1 [ fail ]
        Test ended at 2015-11-13 01:38:08

CURRENT_TEST: main.mysqlpump
Dump progress: 1/1 tables, 0/0 rows
Dump completed in 3395 milliseconds
Dump completed in 3290 milliseconds
Dump completed in 3213 milliseconds
Dump progress: 1/1 tables, 0/0 rows
=================================================================
==95316==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000042920 at pc 0x00010ab57d20 bp 0x700000209bf0 sp 0x700000209be8
READ of size 8 at 0x602000042920 thread T4
    #0 0x10ab57d1f in Mysql::Tools::Dump::Abstract_dump_task::set_completed() abstract_dump_task.cc:63
    #1 0x10ab52d2b in Mysql::Tools::Dump::Abstract_chain_element::item_completion_in_child_completes_task_callback(Mysql::Tools::Dump::Item_processing_data*) abstract_chain_element.cc:145
    #2 0x10ab6736b in Mysql::Tools::Dump::Item_processing_data::end_processing() item_processing_data.cc:64
    #3 0x10ab532a9 in Mysql::Tools::Dump::Abstract_chain_element::object_processing_ends(Mysql::Tools::Dump::Item_processing_data*) abstract_chain_element.cc:100
    #4 0x10ab6736b in Mysql::Tools::Dump::Item_processing_data::end_processing() item_processing_data.cc:64
    #5 0x10ab532a9 in Mysql::Tools::Dump::Abstract_chain_element::object_processing_ends(Mysql::Tools::Dump::Item_processing_data*) abstract_chain_element.cc:100
    #6 0x10ab8a2a6 in Mysql::Tools::Dump::Object_queue::queue_thread() object_queue.cc:71
    #7 0x10ab9195b in boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > >::operator()() bind_template.hpp:20
    #8 0x10ab9186d in my_boost::thread::context<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > > >::entry_point(void*) thread.h:55
    #9 0x7fff8f6739b0 in _pthread_body (/usr/lib/system/libsystem_pthread.dylib+0x39b0)
    #10 0x7fff8f67392d in _pthread_start (/usr/lib/system/libsystem_pthread.dylib+0x392d)
    #11 0x7fff8f671384 in thread_start (/usr/lib/system/libsystem_pthread.dylib+0x1384)

0x602000042920 is located 16 bytes to the left of 8-byte region [0x602000042930,0x602000042938)
allocated by thread T0 here:
    #0 0x10b49705b in wrap__Znwm (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/7.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x4505b)
    #1 0x10ab59780 in std::__1::__split_buffer<Mysql::Tools::Dump::Abstract_dump_task const*, std::__1::allocator<Mysql::Tools::Dump::Abstract_dump_task const*>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<Mysql::Tools::Dump::Abstract_dump_task const*>&) new:156
    #2 0x10ab59104 in void std::__1::vector<Mysql::Tools::Dump::Abstract_dump_task const*, std::__1::allocator<Mysql::Tools::Dump::Abstract_dump_task const*> >::__push_back_slow_path<Mysql::Tools::Dump::Abstract_dump_task const* const>(Mysql::Tools::Dump::Abstract_dump_task const* const&) vector:1575
    #3 0x10ab57fd6 in Mysql::Tools::Dump::Abstract_dump_task::add_dependency(Mysql::Tools::Dump::Abstract_dump_task*) vector:1596
    #4 0x10ab6c417 in Mysql::Tools::Dump::Mysql_crawler::enumerate_table_triggers(Mysql::Tools::Dump::Table const&, Mysql::Tools::Dump::Abstract_dump_task*) mysql_crawler.cc:401
    #5 0x10ab6b014 in Mysql::Tools::Dump::Mysql_crawler::enumerate_tables(Mysql::Tools::Dump::Database const&) mysql_crawler.cc:206
    #6 0x10ab68a43 in Mysql::Tools::Dump::Mysql_crawler::enumerate_database_objects(Mysql::Tools::Dump::Database const&) mysql_crawler.cc:114
    #7 0x10ab682f2 in Mysql::Tools::Dump::Mysql_crawler::enumerate_objects() mysql_crawler.cc:82
    #8 0x10ab4d7bb in Mysql::Tools::Dump::Program::execute(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >) program.cc:183
    #9 0x10abbf99b in Mysql::Tools::Base::Abstract_program::run(int, char**) abstract_program.cc:98
    #10 0x10ab4e6f9 in main program.cc:254
    #11 0x7fff852ca5ac in start (/usr/lib/system/libdyld.dylib+0x35ac)
    #12 0xa  (<unknown module>)

Thread T4 created by T0 here:
    #0 0x10b48a8d9 in wrap_pthread_create (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/7.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x388d9)
    #1 0x10ab917c0 in my_boost::thread::thread<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > > >(boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > >) thread.h:33
    #2 0x10ab90cd6 in my_boost::thread::thread<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > > >(boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > >) thread.h:30
    #3 0x10ab8c0b3 in void my_boost::thread_group::create_thread<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > > >(boost::_bi::bind_t<void, boost::_mfi::mf0<void, Mysql::Tools::Dump::Object_queue>, boost::_bi::list1<boost::_bi::value<Mysql::Tools::Dump::Object_queue*> > >) thread_group.h:31
    #4 0x10ab8b7ee in Mysql::Tools::Dump::Object_queue::Object_queue(Mysql::I_callable<bool, Mysql::Tools::Base::Message_data const&>*, Mysql::Tools::Dump::Simple_id_generator*, unsigned int, Mysql::I_callable<void, bool>*) object_queue.cc:164
    #5 0x10ab76139 in Mysql::Tools::Dump::Mysqldump_tool_chain_maker::create_chain(Mysql::Tools::Dump::Chain_data*, Mysql::Tools::Dump::I_dump_task*) mysqldump_tool_chain_maker.cc:131
    #6 0x10ab54327 in Mysql::Tools::Dump::Abstract_crawler::process_dump_task(Mysql::Tools::Dump::I_dump_task*) abstract_crawler.cc:52
    #7 0x10ab6aff3 in Mysql::Tools::Dump::Mysql_crawler::enumerate_tables(Mysql::Tools::Dump::Database const&) mysql_crawler.cc:203
    #8 0x10ab68a43 in Mysql::Tools::Dump::Mysql_crawler::enumerate_database_objects(Mysql::Tools::Dump::Database const&) mysql_crawler.cc:114
    #9 0x10ab682f2 in Mysql::Tools::Dump::Mysql_crawler::enumerate_objects() mysql_crawler.cc:82
    #10 0x10ab4d7bb in Mysql::Tools::Dump::Program::execute(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >) program.cc:183
    #11 0x10abbf99b in Mysql::Tools::Base::Abstract_program::run(int, char**) abstract_program.cc:98
    #12 0x10ab4e6f9 in main program.cc:254
    #13 0x7fff852ca5ac in start (/usr/lib/system/libdyld.dylib+0x35ac)
    #14 0xa  (<unknown module>)

SUMMARY: AddressSanitizer: heap-buffer-overflow abstract_dump_task.cc:63 Mysql::Tools::Dump::Abstract_dump_task::set_completed()
Shadow bytes around the buggy address:
  0x1c04000084d0: fa fa fd fa fa fa fd fa fa fa 00 fa fa fa 00 fa
  0x1c04000084e0: fa fa 00 00 fa fa 00 00 fa fa 00 fa fa fa 00 fa
  0x1c04000084f0: fa fa 00 fa fa fa 00 fa fa fa 00 fa fa fa fd fa
  0x1c0400008500: fa fa fd fa fa fa fd fa fa fa 00 fa fa fa 00 fa
  0x1c0400008510: fa fa fd fa fa fa fc fa fa fa 00 00 fa fa 00 fa
=>0x1c0400008520: fa fa fd fd[fa]fa 00 fa fa fa 00 fa fa fa 00 fa
  0x1c0400008530: fa fa 00 fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x1c0400008540: fa fa 00 fa fa fa 00 fa fa fa fd fd fa fa fd fa
  0x1c0400008550: fa fa fd fa fa fa fd fa fa fa 00 fa fa fa 00 fa
  0x1c0400008560: fa fa 00 fa fa fa fd fa fa fa fc fa fa fa 00 00
  0x1c0400008570: fa fa 00 00 fa fa 00 00 fa fa 00 fa fa fa fd fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==95316==ABORTING
sh: line 1: 95316 Abort trap: 6           /Users/laurynas/percona/obj-percona-server-5.7.9-asan/client//mysqlpump --defaults-file=/Users/laurynas/percona/obj-percona-server-5.7.9-asan/mysql-test/var/1/my.cnf --defaults-group-suffix=.1 --parallel-schemas=db3 -B db3 db1 --routines --events --triggers --protocol=tcp > /Users/laurynas/percona/obj-percona-server-5.7.9-asan/mysql-test/var/1/tmp/output_file_4.sql
mysqltest: At line 93: command "$MYSQL_PUMP --parallel-schemas=db3 -B db3 db1 --routines --events --triggers --protocol=tcp > $MYSQLTEST_VARDIR/tmp/output_file_4.sql" failed

Output from before failure:
exec of '/Users/laurynas/percona/obj-percona-server-5.7.9-asan/client//mysqlpump --defaults-file=/Users/laurynas/percona/obj-percona-server-5.7.9-asan/mysql-test/var/1/my.cnf --defaults-group-suffix=.1 --parallel-schemas=db3 -B db3 db1 --routines --events --triggers --protocol=tcp > /Users/laurynas/percona/obj-percona-server-5.7.9-asan/mysql-test/var/1/tmp/output_file_4.sql' failed, error: 34304, status: 134, errno: 22

The result from queries just before the failure was:
< snip >
supplier
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA='db1_1gb' AND TABLE_TYPE= 'VIEW'
       ORDER BY TABLE_NAME;
TABLE_NAME
SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_SCHEMA='db1_1gb' AND ROUTINE_TYPE= 'PROCEDURE'
       ORDER BY ROUTINE_NAME;
ROUTINE_NAME
SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_SCHEMA='db1_1gb' AND ROUTINE_TYPE= 'FUNCTION'
       ORDER BY ROUTINE_NAME;
ROUTINE_NAME
SELECT EVENT_NAME FROM INFORMATION_SCHEMA.EVENTS
WHERE EVENT_SCHEMA='db1_1gb' ORDER BY EVENT_NAME;
EVENT_NAME
SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS
WHERE TRIGGER_SCHEMA='db1_1gb' ORDER BY TRIGGER_NAME;
TRIGGER_NAME

safe_process[95244]: Child process: 95245, exit: 1

 - the logfile can be found in '/Users/laurynas/percona/obj-percona-server-5.7.9-asan/mysql-test/var/log/main.mysqlpump/mysqlpump.log'

Fixed in the upcoming 5.7.13/8.0.0 releases.