MySQL Bugs: #101591: main.greedy_optimizer MTR triggers AddressSanitizer

Bug #101591	main.greedy_optimizer MTR triggers AddressSanitizer
Submitted:	12 Nov 2020 19:53	Modified:	4 Jan 2021 14:03
Reporter:	Kamil Holubicki (OCA)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Tests	Severity:	S7 (Test Cases)
Version:	8.0.22	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
Running main.greedy_optimizer (and others like main.subquery_sj* ) MTR tests trigger AddressSanitizer stack-overflow error.

How to repeat:
1. Build the server with the following options:
-DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON

2. ./mysql-test/mtr main.greedy_optimizer

Result:
ASAN:DEADLYSIGNAL
=================================================================
==4608==ERROR: AddressSanitizer: stack-overflow on address 0x7ff3e3cacef8 (pc 0x7ff40699d1e6 bp 0x7ff3e3cad790 sp 0x7ff3e3cacf00 T38)
    #0 0x7ff40699d1e5  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x5b1e5)
    #1 0x55c7dd47c68b in strchr /usr/include/string.h:220
    #2 0x55c7dd47c68b in AutoDebugTrace::AutoDebugTrace(char const*, char const*, int) /home/kamil/repo/upstream/8.0/mysql-server/include/my_dbug.h:109
    #3 0x55c7dfe1b9a7 in MEM_ROOT::AllocSlow(unsigned long) /home/kamil/repo/upstream/8.0/mysql-server/mysys/my_alloc.cc:87
    #4 0x55c7dd47cad6 in MEM_ROOT::Alloc(unsigned long) /home/kamil/repo/upstream/8.0/mysql-server/include/my_alloc.h:154
    #5 0x55c7dd4e8e88 in operator new(unsigned long, MEM_ROOT*, std::nothrow_t const&) /home/kamil/repo/upstream/8.0/mysql-server/include/my_alloc.h:362
    #6 0x55c7de04c59c in std::unique_ptr<RowIterator, Destroy_only<RowIterator> > NewIterator<TableScanIterator, TABLE*&, QEP_TAB*&, unsigned long long*&>(THD*, TABLE*&, QEP_TAB*&, unsigned long long*&) /home/kamil/repo/upstream/8.0/mysql-server/sql/timing_iterator.h:168
    #7 0x55c7de041d3e in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:367
    #8 0x55c7de0431c4 in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:528
    #9 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #10 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #11 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #12 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #13 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #14 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #15 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #16 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #17 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #18 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #19 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #20 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #21 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #22 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #23 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #24 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #25 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #26 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #27 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #28 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #29 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #30 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #31 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #32 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #33 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #34 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #35 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #36 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #37 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #38 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #39 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #40 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #41 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #42 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #43 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #44 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #45 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #46 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #47 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #48 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #49 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #50 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #51 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #52 0x55c7de04323d in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:530
    #53 0x55c7de043eb6 in CreateIteratorFromAccessPath(THD*, AccessPath*, JOIN*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/join_optimizer/access_path.cc:595
    #54 0x55c7dd929bee in SELECT_LEX_UNIT::optimize(THD*, TABLE*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/sql_union.cc:811
    #55 0x55c7dd7eb1d2 in Sql_cmd_dml::execute_inner(THD*) /home/kamil/repo/upstream/8.0/mysql-server/sql/sql_select.cc:924
    #56 0x55c7dd806a6a in Sql_cmd_dml::execute(THD*) /home/kamil/repo/upstream/8.0/mysql-server/sql/sql_select.cc:725
    #57 0x55c7dd711e9d in mysql_execute_command(THD*, bool) /home/kamil/repo/upstream/8.0/mysql-server/sql/sql_parse.cc:4344
    #58 0x55c7dd714a6d in mysql_parse(THD*, Parser_state*) /home/kamil/repo/upstream/8.0/mysql-server/sql/sql_parse.cc:4965
    #59 0x55c7dd717281 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/kamil/repo/upstream/8.0/mysql-server/sql/sql_parse.cc:1784
    #60 0x55c7dd71a781 in do_command(THD*) /home/kamil/repo/upstream/8.0/mysql-server/sql/sql_parse.cc:1309
    #61 0x55c7dda5b0ca in handle_connection /home/kamil/repo/upstream/8.0/mysql-server/sql/conn_handler/connection_handler_per_thread.cc:301
    #62 0x55c7e0be180f in pfs_spawn_thread /home/kamil/repo/upstream/8.0/mysql-server/storage/perfschema/pfs.cc:2880
    #63 0x7ff40672a6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
    #64 0x7ff404ba3a3e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x121a3e)

SUMMARY: AddressSanitizer: stack-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x5b1e5) 
Thread T38 created by T0 here:
    #0 0x7ff406979d2f in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x37d2f)
    #1 0x55c7dfe31dfd in my_thread_create(my_thread_handle*, pthread_attr_t const*, void* (*)(void*), void*) /home/kamil/repo/upstream/8.0/mysql-server/mysys/my_thread.cc:59
    #2 0x55c7e0be522f in pfs_spawn_thread_vc(unsigned int, my_thread_handle*, pthread_attr_t const*, void* (*)(void*), void*) /home/kamil/repo/upstream/8.0/mysql-server/storage/perfschema/pfs.cc:2930
    #3 0x55c7dda5a1d9 in inline_mysql_thread_create /home/kamil/repo/upstream/8.0/mysql-server/include/mysql/psi/mysql_thread.h:140
    #4 0x55c7dda5b404 in Per_thread_connection_handler::add_connection(Channel_info*) /home/kamil/repo/upstream/8.0/mysql-server/sql/conn_handler/connection_handler_per_thread.cc:409
    #5 0x55c7ddc6d793 in Connection_handler_manager::process_new_connection(Channel_info*) /home/kamil/repo/upstream/8.0/mysql-server/sql/conn_handler/connection_handler_manager.cc:260
    #6 0x55c7dd4812c6 in Connection_acceptor<Mysqld_socket_listener>::connection_event_loop() (/home/kamil/repo/upstream/8.0/bld/runtime_output_directory/mysqld+0x388a2c6)
    #7 0x55c7dd47badf in mysqld_main(int, char**) /home/kamil/repo/upstream/8.0/mysql-server/sql/mysqld.cc:7466
    #8 0x55c7dd454512 in main /home/kamil/repo/upstream/8.0/mysql-server/sql/main.cc:25
    #9 0x7ff404aa3b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

==4608==ABORTING

Suggested fix:
I see following problems here:
1. AddressSanitizer adds guards around local variables. This increases functions stacks
2. Default stack size is 280kB (my_thread.h)
3. CreateIteratorFromAccessPath() uses a lot of local variables. Together with ASAN guards this causes the function stack to be c.a. 5kB
4. CreateIteratorFromAccessPath() is called recursively (I didn't analyze it with details, but it does not look safe)

How to fix:
1. I think there is no problem without ASAN, however would be nice to have this alert handled gracefuly
2. Decomposing CreateIteratorFromAccessPath() into separate functions would help as the stack will not grow with variables not needed for particular execution path
3. Test is OK if DEFAULT_THREAD_STACK is set to 16MB

Thank you for the bug report. On which OS you got that result?. I couldn't repeat on Ubuntu with release and debug built:

miguel@tikal:~/tmp/mysql-8.0.22$ runtime_output_directory/mysqld --version
/home/miguel/tmp/mysql-8.0.22/runtime_output_directory/mysqld  Ver 8.0.22-debug-asan for Linux on x86_64 (Source distribution)
miguel@tikal:~/tmp/mysql-8.0.22$ cd mysql-test/
miguel@tikal:~/tmp/mysql-8.0.22/mysql-test$ ./mtr main.greedy_optimizer
Logging: ./mtr  main.greedy_optimizer
MySQL Version 8.0.22
Checking supported features
 - Binaries are debug compiled
Using 'all' suites
Collecting tests
Checking leftover processes
Removing old var directory
Creating var directory '/home/miguel/tmp/mysql-8.0.22/mysql-test/var'
Installing system database
Using parallel: 1

==============================================================================
                  TEST NAME                       RESULT  TIME (ms) COMMENT
------------------------------------------------------------------------------
[ 50%] main.greedy_optimizer                     [ pass ]  270310
[100%] shutdown_report                           [ pass ]
------------------------------------------------------------------------------
The servers were restarted 0 times
The servers were reinitialized 0 times
Spent 270.310 of 492 seconds executing testcases

Completed: All 2 tests were successful.

miguel@tikal:~/tmp/mysql-8.0.22/mysql-test$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.1 LTS
Release:        20.04
Codename:       focal

5.4.0-52-generic #57~18.04.1-Ubuntu SMP Thu Oct 15 14:04:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

configure options:
"WITH_BOOST": 		"/home/kamil/tmp/mysql-boost/8.0", 
"DOWNLOAD_BOOST": 	1,
"CMAKE_BUILD_TYPE":	"Debug",
"BUILD_CONFIG":		"mysql_release",
"WITH_UNIT_TESTS":	"OFF",
"WITH_READLINE": 	"system",
"ENABLE_DOWNLOADS":	"1",
"WITH_ROUTER":		"OFF",
"WITH_ASAN":		"ON",
"WITH_ASAN_SCOPE":	"ON,
"WITH_MYSQLX":		"OFF"

g++ --version
g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 127301
max locked memory       (kbytes, -l) 16384
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 127301
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

main.greedy_optimizer MTR execution log

Attachment: greedy_optimizer.log (text/x-log), 27.90 KiB.

Hi Mr. Holubicki,

Thank you for your bug report.

We were not able to repeat the behaviour on the latest Ubuntu stable release, which is 20.4. Can you try whether you have problems on that version ????

Also, what version of ASAN were you using ???

Thank you very much in advance.

Seems to be related to WITH_ASAN_SCOPE.

Hi Mr. Holubicki,

Thank you for your contribution.

This bug is now verified.

Posted by developer:
 
To reproduce, build with clang, and do:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index ce39d19eb60..3a540edbeca 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -776,7 +776,7 @@ IF(WITH_ASAN)
     SET(ASAN_LIB_DIR "${ASAN_LIB_DIR}/../lib/clang/${CMAKE_CXX_COMPILER_VERSION}/lib/windows")
     GET_FILENAME_COMPONENT(ASAN_LIB_DIR ${ASAN_LIB_DIR} REALPATH)
   ELSE()
-    MY_SANITIZER_CHECK("-fsanitize=address" TRUE WITH_ASAN_OK)
+    MY_SANITIZER_CHECK("-fsanitize=address" FALSE WITH_ASAN_OK)

The patch for 
Bug #31749396 OUT OF THREAD STACK IN CREATEITERATORFROMACCESSPATH
reduced stack usage, but enough for clang wit all optimizations switched off.

Thank you Tor, so much .....

I guess that all optimisations that lead to inline code would also trigger the bug ...... Also, arrays of the known size might cause optimising compiler to use more stack for it ..... Etc, etc ....

Posted by developer:
 
Fixed in 8.0.24.

Work was done for test suite. No changelog entry required.

Thank you, Paul .......