Description:
When using the async client API, a heap-use-after-free error occurs in cli_read_rows_nonblocking() when a connection is killed while reading result rows.
The issue is that cli_read_rows_nonblocking() caches the NET_ASYNC pointer from the net structure in a local variable. However, this pointer can be freed during calls to cli_safe_read_nonblocking() when the connection is killed, which calls end_server() -> net_end() -> net_extension_free() to free the net_async structure. After the free, the function continues to access the stale net_async pointer, resulting in a use-after-free.
And there is a potential memory leak: The async_context->rows_result_buffer should be freed regardless of the net_async state in the function cli_read_rows_nonblocking. Currently, if there are other code paths where pkt_len == packet_error is not properly handled, memory allocated for rows_result_buffer could be leaked. The memory cleanup should always be performed unconditionally when an error occurs, independent of the network async state.
The stack of use-after-free:
==79867==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000000938 at pc 0x000000850447 bp 0x7ffc21bd5160 sp 0x7ffc21bd5150
READ of size 1 at 0x611000000938 thread T0
#0 0x850446 in cli_read_rows_nonblocking(MYSQL*, MYSQL_FIELD*, unsigned int, MYSQL_DATA**) /u01/huyutuo/code/mysql-server/sql-common/client.cc:2872
#1 0x85f85d in mysql_store_result_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:8184
#2 0x79d37d in async_mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:766
#3 0x79d483 in mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:896
#4 0x7b7204 in run_query_normal /u01/huyutuo/code/mysql-server/client/mysqltest.cc:8637
#5 0x7b8e2e in run_query /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9099
#6 0x7c63cb in main /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9978
#7 0x7f44eb422554 in __libc_start_main ../csu/libc-start.c:266
#8 0x792c68 (/u01/huyutuo/code/mysql-server/build/runtime_output_directory/mysqltest+0x792c68)
0x611000000938 is located 56 bytes inside of 248-byte region [0x611000000900,0x6110000009f8)
freed by thread T0 here:
#0 0x7f44edeb43f7 in free (/lib64/libasan.so.6+0xb43f7)
#1 0x985b3b in redirecting_deallocator /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:299
#2 0x985b46 in my_raw_free<redirecting_deallocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:360
#3 0x985cb4 in my_internal_free<redirecting_deallocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:407
#4 0x985fd5 in my_free(void*) /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:469
#5 0x884bad in net_extension_free(NET*) /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:120
#6 0x885359 in net_end(NET*) /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:205
#7 0x84a948 in end_server /u01/huyutuo/code/mysql-server/sql-common/client.cc:1905
#8 0x84ae78 in cli_safe_read_with_ok_complete /u01/huyutuo/code/mysql-server/sql-common/client.cc:1207
#9 0x84b91f in cli_safe_read_with_ok_nonblocking(MYSQL*, bool, bool*, unsigned long*) /u01/huyutuo/code/mysql-server/sql-common/client.cc:1141
#10 0x84ba68 in cli_safe_read_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:1158
#11 0x85013e in cli_read_rows_nonblocking(MYSQL*, MYSQL_FIELD*, unsigned int, MYSQL_DATA**) /u01/huyutuo/code/mysql-server/sql-common/client.cc:2865
#12 0x85f85d in mysql_store_result_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:8184
#13 0x79d37d in async_mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:766
#14 0x79d483 in mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:896
#15 0x7b7204 in run_query_normal /u01/huyutuo/code/mysql-server/client/mysqltest.cc:8637
#16 0x7b8e2e in run_query /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9099
#17 0x7c63cb in main /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9978
#18 0x7f44eb422554 in __libc_start_main ../csu/libc-start.c:266
previously allocated by thread T0 here:
#0 0x7f44edeb4917 in __interceptor_calloc (/lib64/libasan.so.6+0xb4917)
#1 0x985b72 in redirecting_allocator /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:278
#2 0x986009 in my_raw_malloc<redirecting_allocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:323
#3 0x986157 in my_internal_malloc<redirecting_allocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:373
#4 0x986250 in my_malloc(unsigned int, unsigned long, int) /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:387
#5 0x884b12 in net_extension_init() /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:109
#6 0x884f70 in my_net_init(NET*, Vio*) /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:178
#7 0x852d3f in csm_complete_connect /u01/huyutuo/code/mysql-server/sql-common/client.cc:6751
#8 0x86af2b in mysql_real_connect_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:6255
#9 0x79e6ba in async_mysql_real_connect_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:845
#10 0x79e7c0 in mysql_real_connect_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:955
#11 0x7a707d in connect_n_handle_errors /u01/huyutuo/code/mysql-server/client/mysqltest.cc:6558
#12 0x7ad65e in do_connect /u01/huyutuo/code/mysql-server/client/mysqltest.cc:6881
#13 0x7c59d4 in main /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9744
#14 0x7f44eb422554 in __libc_start_main ../csu/libc-start.c:266
SUMMARY: AddressSanitizer: heap-use-after-free /u01/huyutuo/code/mysql-server/sql-common/client.cc:2872 in cli_read_rows_nonblocking(MYSQL*, MYSQL_FIELD*, unsigned int, MYSQL_DATA**)
Shadow bytes around the buggy address:
0x0c227fff80d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c227fff80e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa
0x0c227fff80f0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
0x0c227fff8100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c227fff8110: 00 00 00 00 fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c227fff8120: fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd
0x0c227fff8130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c227fff8140: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
0x0c227fff8150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c227fff8160: 00 00 00 00 fa fa fa fa fa fa fa fa fa fa fa fa
0x0c227fff8170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==79867==ABORTING
safe_process[79866]: Child process: 79867, exit: 42
How to repeat:
Build with valgrind.
Create a test file mysql-test/t/async_client_kill.test (in the files of this bug)
mtr --valgrind --mem --async-client main.async_client_kill
Suggested fix:
Check for nullptr before accessing net_async when pkt_len == packet_error in the function cli_advanced_command_nonblocking, cli_read_rows_nonblocking
Such as(This diff based on commit 153abc27b7dda30b455c0d99e657dac4e39e3505 of the MySQL source codeļ¼:
diff --git a/sql-common/client.cc b/sql-common/client.cc
index e2653f9d293..93a96355a24 100644
--- a/sql-common/client.cc
+++ b/sql-common/client.cc
@@ -1604,7 +1604,7 @@ net_async_status cli_advanced_command_nonblocking(
#endif
}
end:
- if (net_async)
+ if (NET_ASYNC_DATA(net) != nullptr)
net_async->async_send_command_status = NET_ASYNC_SEND_COMMAND_IDLE;
DBUG_PRINT("exit", ("result: %d", result));
*ret = result;
@@ -2869,11 +2869,11 @@ net_async_status cli_read_rows_nonblocking(MYSQL *mysql,
mysql->packet_length = pkt_len;
if (pkt_len == packet_error) {
- if (net_async->read_rows_is_first_read) {
- free_rows(async_context->rows_result_buffer);
- async_context->rows_result_buffer = nullptr;
+ if (NET_ASYNC_DATA(net) != nullptr) {
+ net_async->read_rows_is_first_read = true;
}
- net_async->read_rows_is_first_read = true;
+ free_rows(async_context->rows_result_buffer);
+ async_context->rows_result_buffer = nullptr;
return NET_ASYNC_COMPLETE;
}
@@ -2952,7 +2952,9 @@ net_async_status cli_read_rows_nonblocking(MYSQL *mysql,
if (pkt_len == packet_error) {
free_rows(async_context->rows_result_buffer);
async_context->rows_result_buffer = nullptr;
- net_async->read_rows_is_first_read = true;
+ if (NET_ASYNC_DATA(net) != nullptr) {
+ net_async->read_rows_is_first_read = true;
+ }
return NET_ASYNC_COMPLETE;
}
}
Description: When using the async client API, a heap-use-after-free error occurs in cli_read_rows_nonblocking() when a connection is killed while reading result rows. The issue is that cli_read_rows_nonblocking() caches the NET_ASYNC pointer from the net structure in a local variable. However, this pointer can be freed during calls to cli_safe_read_nonblocking() when the connection is killed, which calls end_server() -> net_end() -> net_extension_free() to free the net_async structure. After the free, the function continues to access the stale net_async pointer, resulting in a use-after-free. And there is a potential memory leak: The async_context->rows_result_buffer should be freed regardless of the net_async state in the function cli_read_rows_nonblocking. Currently, if there are other code paths where pkt_len == packet_error is not properly handled, memory allocated for rows_result_buffer could be leaked. The memory cleanup should always be performed unconditionally when an error occurs, independent of the network async state. The stack of use-after-free: ==79867==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000000938 at pc 0x000000850447 bp 0x7ffc21bd5160 sp 0x7ffc21bd5150 READ of size 1 at 0x611000000938 thread T0 #0 0x850446 in cli_read_rows_nonblocking(MYSQL*, MYSQL_FIELD*, unsigned int, MYSQL_DATA**) /u01/huyutuo/code/mysql-server/sql-common/client.cc:2872 #1 0x85f85d in mysql_store_result_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:8184 #2 0x79d37d in async_mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:766 #3 0x79d483 in mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:896 #4 0x7b7204 in run_query_normal /u01/huyutuo/code/mysql-server/client/mysqltest.cc:8637 #5 0x7b8e2e in run_query /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9099 #6 0x7c63cb in main /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9978 #7 0x7f44eb422554 in __libc_start_main ../csu/libc-start.c:266 #8 0x792c68 (/u01/huyutuo/code/mysql-server/build/runtime_output_directory/mysqltest+0x792c68) 0x611000000938 is located 56 bytes inside of 248-byte region [0x611000000900,0x6110000009f8) freed by thread T0 here: #0 0x7f44edeb43f7 in free (/lib64/libasan.so.6+0xb43f7) #1 0x985b3b in redirecting_deallocator /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:299 #2 0x985b46 in my_raw_free<redirecting_deallocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:360 #3 0x985cb4 in my_internal_free<redirecting_deallocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:407 #4 0x985fd5 in my_free(void*) /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:469 #5 0x884bad in net_extension_free(NET*) /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:120 #6 0x885359 in net_end(NET*) /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:205 #7 0x84a948 in end_server /u01/huyutuo/code/mysql-server/sql-common/client.cc:1905 #8 0x84ae78 in cli_safe_read_with_ok_complete /u01/huyutuo/code/mysql-server/sql-common/client.cc:1207 #9 0x84b91f in cli_safe_read_with_ok_nonblocking(MYSQL*, bool, bool*, unsigned long*) /u01/huyutuo/code/mysql-server/sql-common/client.cc:1141 #10 0x84ba68 in cli_safe_read_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:1158 #11 0x85013e in cli_read_rows_nonblocking(MYSQL*, MYSQL_FIELD*, unsigned int, MYSQL_DATA**) /u01/huyutuo/code/mysql-server/sql-common/client.cc:2865 #12 0x85f85d in mysql_store_result_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:8184 #13 0x79d37d in async_mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:766 #14 0x79d483 in mysql_store_result_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:896 #15 0x7b7204 in run_query_normal /u01/huyutuo/code/mysql-server/client/mysqltest.cc:8637 #16 0x7b8e2e in run_query /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9099 #17 0x7c63cb in main /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9978 #18 0x7f44eb422554 in __libc_start_main ../csu/libc-start.c:266 previously allocated by thread T0 here: #0 0x7f44edeb4917 in __interceptor_calloc (/lib64/libasan.so.6+0xb4917) #1 0x985b72 in redirecting_allocator /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:278 #2 0x986009 in my_raw_malloc<redirecting_allocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:323 #3 0x986157 in my_internal_malloc<redirecting_allocator> /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:373 #4 0x986250 in my_malloc(unsigned int, unsigned long, int) /u01/huyutuo/code/mysql-server/mysys/my_malloc.cc:387 #5 0x884b12 in net_extension_init() /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:109 #6 0x884f70 in my_net_init(NET*, Vio*) /u01/huyutuo/code/mysql-server/sql-common/net_serv.cc:178 #7 0x852d3f in csm_complete_connect /u01/huyutuo/code/mysql-server/sql-common/client.cc:6751 #8 0x86af2b in mysql_real_connect_nonblocking /u01/huyutuo/code/mysql-server/sql-common/client.cc:6255 #9 0x79e6ba in async_mysql_real_connect_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:845 #10 0x79e7c0 in mysql_real_connect_wrapper /u01/huyutuo/code/mysql-server/client/mysqltest.cc:955 #11 0x7a707d in connect_n_handle_errors /u01/huyutuo/code/mysql-server/client/mysqltest.cc:6558 #12 0x7ad65e in do_connect /u01/huyutuo/code/mysql-server/client/mysqltest.cc:6881 #13 0x7c59d4 in main /u01/huyutuo/code/mysql-server/client/mysqltest.cc:9744 #14 0x7f44eb422554 in __libc_start_main ../csu/libc-start.c:266 SUMMARY: AddressSanitizer: heap-use-after-free /u01/huyutuo/code/mysql-server/sql-common/client.cc:2872 in cli_read_rows_nonblocking(MYSQL*, MYSQL_FIELD*, unsigned int, MYSQL_DATA**) Shadow bytes around the buggy address: 0x0c227fff80d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c227fff80e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa 0x0c227fff80f0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00 0x0c227fff8100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c227fff8110: 00 00 00 00 fa fa fa fa fa fa fa fa fa fa fa fa =>0x0c227fff8120: fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd 0x0c227fff8130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa 0x0c227fff8140: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00 0x0c227fff8150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c227fff8160: 00 00 00 00 fa fa fa fa fa fa fa fa fa fa fa fa 0x0c227fff8170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==79867==ABORTING safe_process[79866]: Child process: 79867, exit: 42 How to repeat: Build with valgrind. Create a test file mysql-test/t/async_client_kill.test (in the files of this bug) mtr --valgrind --mem --async-client main.async_client_kill Suggested fix: Check for nullptr before accessing net_async when pkt_len == packet_error in the function cli_advanced_command_nonblocking, cli_read_rows_nonblocking Such as(This diff based on commit 153abc27b7dda30b455c0d99e657dac4e39e3505 of the MySQL source codeļ¼: diff --git a/sql-common/client.cc b/sql-common/client.cc index e2653f9d293..93a96355a24 100644 --- a/sql-common/client.cc +++ b/sql-common/client.cc @@ -1604,7 +1604,7 @@ net_async_status cli_advanced_command_nonblocking( #endif } end: - if (net_async) + if (NET_ASYNC_DATA(net) != nullptr) net_async->async_send_command_status = NET_ASYNC_SEND_COMMAND_IDLE; DBUG_PRINT("exit", ("result: %d", result)); *ret = result; @@ -2869,11 +2869,11 @@ net_async_status cli_read_rows_nonblocking(MYSQL *mysql, mysql->packet_length = pkt_len; if (pkt_len == packet_error) { - if (net_async->read_rows_is_first_read) { - free_rows(async_context->rows_result_buffer); - async_context->rows_result_buffer = nullptr; + if (NET_ASYNC_DATA(net) != nullptr) { + net_async->read_rows_is_first_read = true; } - net_async->read_rows_is_first_read = true; + free_rows(async_context->rows_result_buffer); + async_context->rows_result_buffer = nullptr; return NET_ASYNC_COMPLETE; } @@ -2952,7 +2952,9 @@ net_async_status cli_read_rows_nonblocking(MYSQL *mysql, if (pkt_len == packet_error) { free_rows(async_context->rows_result_buffer); async_context->rows_result_buffer = nullptr; - net_async->read_rows_is_first_read = true; + if (NET_ASYNC_DATA(net) != nullptr) { + net_async->read_rows_is_first_read = true; + } return NET_ASYNC_COMPLETE; } }