Bug #111177 Mysqlrouter crash "Application got fatal signal: 11"

Submitted: 29 May 2023 3:03 Modified: 7 Nov 2024 13:06
Reporter: Duong Mai Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Router Severity:S2 (Serious)
Version:8.0.32, 8.0.33 OS:Debian
Assigned to: CPU Architecture:Any

[29 May 2023 3:03] Duong Mai
Description:
Hi
We are currently using mysqlrouter 8.0.32 in container (official images) with mysql innodb cluster (also in container using official images)
Mysql router containers were crashed some times with this backtrace:
```
Application got fatal signal: 11
stack_bottom = 0 thread_stack 0x0
/usr/lib64/mysqlrouter/private/libmysqlharness.so.1(my_print_stacktrace(unsigned char const*, unsigned long)+0x41) [0x7f147816bd41]
/usr/lib64/mysqlrouter/private/libmysqlharness.so.1(+0xd87f1) [0x7f14781567f1]
/lib64/libpthread.so.0(+0x12d00) [0x7f1477d8cd00]
/usr/lib64/mysqlrouter/private/libmysqlrouter_routing.so.1(MysqlRoutingClassicConnection::get_server_address[abi:cxx11]() const+0x42) [0x7f1474a85882]
/usr/lib64/mysqlrouter/private/libmysqlrouter_routing.so.1(void concurrent_map<MySQLRoutingConnectionBase*, std::shared_ptr<MySQLRoutingConnectionBase>, std::hash<MySQLRoutingConnectionBase*> >::for_each<ConnectionContainer::get_all_connections_info()::{lambda(std::pair<MySQLRoutingConnectionBase* const, std::shared_ptr<MySQLRoutingConnectionBase> > const&)#1}>(ConnectionContainer::get_all_connections_info()::{lambda(std::pair<MySQLRoutingConnectionBase* const, std::shared_ptr<MySQLRoutingConnectionBase> >/usr/lib64/mysqlrouter/private/libmysqlrouter_routing.so.1(MySQLRouting::get_connections()+0x38) [0x7f1474a5e6e8]
/usr/lib64/mysqlrouter/private/libmysqlrouter_routing.so.1(MySQLRoutingAPI::get_connections() const+0x29) [0x7f1474a807e9]
/usr/lib64/mysqlrouter/rest_routing.so(RestRoutingConnections::on_handle_request(HttpRequest&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)+0x1de) [0x7f1474b974ae]
/usr/lib64/mysqlrouter/rest_api.so(RestApi::handle_paths(HttpRequest&)+0x729) [0x7f1474c14379]
/usr/lib64/mysqlrouter/http_server.so(HttpRequestRouter::route(HttpRequest)+0x16e) [0x7f1474c8cb2e]
/usr/lib64/mysqlrouter/http_server.so(+0x37e0c) [0x7f1474c8ce0c]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0xd6b5) [0x7f14780056b5]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x3076c) [0x7f147802876c]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x18b75) [0x7f1478010b75]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x34e5d) [0x7f147802ce5d]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x34e86) [0x7f147802ce86]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x1ff42) [0x7f1478017f42]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x20767) [0x7f1478018767]
/usr/lib64/mysqlrouter/http_server.so(HttpRequestThread::wait_and_dispatch()+0x35) [0x7f1474c7b5d5]
/lib64/libstdc++.so.6(+0xc2ba3) [0x7f1476514ba3]
/lib64/libpthread.so.0(+0x81da) [0x7f1477d821da]
/lib64/libc.so.6(clone+0x43) [0x7f1475b2ae73]
```

How to repeat:
This bug was happened randomly
[29 May 2023 3:18] Duong Mai
we're also using a custom mysql router exporter to query REST API (60s interval) and translate to exporter format
[30 May 2023 1:22] MySQL Verification Team
Hi,

I cannot reproduce this. Can you test 8.0.33 ?
[30 May 2023 2:38] Duong Mai
I also want to use 8.0.33 but I can not find 8.0.33 image on https://hub.docker.com/r/mysql/mysql-router/tags (newest is 8.0.32)
Where could I find newest image?
[30 May 2023 5:21] MySQL Verification Team
Please use Oracle container registry. You are using images that are not maintained by Oracle. We suggest you switch to Oracle repo

https://container-registry.oracle.com/ords/f?p=113:1:5838602727651:::1:P1_BUSINESS_AREA:6&...

docker pull container-registry.oracle.com/mysql/community-server:8.0.33

You can see that 8.0.33 is available since 5 weeks ago:

https://container-registry.oracle.com/ords/f?p=113:4:10808482767135:::RP,4:P4_REPOSITORY,A...
[30 May 2023 8:57] Duong Mai
Okay, I will try 8.0.33 and report result later
Thanks
[16 Jun 2023 7:16] Duong Mai
Hi, I tested with mysqlrouter 8.0.33 and still get a bug after 10 days. It happen randomly, not affect to all router instance 
Do you have any clues if look up stack strace ???

Application got fatal signal: 11
stack_bottom = 0 thread_stack 0x0
/usr/lib64/mysqlrouter/private/libmysqlharness.so.1(my_print_stacktrace(unsigned char const*, unsigned long)+0x41) [0x7ff388a10d71]
/usr/lib64/mysqlrouter/private/libmysqlharness.so.1(+0xdc3b1) [0x7ff3889fb3b1]
/lib64/libpthread.so.0(+0x12d00) [0x7ff388642d00]
/usr/lib64/mysqlrouter/private/libmysqlrouter_routing.so.1(MysqlRoutingClassicConnectionBase::get_server_address[abi:cxx11]() const+0x44) [0x7ff3852e1b14]
/usr/lib64/mysqlrouter/private/libmysqlrouter_routing.so.1(MySQLRouting::get_connections()+0x214) [0x7ff3852b96b4]
/usr/lib64/mysqlrouter/private/libmysqlrouter_routing.so.1(MySQLRoutingAPI::get_connections() const+0x28) [0x7ff3852dc538]
/usr/lib64/mysqlrouter/rest_routing.so(RestRoutingConnections::on_handle_request(HttpRequest&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)+0x1e0) [0x7ff3853f88d0]
/usr/lib64/mysqlrouter/rest_api.so(RestApi::handle_paths(HttpRequest&)+0x6d2) [0x7ff385481902]
/usr/lib64/mysqlrouter/http_server.so(HttpRequestRouter::route(HttpRequest)+0x15e) [0x7ff38550a9ae]
/usr/lib64/mysqlrouter/http_server.so(+0x39c9b) [0x7ff38550ac9b]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0xc052) [0x7ff388899052]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x1563c) [0x7ff3888a263c]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x21f14) [0x7ff3888aef14]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x19c3d) [0x7ff3888a6c3d]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x19c68) [0x7ff3888a6c68]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x2a034) [0x7ff3888b7034]
/usr/lib64/mysqlrouter/private/libmysqlrouter_http.so.1(+0x2a82f) [0x7ff3888b782f]
/usr/lib64/mysqlrouter/http_server.so(HttpRequestThread::wait_and_dispatch()+0x34) [0x7ff3854f7214]
/lib64/libstdc++.so.6(+0xc2ba3) [0x7ff386dc0ba3]
/lib64/libpthread.so.0(+0x81da) [0x7ff3886381da]
/lib64/libc.so.6(clone+0x43) [0x7ff3863d6e73]

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.055 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.