MySQL Bugs: #107665: mysql router crash

Bug #107665	mysql router crash
Submitted:	27 Jun 6:02	Modified:	1 Jul 2:31
Reporter:	WANG FENG	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Router	Severity:	S1 (Critical)
Version:	8.0.29	OS:	CentOS (7.7.1908)
Assigned to:	MySQL Verification Team	CPU Architecture:	x86

Description:
I had three nodes innodbCluster. And Mysql router was deployed on every three nodes. 
Mysql version was 8.0.26-el7-x86_64.
Mysql router version mysql-router-8.0.29-el7-x86_64.

Today,5:01:45AM, all mysqlRouter instance crashed about the same time.
I got nothing message exception following 
"
[root@WCM-MYSQLDB-03-LIUSJ log]# grep router messages
Jun 26 05:01:44 WCM-MYSQLDB-03-LIUSJ kernel: io-2[20582]: segfault at 0 ip 00007ff1c15b04a5 sp 00007ff1bbffeb50 error 4 in libmysqlrouter_routing.so.1[7ff1c155b000+63000]

[root@WCM-MYSQLDB-02-LIUSJ log]# grep router messages
Jun 26 05:01:45 WCM-MYSQLDB-02-LIUSJ kernel: io-2[22372]: segfault at 0 ip 00007fbb8b4884a5 sp 00007fbb8a37fb50 error 4 in libmysqlrouter_routing.so.1[7fbb8b433000+63000]

[root@wcm-mysqldb-01-liusj log]# grep router messages
Jun 26 05:01:45 WCM-MYSQLDB-01-LIUSJ kernel: io-2[6657]: segfault at 0 ip 00007f35735f94a5 sp 00007f35724f0b60 error 4 in libmysqlrouter_routing.so.1[7f35735a4000+63000]

"

How to repeat:
have no idea.

Suggested fix:
have no idea.

Hi,

Thank you for the report but without a way to reproduce this there is nothing we can do.

Any advice about  this message "segfault at 0 ip 00007ff1c15b04a5 sp 00007ff1bbffeb50 error 4 in libmysqlrouter_routing.so.1[7ff1c155b000+63000]"? Or you can give me some advices to research this ? Thanks.

Hi,

IMHO segfault is always a bug, and since it is our library that is segfaulting it is our bug so we should fix it. The problem is - without ability to reproduce it chances of fixing it are next to none. There is always a chance of external influence (e.g. bar hardware) but that is another story. 

We need to know when it happens so if you can somehow figure out how / when you are getting this segfault - what is the order of operations, and get to at least some vicinity of reproducible test case I can take over and try to make a reproducible test case. But with just the segfault info not much we can do.

The other thing would be to run a debug build so we can extract more data from the crash when it happens.

Thanks

Hi,
   Thanks for your advices. It shoud be a reasonable approach for me to running a debug build at this point. Can any where i get a mysqlrouter debug build release? Thanks again.

Hi,
   Thanks for your advices. It shoud be a reasonable approach for me to running a debug build at this point. Can any where i get a mysqlrouter debug build release? Thanks again.