MySQL Bugs: #90443: mysqlsh using 100% cpu after tty is gone

Bug #90443	mysqlsh using 100% cpu after tty is gone
Submitted:	16 Apr 2018 6:24	Modified:	17 Apr 2018 15:12
Reporter:	Kenny Gryp	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	Shell AdminAPI InnoDB Cluster / ReplicaSet	Severity:	S3 (Non-critical)
Version:	8.0.4	OS:	Any
Assigned to:	MySQL Verification Team	CPU Architecture:	Any

Description:
While testing mysqlsh/mysql group replication I found a mysqlsh using 100% on one of my nodes, apparently that mysqlsh was from a ssh/bash session that I killed before.

 # top
 PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                                                                                                                          
 8222 root      20   0  609928  12432   7972 R 99.3  1.2  47:46.77 mysqlsh                                                                                                                                                          

# sudo lsof -p 8222

read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0
read(0, "", 1)                          = 0

reading from 0

[vagrant@node2 ~]$ sudo lsof -p 8222
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
mysqlsh 8222 root  cwd    DIR  253,0      148 17855859 /home/vagrant
mysqlsh 8222 root  rtd    DIR  253,0      239       64 /
mysqlsh 8222 root  txt    REG  253,0 15921184  8388918 /usr/bin/mysqlsh
mysqlsh 8222 root  mem    REG  253,0    62184 17097839 /usr/lib64/libnss_files-2.17.so
mysqlsh 8222 root  mem    REG  253,0    14872 17104228 /usr/lib64/libutil-2.17.so
mysqlsh 8222 root  mem    REG  253,0  2127336 17097820 /usr/lib64/libc-2.17.so
mysqlsh 8222 root  mem    REG  253,0    88720 16820959 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
mysqlsh 8222 root  mem    REG  253,0  1139680 17097828 /usr/lib64/libm-2.17.so
mysqlsh 8222 root  mem    REG  253,0   995840 17104248 /usr/lib64/libstdc++.so.6.0.19
mysqlsh 8222 root  mem    REG  253,0   174576 17104283 /usr/lib64/libtinfo.so.5.9
mysqlsh 8222 root  mem    REG  253,0   163696 17104273 /usr/lib64/libncurses.so.5.9
mysqlsh 8222 root  mem    REG  253,0    19776 17097826 /usr/lib64/libdl-2.17.so
mysqlsh 8222 root  mem    REG  253,0    44448 17104224 /usr/lib64/librt-2.17.so
mysqlsh 8222 root  mem    REG  253,0  1847240 16797898 /usr/lib64/libpython2.7.so.1.0
mysqlsh 8222 root  mem    REG  253,0   144792 17097848 /usr/lib64/libpthread-2.17.so
mysqlsh 8222 root  mem    REG  253,0   164112 17097812 /usr/lib64/ld-2.17.so
mysqlsh 8222 root    0u   CHR  136,0      0t0        3 /dev/pts/0 (deleted)
mysqlsh 8222 root    1u   CHR  136,0      0t0        3 /dev/pts/0 (deleted)
mysqlsh 8222 root    2u   CHR  136,0      0t0        3 /dev/pts/0 (deleted)
mysqlsh 8222 root    3w   REG  253,0     4793  8814150 /root/.mysqlsh/mysqlsh.log
mysqlsh 8222 root    4u  sock    0,7      0t0    72463 protocol: TCP
mysqlsh 8222 root    5r   CHR    1,9      0t0     4792 /dev/urandom
mysqlsh 8222 root    6r   CHR    1,9      0t0     4792 /dev/urandom

however 0 is /dev/pts/0, which does not longer exist.

How to repeat:
It seems to be related to mysqlsh running through an SSH session which at some point got aborted.
mysqlsh still thinks the tty is there, tries to read from it, but fails and is stuck in a loop trying to read it

I tried in various ways to repeat it, I know I did a suspend of the virtual machine, but broken/idle TCP connections are hard to kill. giving up.

Suggested fix:
 end the mysqlsh process instead of wasting all that cpu

Are you able to reproduce this? Was the shell session doing something at the time?

Here's what I'm trying:

ssh to remote Linux host
start mysqlsh
one of:
   terminate ssh session with ~.
   kill parent sshd process 
   kill parent bash process

in all cases, the shell process just disappears immediately.

@Alfredo: Yeah that doesn't work. I tried that too.

Nothing was happening on the shell as far as I know. The only thing possible is that is might have been waiting on user input (like a password)

Hi,
I tried everything I can think of (including running in vm and suspending vm's, running on real hw and killing the network physically, running on vm using double switch killing the middle one so that physically network exist but data is not going trough ...) and I can't reproduce this, so this might be some fluke on your system at the time and not a bug. If you reproduce this again please remember what you did :D

thanks
Bogdan