Bug #33478 mysqld locks up with "too many connections"
Submitted: 22 Dec 2007 10:37 Modified: 25 Dec 2007 11:04
Reporter: Philip Stoev Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:6.0.4 OS:Any
Assigned to: Philip Stoev CPU Architecture:Any

[22 Dec 2007 10:37] Philip Stoev
Description:
When running the 5.1 systems test, scenario iuds6, with storage engine Falcon, mysqld locks up and refuses further connections with message "too many connections", even though the number of threads being used in the test is 50, which is nowhere near the default setting for max_connections of 100.

Even though the documentation states that an extra connection is available for the SUPER user, connection attemps using -uroot resulted in the same error (though the test may have removed that account or its SUPER privilige).

Killing all processes connecting to the server did not cause the message to go away. Here is the netstat output after all processes except the mysqld were cleared:

[qauser@dl360-g5-a log]$ netstat -p
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
.....
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node PID/Program name    Path
unix  2      [ ]         STREAM     CONNECTED     39431655 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39431250 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429695 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429654 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429573 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429499 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429436 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429389 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429333 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429294 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429196 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429105 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39429029 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428974 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428849 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428807 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428720 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428636 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428538 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428523 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428448 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428437 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428365 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428354 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428268 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428262 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428155 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428146 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428046 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39428038 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39427933 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39427927 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39427906 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39427883 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39427853 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39427820 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock
unix  2      [ ]         STREAM     CONNECTED     39427710 12965/mysqld        /data1/6.0.4/systest_vardir/tmp/master.sock

a total of less than 50 open sockets.

How to repeat:
Run the 5.1 systems test with iuds6 scenario and Falcon engine. I will be working to provide a simpler scenario.