Bug #72437 Error reading GTIDs from binary log: -1
Submitted: 24 Apr 2014 12:50 Modified: 24 Aug 2018 19:02
Reporter: Shahriyar Rzayev (OCA) Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S1 (Critical)
Version:5.6.17, 5.6.19 OS:Linux (CentOS 6.5)
Assigned to: CPU Architecture:Any

[24 Apr 2014 12:50] Shahriyar Rzayev
Description:
Dear experts,

CDbConnection failed to open the DB connection: SQLSTATE[HY000] [2013] Lost connection to MySQL server at 'reading initial communication packet', system error: 111 

Quite interesting error for PHP developers that's why they argued momentarily.
As after examination we found that is not because of Firewall issue that , most of google search results said.

At the end  we can figure out that, due to partitioning issue in Linux Server MySQL has rised disk full error. Here is a portion from error log:

2014-04-24 14:42:47 20672 [Warning] Disk is full writing '/var/lib/mysql/data/mysql-bin.000045' (Errcode: 28 - No space left on device). Waiting for someone to free space...

Also There is a different error messages:

2014-04-24 14:39:17 19994 [ERROR] /usr/sbin/mysqld: Incorrect key file for table '/tmp/#sql_4e1a_1.MYI'; try to repair it
2014-04-24 14:39:17 19994 [ERROR] Got an error from unknown thread, /pb2/build/sb_0-11763321-1394824719.47/rpm/BUILD/mysql-5.6.17/mysql-5.6.17/storage/myisam/mi_write.c:226

Also i think it is because of full /tmp disk too.
Another interesting information from error log:

2014-04-24 14:42:45 20672 [Note] Recovering after a crash using /var/lib/mysql/data/mysql-bin
2014-04-24 14:42:45 20672 [ERROR] Error in Log_event::read_log_event(): 'read error', data_len: 8214, event_type: 31
2014-04-24 14:42:45 20672 [Note] Starting crash recovery...
2014-04-24 14:42:45 20672 [Note] Found 1 prepared transaction(s) in InnoDB
2014-04-24 14:42:45 20672 [Note] Crash recovery finished.
2014-04-24 14:42:45 20672 [Note] Crashed binlog file /var/lib/mysql/data/mysql-bin.000044 size is 7725056, but recovered up to 7669199. Binlog trimmed to 7669199 bytes.

Further reading error log:

2014-04-24 14:42:47 20672 [ERROR] Error in Log_event::read_log_event(): 'read error', data_len: 2070, event_type: 29
2014-04-24 14:42:47 20672 [Warning] Error reading GTIDs from binary log: -1

"-1"  what is "-1" here?

So going on:

In server /home directory there is enough space to moving all binary logs:
/dev/mapper/VolGroup-lv_home 418G  available.

I tried to change binary log path to /home/data.

mkdir data
chown -R mysql:mysql data

[root@slavesrv1 ~]# ls -l /home | grep data
drwxr-xr-x. 2 mysql mysql       4096 2014-04-24 15:44 data

moved all bin_log files to /home/data after Shutting down MySQL..

[root@slavesrv1 ~]# ls -l /home/data/
total 41790100
-rwxr-xr-x. 1 mysql mysql 1074426588 2014-04-24 15:08 mysql-bin.000001
-rwxr-xr-x. 1 mysql mysql 1074200519 2014-04-24 15:09 mysql-bin.000002
-rwxr-xr-x. 1 mysql mysql 1075578308 2014-04-24 15:09 mysql-bin.000003
.
.
Keep only mysql-bin.index file which i manually updated to new path.

[root@slavesrv1 data]# pwd
/var/lib/mysql/data
[root@slavesrv1 data]# ls
mysql-bin.index  mysql-slow.log

But When i start the server it says that permission denied.
But as you see /home/data owner is mysql user.

This server is slave server. So i loose my slave and at this moment can not fix this issue.

How to repeat:
Repeating maybe impossible.
But i setup replication setup with one master and one slave server.
Based on error from PHP script:

CDbConnection failed to open the DB connection: SQLSTATE[HY000] [2013] Lost connection to MySQL server at 'reading initial communication packet', system error: 111 

Figure out that it was because of disk full error. Also MySQL crashes and recovery doesn't work.
After i want to change bin_log dir to /home/data folder but it was impossible too.
So make a replication setup on CEntos with 5.6.17 , make a full disk error on Slave side. Also Performance Schema is enabled on both servers.

Suggested fix:
Can not find exact solution.
Will provide full error log.
[24 Apr 2014 12:54] Shahriyar Rzayev
Full Slave Error Log File

Attachment: slavesrv1.err (application/octet-stream, text), 106.63 KiB.

[28 Apr 2014 15:58] Shahriyar Rzayev
Dear all,
this failed slave server  was a production one and due to Linux hard drive partition problem we have to re-install Linux.
sadly i will not be able to provide you any further information.

But from previous BUG reports i know that in investigation ibdata* files will be helpful.
So at least now i can provide you with ibdata file.
Thank you.
[28 Apr 2014 16:17] Shahriyar Rzayev
Added mysql-bug-ibdata1-72437.zip file to sftp.oracle.com:/support/incoming
[22 Jul 2014 6:23] Shahriyar Rzayev
Could reproduce same error with single MySQL instance version 5.6.19 (with GTID enabled):

2014-07-21 09:55:42 3064 [ERROR] Error in Log_event::read_log_event(): 'read error', data_len: 8105, event_type: 30
2014-07-21 09:55:42 3064 [Warning] Error reading GTIDs from binary log: -1
[24 Jul 2018 19:02] Miguel Solorzano
Sorry for the delay. Please check with latest release. Thanks.
[25 Aug 2018 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".