| Bug #82019 | Is client library supposed to retry EINTR indefinitely or not | ||
|---|---|---|---|
| Submitted: | 27 Jun 2016 14:04 | Modified: | 12 Aug 2016 17:12 |
| Reporter: | Laurynas Biveinis (OCA) | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Connection Handling | Severity: | S3 (Non-critical) |
| Version: | 5.5, 5.6, 5.7 | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
| Tags: | EINTR, libmysql | ||
[27 Jun 2016 17:06]
MySQL Verification Team
Hi Laurynas, Your analysis is quite correct. Let me introduce you to the history of this part of code. Some early Linux kernels, most of all Red Hat ones, did send a lot of EINTR which were to be ignored. Situation is fixed now and OS function `recv` is called in the VIO layer. This, however requires to be documented properly.
[27 Jun 2016 19:19]
Laurynas Biveinis
One thing I have seen was that running at least one client program (5.5 mysqlimport) with AddressSanitizer apparently makes it observe enough EINTRs to abort early
[28 Jun 2016 4:54]
MySQL Verification Team
perhaps this is interesting too: http://bugs.mysql.com/bug.php?id=81258 (please document the mysql_options() MYSQL_OPT_RETRY_COUNT)
[28 Jun 2016 14:07]
MySQL Verification Team
This should be all fixed in 5.7.
[11 Jul 2016 14:05]
Larry Adams
Can someone be specific on what versions of the various clients this is fixed in? Is it MySQL 5.5-XX, MySQL 5.6-XX, MySQL 5.7-XX? I'm using version 5.7-12-1, and do not see either the references to MYSQL_OPT_RETRY_COUNT, or the blocking of EINTR calls, and the workaround is ..... horrible.
[12 Jul 2016 13:55]
MySQL Verification Team
Hi, Due to the series of unfortunate mishaps, this bug has yet to be fixed in 5.5, 5.6 and 5.7. You will be informed on this page when it is achieved.
[15 Jul 2016 13:22]
Laurynas Biveinis
Bug 82019 fix for 5.5 (*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.
Contribution: bug82019-5.5.patch (application/octet-stream, text), 1.06 KiB.
[15 Jul 2016 13:23]
Laurynas Biveinis
Bug 82019 fix for 5.6 (*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.
Contribution: bug82019-5.6.patch (application/octet-stream, text), 898 bytes.
[15 Jul 2016 13:23]
Laurynas Biveinis
Bug 82019 fix for 5.7 (*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.
Contribution: bug82019-5.7.patch (application/octet-stream, text), 893 bytes.
[15 Jul 2016 13:40]
Larry Adams
Thanks for that. Can Oracle provide a release schedule for these updates? TheWitness
[15 Jul 2016 14:08]
MySQL Verification Team
Laurynas, Thank you for your contributions, which are welcome although we made lots of progress already. When the changes will be pushed , this bug report will be closed by Development , with the info of the releases where the patches are pushed into.
[12 Aug 2016 17:12]
Paul DuBois
Posted by developer: Noted in 5.5.52, 5.6.34, 5.7.15 changelogs. EINTR handling in the client library has been fixed so that interrupted read and write calls are retried. Previously, EINTR was ignored.
[12 Aug 2016 17:17]
Paul DuBois
5.6.33, not 5.6.34.
[12 Aug 2016 18:16]
Larry Adams
Thanks everyone for getting this critical bug addressed!
[31 Aug 2016 12:51]
Ståle Deraas
Thanks Laurynas for the contributions. We changed them a bit.
[22 Sep 2016 21:18]
Larry Adams
I just tested using 5.7.15, and this is NOT fixed in the client library. Darn. I'm still receiving EINTR errors for various client calls. I can reproduce 100% of the time. I someone in Oracle would like a test case, you can reach out via my registered email. Can we please re-open this ticket? TheWitness
[22 Sep 2016 22:14]
Larry Adams
Adding additional information. The error seems to be taking place in the mysql_real_connect() call only at the moment. The other API calls are behaving better now. Looking into that code, is it also using the vio subsystem?
[22 Sep 2016 22:43]
Roel Van de Paar
Larry, your best bet is to always open a new bug. Re-opening a bug is highly unlikely to happen.
[22 Sep 2016 22:43]
Roel Van de Paar
If you do, please add the URL here for tracking.
[22 Sep 2016 22:46]
Larry Adams
Sure. I'm digging into the code right now. Very little documentation. At some point I'll either have an aha moment, or go eat dinner. Either way, I'll open a new bug and link.
[22 Sep 2016 23:20]
Larry Adams
Here is a link to the new bug. Suspecting somewhere in the low level stuff net_serv.c still. But no idea where. http://bugs.mysql.com/bug.php?id=83109
Description: It is not clear from the source code reading what's the intended behavior for EINTR handling in libmysqlclient, and whether the intended behavior and actual implementation agree. Here's the background: 1) Up to 5.1 inclusive, client library retries EINTR indefinitely if --enable-thread-safe-client was given (which was the default): static ulong my_real_read(NET *net, size_t *complen) ... #if defined(THREAD_SAFE_CLIENT) && !defined(MYSQL_SERVER) if (vio_errno(net->vio) == SOCKET_EINTR) { DBUG_PRINT("warning",("Interrupted read. Retrying...")); continue; } #endif THREAD_SAFE_CLIENT is set by Autoconf. 2) 5.5 removed the threading-related preprocessor symbols in [1], but failed to remove the checks in the source code, effectively disabling the code above. 3) bug 65956 was logged for 5.5 stating the same (EINTR is not retried indefinitely anymore in the client), and there are interesting comments there: 3.1) "[25 Jul 2012 14:42] Davi Arnaut Also, this behavior was changed in MySQL 5.6 and up. For the client library, interrupted I/O operations are now always retried." But 5.6 appears to contain exactly the same EINTR logic as 5.5, only refactored, complete with the same never-defined THREAD_SAFE_CLIENT. 3.2) "[10 Jan 2013 11:07] Erlend Dahl (...) 5.6 has an implementation where read is done indefinitely in case of socket interrupts(func: net_should_retry in net_serv.cc). " Again, no indefinite retry logic in that function unless one assumes that THREAD_SAFE_CLIENT is actually defined. 3.3) "[16 Jan 2013 6:50] xiaobin lin (...) Davi , you are right, in 5.6, SA_RESTART makes the system call "recv " locks like blocking." But nothing in the 5.6 source tree appears to ever set SA_RESTART, and if it did, it would appear that it would replace EINTR handling only partially. 3.4) in the same comment, "But in 5.5, defined(THREAD_SAFE_CLIENT) is *ALWARYS* false, so error comes out.", which nobody replied to. 4) Finally, 5.7 removed the never-defined THREAD_SAFE_CLIENT symbol in what appears to be a mechanical cleanup [2]. So, is the client library supposed to retry EINTR indefinitely or not? If yes, does it actually do so, in 5.5/5.6/5.7? [1]: commit ebd24626ca38e7fa1e3da2acdcf88540be70fabe Author: Magne Mahre <magne.mahre@oracle.com> Date: Tue Jan 11 10:07:37 2011 +0100 Remove configuration preprocessor symbols 'THREAD' and 'THREAD_SAFE_CLIENT'. As of MySQL 5.5, we no longer support non-threaded builds. This patch removes all references to the obsolete THREAD and THREAD_SAFE_CLIENT preprocessor symbols. These were used to distinguish between threaded and non-threaded builds. [2]: commit de956a045830da4bd68cc870aa42c35177fdad4a Author: Jon Olav Hauglid <jon.hauglid@oracle.com> Date: Fri May 24 15:50:06 2013 +0200 Bug#16792349: YET MORE UNUSED #IFDEFS Additional patch, remove the following unused #ifdefs: DONT_ALLOW_UNSIGNED_PLUS IN_THE_FUTURE LIKE_CMP_TOUPPER TO_BE_REMOVED SECURE_SOCKETS THREAD_SPECIFIC_SIGPIPE IGNORE_SIGHUP_SIGQUIT THREAD_SAFE_CLIENT SHOULD_BE_CHECKED DONT_ALLOW_FULL_LOAD_DATA_PATHS TO_BE_IMPLEMENTED ALLOW_LINESEPARATOR_IN_STRINGS NOT_NEEDED ERROR_ON_NO_SIZEOF_PLUGIN_SYMBOL FIX_LATER RESTRICTED_GROUP QQ_ALL_HANDLERS_SUPPORT_VARCHAR TESTTIME WE_WANT_TO_SUPPORT_VERY_OLD_FRM_FILES TO_BE_DELETED_ON_PRODUCTION WE_WANT_TO_HANDLE_UNORMALIZED_DATES How to repeat: See above Suggested fix: Please clarify what's intended here, and if the actual implementation differs, please fix it.