Bug #43490 | Falcon internal thread terminate after throwing an instance of 'SQLError' | ||
---|---|---|---|
Submitted: | 9 Mar 2009 7:57 | Modified: | 26 May 2010 17:52 |
Reporter: | Olav Sandstå | Email Updates: | |
Status: | Unsupported | Impact on me: | |
Category: | MySQL Server: Falcon storage engine | Severity: | S2 (Serious) |
Version: | 6.0.11-alpha | OS: | Any |
Assigned to: | Olav Sandstå | CPU Architecture: | Any |
Tags: | F_ERROR HANDLING |
[9 Mar 2009 7:57]
Olav Sandstå
[24 Jun 2009 13:32]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/77044 2742 Olav.Sandstaa@sun.com 2009-06-24 Bug #43490 Falcon internal thread terminate after throwing an instance of 'SQLError' When this error occurs MySQL terminates and all that is written to the log file is that there was on uncaught SQLError exception. The Thread class has code for writing out the exception string but unless the correct debug flag is specified nothing is written to the log file. This patch changes the Thread class to use Log::fatal() for writing the excpetion's string so that it is more likely that it get written to the log before the process is terminated. The patch does not fix the problem but will hopefully make it easier to understand what caused this exception to occure.
[24 Jun 2009 15:52]
John Embretsen
Olav, the patch looks safe to push. I hope it helps in order to figure out what is going on in these cases.
[24 Jun 2009 18:52]
Olav Sandstå
Patch that improves the error reporting is pushed. Setting status back to "In progress".
[25 Jun 2009 8:48]
Olav Sandstå
With the latest patch in place we now get the following written to the log when this crash happens when running falcon_ddl: Thread::thread: thread 1095825728: can't continue after fatal error terminate called after throwing an instance of 'SQLError' This does not say much about what the actual error causing this was. Still, it gives us that this is related to a disk IO problem. This exception is only thrown from IO.cpp during either writing or reading a block to/from the data device. Seems like there is more room for improvements in how we log information about this in the error log.
[26 Jun 2009 11:14]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/77310 2748 Olav.Sandstaa@sun.com 2009-06-26 Bug#43490 Falcon internal thread terminate after throwing an instance of 'SQLError' The thrown exception contains the error string "can't continue after fatal error ". This exception is only thrown by the Falcon IO system. It is thrown after a fatal IO error has occurred. Unfortunately it gives very little information about what the actual error was. This patch improves the logging of "fatal" IO errors by writing to the error log file information about the error when it occurs. With this patch a message similar to this example will be written when IO::declareFatalError() is called: Falcon: Fatal IO error occurred in "IO::writePages": file "/home/olav/mysql/develop/falcon-elog/mysql-test/var/mysqld.1/data/falcon_user.fts", page 21: No space left on device (28) Note that this patch does not fix the problem but only improves the logging when the actually error occurs, hopefully making it easier to find the real error.
[26 Jun 2009 20:55]
Olav Sandstå
Patch for adding logging to error log for fatal IO errors is pushed to mysql-6.0-falcon-team. Setting status back to "In progress".
[28 Aug 2009 10:23]
Olav Sandstå
Here is the new output from a new similar crash that includes the extra logging introduced by the previous patches: # 19:53:00 Falcon: Fatal IO error occurred in "IO::writePages": file "/export/home/pb2/test/sb_1-710226-1251307967.68/mysql-6.0.12-alpha-linux-i686-test/vardirs/master-data_recovery/falcon_temporary.fts", page 1: Input/output error (5) # 19:53:00 Exception: write error on page 1 (4096/4096/7) of "/export/home/pb2/test/sb_1-710226-1251307967.68/mysql-6.0.12-alpha-linux-i686-test/vardirs/master-data_recovery/falcon_temporary.fts": Input/output error (5) # 19:53:00 Thread::thread: thread -1208513648: write error on page 1 (4096/4096/7) of "/export/home/pb2/test/sb_1-710226-1251307967.68/mysql-6.0.12-alpha-linux-i686-test/vardirs/master-data_recovery/falcon_temporary.fts": Input/output error (5) # 19:53:00 terminate called after throwing an instance of 'SQLError' # 19:53:00 090826 19:53:00 - mysqld got signal 6 ;