Bug #47357 Mysql crashes when repairing ARCHIVE table
Submitted: 16 Sep 2009 9:05 Modified: 22 Oct 2009 7:26
Reporter: Davide Ferrari Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server: Archive storage engine Severity:S1 (Critical)
Version:5.1.36 OS:Linux
Assigned to: CPU Architecture:Any

[16 Sep 2009 9:05] Davide Ferrari
Description:
I was issueing a REPAIR TABLE on every ARCHIVE table I have in a database to detect any corrupt table (I have a few corrupt tables for sure) and mysqld crashed suddenly.

Here is the log

Sep 16 03:44:46  mysqld: 090916  3:44:46 - mysqld got signal 11 ;
Sep 16 03:44:46  mysqld: This could be because you hit a bug. It is also possible that this binary
Sep 16 03:44:46  mysqld: or one of the libraries it was linked against is corrupt, improperly built,
Sep 16 03:44:46  mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
Sep 16 03:44:46  mysqld: We will try our best to scrape up some info that will hopefully help diagnose
Sep 16 03:44:46  mysqld: the problem, but since we have already crashed, something is definitely wrong
Sep 16 03:44:46  mysqld: and this may fail.
Sep 16 03:44:46  mysqld:
Sep 16 03:44:46  mysqld: key_buffer_size=16777216
Sep 16 03:44:46  mysqld: read_buffer_size=131072
Sep 16 03:44:46  mysqld: max_used_connections=1218
Sep 16 03:44:46  mysqld: max_threads=2500
Sep 16 03:44:46  mysqld: threads_connected=347
Sep 16 03:44:46  mysqld: It is possible that mysqld could use up to
Sep 16 03:44:46  mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5481833 K
Sep 16 03:44:46  mysqld: bytes of memory
Sep 16 03:44:46  mysqld: Hope that's ok; if not, decrease some variables in the equation.
Sep 16 03:44:46  mysqld:
Sep 16 03:44:46  mysqld: thd: 0x7fe8a208ea80
Sep 16 03:44:46  mysqld: Attempting backtrace. You can use the following information to find out
Sep 16 03:44:46  mysqld: where mysqld died. If you see no messages after this, something went
Sep 16 03:44:46  mysqld: terribly wrong...
Sep 16 03:44:46  mysqld: stack_bottom = 0x44792100 thread_stack 0x20000
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x24) [0x8c8854]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(handle_segfault+0x322) [0x5fd212]
Sep 16 03:44:46  mysqld: /lib/libpthread.so.0 [0x7fe9cb211a80]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(my_pthread_fastmutex_lock+0xd) [0x8cb68d]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(thr_lock+0x4c) [0x8ca73c]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(thr_multi_lock+0xc4) [0x8cb1a4]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(mysql_lock_tables(THD*, st_table**, unsigned int, unsigned int, bool*)+0x3bc) [0x5f5afc]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(lock_tables(THD*, TABLE_LIST*, unsigned int, bool*)+0x437) [0x642c87]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld [0x7061e6]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(mysql_repair_table(THD*, TABLE_LIST*, st_ha_check_opt*)+0x61) [0x707431]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(mysql_execute_command(THD*)+0x2223) [0x60f833]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x357) [0x613137]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0xe63) [0x613fa3]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(do_command(THD*)+0xe6) [0x614866]
Sep 16 03:44:46  mysqld: /usr/sbin/mysqld(handle_one_connection+0x246) [0x607216]
Sep 16 03:44:46  mysqld: /lib/libpthread.so.0 [0x7fe9cb209fc7]
Sep 16 03:44:46  mysqld: /lib/libc.so.6(clone+0x6d) [0x7fe9c9d655ad]
Sep 16 03:44:46  mysqld: Trying to get some variables.
Sep 16 03:44:46  mysqld: Some pointers may be invalid and cause the dump to abort...
Sep 16 03:44:46  mysqld: thd->query at 0x7fe8a05bfb50 is an invalid pointer
Sep 16 03:44:46  mysqld: thd->thread_id=1692213
Sep 16 03:44:46  mysqld: thd->killed=NOT_KILLED
Sep 16 03:44:46  mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Sep 16 03:44:46  mysqld: information that should help you find out what is causing the crash.
Sep 16 03:44:47  mysqld_safe: Number of processes running now: 0
Sep 16 03:44:46  mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Sep 16 03:44:46  mysqld: information that should help you find out what is causing the crash.
Sep 16 03:44:47  mysqld_safe: Number of processes running now: 0
Sep 16 03:44:47  mysqld_safe: mysqld restarted

It's a production server so it was working at the moment although working only with another database (not the one which contains the corrupted archive tables)

How to repeat:
At the moment I don't have a clear repeat scenario, although repairing a lot of corrupted ARCHIVE tables should do it
[16 Sep 2009 9:41] Valeriy Kravchuk
Thank you for the problem report. Can you try to identify the exact table for which REPAIR crashed?
[18 Sep 2009 9:17] Davide Ferrari
I've have possibly one table but I cannot do anything on this table, not even a show create cause it says:
ERROR 1034 (HY000): Incorrect key file for table 'API_log_requests_090815'; try to repair it

and if I try to repair it

repair table NO_ENTRAR_hist_datalogs.API_log_requests_090815;
+-------------------------------------------------+--------+----------+----------+
| Table                                           | Op     | Msg_type | Msg_text |
+-------------------------------------------------+--------+----------+----------+
| NO_ENTRAR_hist_datalogs.API_log_requests_090815 | repair | error    | Corrupt  |
+-------------------------------------------------+--------+----------+----------+
1 row in set (0.00 sec)
[18 Sep 2009 11:20] Susanne Ebrecht
Did you upgrade from 5.0 or an older 5.1 version?
[18 Sep 2009 11:21] Davide Ferrari
No, they are backup tables generated from InnoDB ones on the same server (with an ALTER TABLE statement)
[21 Sep 2009 8:16] Sveta Smirnova
Thank you for the feedback.

Could you please upload files API_log_requests_090815.ARZ, API_log_requests_090815.ARM and API_log_requests_090815.frm to our FTP server as described in "Files" section?
[21 Sep 2009 8:40] Davide Ferrari
Requested table files uploaded to Mysql's FTP
[21 Sep 2009 9:47] Sveta Smirnova
Thank you for the feedback.

When I try to repair table provided I get not error. But if I convert it to InnoDB, then to Archive I get error "ERROR 1022 (23000): Can't write; duplicate key in table '#sql-73d9_1'" How did you converted InnoDB table to Archive?
[21 Sep 2009 11:04] Davide Ferrari
Ok, sorry for that, this actually was an OK table, I made a little of confusion cause I dropped many corrupted tables trying to get the server running smoothly. I've just uploaded another table, this time is 100% corrupt and REPAIR doesn't repair it. I don't know if it's exactly this table that made the server crash but anyway something went wrong with this and other tables, so if you could take a look at it...
TIA
[21 Sep 2009 18:38] MySQL Verification Team
What is the file name you have uploaded?. Thanks in advance.
[22 Sep 2009 7:26] Sveta Smirnova
Thank you for the feedback.

I still can not repeat crash.

Additionally version 5.1.36 is old. Please try with current version 5.1.39 and if problem still exists indicate accurate name of MySQL package you are using (file name downloaded) and provide your configuration file.
[22 Oct 2009 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".