Bug #77308 Assertion failure in file fil0fil.c line 859 - Unable to lock ibd, error: 37
Submitted: 11 Jun 2015 8:07 Modified: 12 Mar 2016 5:15
Reporter: Stephan Großberndt Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S3 (Non-critical)
Version:5.5.43 OS:Any
Assigned to: CPU Architecture:Any

[11 Jun 2015 8:07] Stephan Großberndt
Description:
On a virtual host in a debian wheezy os.
The host system is configured with a numflock limit of 1000 for each guest.

ulimit -n gives 1024 by default.

InnoDB is configured with innodb_file_per_table=1

There are ~40 databases with ~100-200 tables each on the instance

If ulimit -n, innodb_open_files, innodb_io_capacity, open_files_limit are all configured to be >1000 and the numflock limit is reached, MySQL crashes.

If all tables are MyISAM this does not happen.

InnoDB: Unable to lock ./demo/tx_extbase_cache_object.ibd, error: 37
150610  9:26:02  InnoDB: Assertion failure in thread 140654092109568 in file fil0fil.c line 859
InnoDB: Failing assertion: ret
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
150610  9:26:02 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see http://kb.askmonty.org/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 5.5.43-MariaDB-1~wheezy-log
key_buffer_size=67108864
read_buffer_size=3145728
max_used_connections=41
max_threads=302
thread_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1926397 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0x7fed8ac98000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fec95323e50 thread_stack 0x48000
/usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7fee9fc9cceb]
/usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7fee9f8d35c2]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7fee9efc90a0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7fee9d826165]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x180)[0x7fee9d8293e0]
/usr/sbin/mysqld(+0x703e77)[0x7fee9fafbe77]
/usr/sbin/mysqld(+0x704079)[0x7fee9fafc079]
/usr/sbin/mysqld(+0x70ff14)[0x7fee9fb07f14]
/usr/sbin/mysqld(+0x6de3d7)[0x7fee9fad63d7]
/usr/sbin/mysqld(+0x6df1da)[0x7fee9fad71da]
/usr/sbin/mysqld(+0x6c7398)[0x7fee9fabf398]
/usr/sbin/mysqld(+0x69ecba)[0x7fee9fa96cba]
/usr/sbin/mysqld(+0x6a00f7)[0x7fee9fa980f7]
/usr/sbin/mysqld(+0x6f1caa)[0x7fee9fae9caa]
/usr/sbin/mysqld(+0x6f3180)[0x7fee9faeb180]
/usr/sbin/mysqld(+0x63f797)[0x7fee9fa37797]
/usr/sbin/mysqld(_ZN7handler7ha_openEP5TABLEPKcij+0x3e)[0x7fee9f8d744e]
/usr/sbin/mysqld(_Z21open_table_from_shareP3THDP11TABLE_SHAREPKcjjjP5TABLEb+0x6dc)[0x7fee9f82f20c]
/usr/sbin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP11st_mem_rootP18Open_table_context+0x93b)[0x7fee9f75c48b]
/usr/sbin/mysqld(_Z11open_tablesP3THDPP10TABLE_LISTPjjP19Prelocking_strategy+0x5c5)[0x7fee9f75d0a5]
/usr/sbin/mysqld(_Z20open_and_lock_tablesP3THDP10TABLE_LISTbjP19Prelocking_strategy+0x44)[0x7fee9f75dd84]
/usr/sbin/mysqld(+0x467e6e)[0x7fee9f85fe6e]
/usr/sbin/mysqld(_ZN22Repair_table_statement7executeEP3THD+0xe8)[0x7fee9f861558]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1921)[0x7fee9f7988b1]
/usr/sbin/mysqld(+0x3a4ea7)[0x7fee9f79cea7]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1b51)[0x7fee9f79f331]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x21b)[0x7fee9f8550ab]
/usr/sbin/mysqld(handle_one_connection+0x51)[0x7fee9f855131]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7fee9efc0b50]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fee9d8cf95d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fed0a8d8018): is an invalid pointer
Connection ID (thread ID): 19754
Status: NOT_KILLED

How to repeat:
Create an instance with more than 1000 innodb tables, set numflock to 1000, run mysqlcheck -A

Suggested fix:
MySQL should not crash - this leads to crashed MyISAM tables. The numflock limit should be taken into account.
[11 Jun 2015 8:28] Stephan Großberndt
How to fix temporarily: 

set ulimit -n, innodb_open_files, innodb_io_capacity, open_files_limit to <1000
[12 Mar 2016 5:15] Umesh Shastry
Hello Stephan,

Thank you for the report.
I'm not seeing this issue with reported version and latest GA builds.
If you can provide more information, feel free to add it to this bug and change the status back to 'Open'.

Thank you for your interest in MySQL.

Thanks,
Umesh