Description:
Please see the err log file in the following. The crash happened for the second time on different machines (both HP-UX, Itanium). In both cases, Mysqld failed to write to the first log file (in a group of 2) at offset 0 41983488.
We use C-API. We found no wrong-doings in hardware or os level. The file size, disk dize, quota are all examined.
In our configuration,
Bin log, error log are turned on.
innodb_data_file_path =innodbdata1:6G;innodbdata2:6G:autoextend
innodb_log_group_home_dir = /e911/mysqldata/data
innodb_buffer_pool_size = 800M
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 400M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
The following is from the err log file of the second crash.
-----------------------------------------------------------
060713 17:01:34 InnoDB: Started; log sequence number 0 12693303
/usr/local/mysql/bin/mysqld: ready for connections.
Version: '4.1.18-standard-log' socket: '/tmp/mysql.sock' port: 3306 MySQL Com
munity Edition - Standard (GPL)
060810 21:57:52 InnoDB: Error: Write to file /e911/mysqldata/data/ib_logfile0 f
ailed at offset 0 41983488.
InnoDB: 1024 bytes should have been written, only 512 were written.
InnoDB: Operating system error number 27.
InnoDB: Check that your OS and file system support files of this size.
InnoDB: Check also that the disk is not full or a disk quota exceeded.
InnoDB: Error number 27 means 'File too large'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/mysql/en/Operating_System_error_codes.html
060810 21:57:52InnoDB: Assertion failure in thread 319 in file fil0fil.c line 39
22
InnoDB: Failing assertion: ret
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
InnoDB: Thread 319 stopped in file fil0fil.c line 3922
InnoDB: Thread 7 stopped in file sync0arr.c line 336
InnoDB: Thread 6 stopped in file os0sync.c line 501
-------------------------------------------------------
The following is from the err log file from the first crash on a different machine.
---------------------------------
060713 16:59:36 mysqld started
060713 16:59:36 [Warning] Could not increase number of max_open_files to more than 2048 (request: 751
0)
060713 16:59:38 InnoDB: Started; log sequence number 0 23873583
/usr/local/mysql/bin/mysqld: ready for connections.
Version: '4.1.18-standard-log' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Edition - Stan
dard (GPL)
060731 14:57:06 InnoDB: Error: Write to file /e911/mysqldata/data/ib_logfile0 failed at offset 0 419
83488.
InnoDB: 1024 bytes should have been written, only 512 were written.
InnoDB: Operating system error number 0.
InnoDB: Check that your OS and file system support files of this size.
InnoDB: Check also that the disk is not full or a disk quota exceeded.
InnoDB: Error number 0 means 'Error 0'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/mysql/en/Operating_System_error_codes.html
A
060731 14:57:07InnoDB: Assertion failure in thread 264 in file fil0fil.c line 3922
InnoDB: Failing assertion: ret
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
InnoDB: Thread 264 stopped in file fil0fil.c line 3922
InnoDB: Thread 8 stopped in file ./../include/sync0sync.ic line 111
InnoDB: Thread 6 stopped in file os0sync.c line 501
InnoDB: Thread 7 stopped in file sync0arr.c line 336
…………
How to repeat:
Currently, we don't have a definite way to repeat the crash. It happened for the second time on a different machine on HP-UX Itanium box. When crash happened, all accesses to the database hang.
