Bug #34540 Table crash : Got error 127 when reading table

Description:
Dear all,

here is a problem on a specific table (myISAM) I cannot solve on gentoo linux.
(similar to bug 26939 but with more recents mysql and libs versions)

/*****************/
* OS and libs
/******************/
Linux gentoo x86_64 version 1.6.14

Kernel 2.6.21.5-grsec-xxxx-grs-ipv4-64 #6 SMP 

Proc x86_64 Intel(R) Core(TM)2 Duo CPU E6750 @ 2.66GHz GenuineIntel GNU/Linux

RAM 4GB

Swap 500MB

GNU_LIBPTHREAD_VERSION
NPTL 2.3.6

GNU_LIBC_VERSION
glibc 2.3.6

/*****************************/
*Table structure
/*****************************/
 INFOS_INTERNAUTES | CREATE TABLE `INFOS_INTERNAUTES` (
  `ID_INFOS_INTERNAUTES` mediumint(8) unsigned NOT NULL auto_increment,
  `IP` varchar(50) NOT NULL,
  `USER-AGENT` varchar(150) NOT NULL,
  `PAYS` char(2) default NULL,
  `FAI` varchar(20) default NULL,
  `REFERER` varchar(250) NOT NULL,
  `DATE_PREMIERE_CONNEXION` datetime NOT NULL,
  `DATE_DERNIERE_CONNEXION` datetime default NULL,
  `PAGE_ENTREE` varchar(150) NOT NULL,
  `PAGE_SORTIE` varchar(150) default NULL,
  `SOURCES_TRAFIC_ID` tinyint(3) unsigned NOT NULL default '1',
  `CAMPAGNES_ID` tinyint(3) unsigned NOT NULL default '1',
  `BANNIERES_ID` tinyint(3) unsigned default NULL,
  PRIMARY KEY  (`ID_INFOS_INTERNAUTES`),
  KEY `IP` (`IP`,`REFERER`,`DATE_PREMIERE_CONNEXION`,`DATE_DERNIERE_CONNEXION`,`PAGE_ENTREE`,`PAGE_SORTIE`,`SOURCES_TRAFIC_ID`,`CAMPAGNES_ID`,`BANNIERES_ID`)
) ENGINE=MyISAM AUTO_INCREMENT=927928 DEFAULT CHARSET=latin1

/*****************************/
* my.cnf
/*****************************/
[mysqld_safe]
err-log                                         = /var/log/mysql/mysql.err

[mysqld]
character-set-server            = latin1
init-connect='SET NAMES  latin1'
default-character-set           = latin1
user                                            = mysql
port                                            = 3306
socket                                          = /var/run/mysqld/mysqld.sock
pid-file                                        = /var/run/mysqld/mysqld.pid
log-error                                       = /var/log/mysql/mysqld.err
basedir                                         = /usr/local/mysql
datadir                                         = /var/lib/mysql
skip-locking
language                                        = /usr/local/mysql/share/mysql/english
skip-locking
key_buffer = 256M
max_allowed_packet = 1M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 4
skip-networking
log-bin
server-id                                       = 1
tmpdir                                          = /tmp/
skip-innodb
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
innodb_data_file_path = ibdata1:10M:autoextend:max:128M
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
set-variable = innodb_log_files_in_group=2
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
log-bin                 = /var/run/mysqld/mysqld-bin
expire_logs_days        = 10
max_binlog_size         = 350M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 4
skip-networking
log-bin
server-id                                       = 1
tmpdir                                          = /tmp/
skip-innodb
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
innodb_data_file_path = ibdata1:10M:autoextend:max:128M
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
set-variable = innodb_log_files_in_group=2
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
log-bin                 = /var/run/mysqld/mysqld-bin
expire_logs_days        = 10
max_binlog_size         = 350M
log-slow-queries        = /var/log/mysql/mysqld-slow.log

[mysqldump]
quick
max_allowed_packet                      = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

/***********************/
* TESTS
/***********************/

*In table INFOS_INTERNAUTES, we have more than 927928 rows (and growing again) .

*With mysqld version 5.0.44  :
 in mysqld error log, I have received many times :
  [ERROR] Got error 127 when reading table './orgsex/INFOS_INTERNAUTES'
 Must run a repair table to avoid this problem. This error occurs when mysqld received many write/read requests on this table.

*Upgrading to mysqld version 5.0.51a x64 glibc-2.3 (the last official community server binary)
 in mysqld error log, with big write/read requests context :
  [ERROR] /usr/local/mysql/sbin/mysqld: Incorrect key file for table './orgsex/INFOS_INTERNAUTES.MYI'
; try to repair it

Best regards

Michael

How to repeat:
Don't know how to repeat bug.

Probably sending many write requests on the same time to have concurrent writes (I will try this asap)

Many thanks for writing a bug report.

What happens before the crash?
Has the error log some informations about this?

Also, are there entries at the sytem logs?

I have checked error log and nothing happen before message :
080214  2:22:39 [ERROR] /usr/local/mysql/sbin/mysqld: Incorrect key file for table './orgdb/INFOS_INTERNAUTES.MYI'
; try to repair it

When I run a REPAIR TABLE INFOS_INTERNAUTES; after crash :

080214 11:33:04 [Note] Found 918574 of 918575 rows when repairing './orgdb/INFOS_INTERNAUTES'

There is one row missing

Nothing found in system logs (kern.log, syslog, ...) .

I will try to repeat this bug quickly and report my test here.

Thank you

I have more details when doing a repair table :

orgdb.INFOS_INTERNAUTES        repair  info    Wrong bytesec:   0-  0-  0 at 262155912; Skipped
orgdb.INFOS_INTERNAUTES        repair  warning Number of rows changed from 937949 to 937948
orgdb.INFOS_INTERNAUTES        repair  status  OK

Thank you for the feedback.

Do you have repeatable 127 errors with version 5.0.51a after you have repaired the table?

If yes, have you tried to set thread_cache_size=0 as Shane Bester suggested in bug #26939 If haven't, please do it and say us result.

I have error 127 again after repair.

So, I have set thread_cache_size = 0 and restarted mysqld.

I will check the result and give you a feedback quickly.

Thanks

Same errors but I have more informations on this problem.

Mysql server 64 bits is on a 64bits gentoo and it seems there are some problems between hardware (hosted by another company) and 64bits versions.

So, we can close this bug and I will see with my hoster to provide us a 64bits working hardware .

Thank you

Best regards

Michaël Ricordeau