MySQL Bugs: #40077: "mysqld got signal 10" error message after adding a new collation.

Bug #40077	"mysqld got signal 10" error message after adding a new collation.
Submitted:	16 Oct 2008 12:48	Modified:	27 Oct 2008 9:38
Reporter:	Eric Tchepannou	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: Charsets	Severity:	S2 (Serious)
Version:	5.0.67, 5.1.28	OS:	MacOS
Assigned to:	Sveta Smirnova	CPU Architecture:	Any
Tags:	collation, mysqlimport

Description:
My experience with this has not been very successful. I have successfully followed the steps described here, but adding a new collation had a very bad impact on my MySQL installation. I am importing data using mysqlimport and only now (after adding the new collation) I keep getting the error message below. I have been able to reproduce it on 5.0.67 and not 5.1.28-rc on Mac OS.

<error>
081016 12:41:52 - mysqld got signal 10 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=2
max_threads=151
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 337748 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
</error>

How to repeat:
Modify the utf8 charset description (in file $MYSQL_HOME/share/mysql/charsets/Index.xml) as below, restart mysql. 
To reproduce, I use mysqlimport to load data from a CSV file containing 17000+ rows.
Because of the crashes, I had to split the files in 80-rows sub-files, and repeatedly imported them. Unfortunately, I still get the same error after a certain number of imports. I have tried to clear the resources after each import using mysqladmin flush-tables, but this did not help.

<charset name="utf8">
  <family>Unicode</family>
  <description>UTF-8 Unicode</description>
  <alias>utf-8</alias>
  <collation name="utf8_general_ci"	id="33">
   <flag>primary</flag>
   <flag>compiled</flag>
  </collation>
  <collation name="utf8_bin"		id="83">
    <flag>binary</flag>
    <flag>compiled</flag>
  </collation>
  <!-- added by ET -->
  <collation name="utf8_is" id="56">
    <rules>
      <reset>u</reset><p>\u00FC</p>
      <reset>e</reset><p>\u00E9</p>
      <reset>a</reset><p>\u00E0</p>
      <reset>c</reset><p>\u010C</p><s>\u010D</s>
    </rules>
  </collation>
  <!-- end -->
</charset>

Here is the URL this bug is referring to:
http://dev.mysql.com/doc/refman/5.1/en/adding-collation-unicode-uca.html

I have now decreased the variable's values, but could not get any further. The only change is that mysqladmin is now reporting a lot of slow queries.

<error>
081016 15:00:01 - mysqld got signal 10 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=81920
read_buffer_size=131072
max_used_connections=1
max_threads=151
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 25223 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
</error>

Thank you for the report.

If I understood correctly you hit this error with version 5.0.67 and don't hit with version 5.1.28. Please correct me if I am wrong.

Additionally I can not repeat described behavior with test table and data. Please try to find which query causes the problem and provide it and output of SHOW CREATE TABLE for all tables involved.

Hi,

I was able to reproduce the problem with both versions.

This can only be reproduced with lots of data. I am importing 17k rows from a 1.4 MB csv file with utf-8/LF encoding. The file contains usernames, emails, passwords, firstnames and lastnames. The target MySQL DB table has a column for each value.

This task is part of the migration of user data from an MS SQL Server db to MySQL. The original CSV file gained from MS SQL is in utf-16, so I had to convert it in utf-8 first. Here, when I import it, the system cannot make the difference between usernames 'günni' and 'gunni', thus skips the second entry.
I was hoping to solve this problem by creating my own collation but now, after 5500+ rows, I get the errors reported. As next step, I have splitted the files in 80-rows chunks and with a help of a script, I try to import all of them. Same behaviour.

Also, I found out that when my Java (Hibernate) app establishes the connection with the DB, this one crashes with the same error message.

Hope this helps.

Regards,

Eric

Thank you for the feedback.

Please provide output of SHOW CREATE TABLE for table you have problems with. Also please check if error 10 still occurs with utf8_unicode_ci and utf8_general_ci collations to be sure problem is in custom collation.

Hi,

The problem occurs only with a custom collation. I cannot provide you the output you are requesting because I had to change back to utf8 and the utf8_general_uci.

Thank you for the feedback.

But why can't you provide output of SHOW CREATE TABLE? You can add it as hidden comment or replace names of columns with dummy ones. But other information such as charset, collations, indexes, storage engine and table settings is important.

Here you go. Note that I am not using the custom collation anymore because of the reported problem.

CREATE TABLE `t_user` (
  `id` int(11) NOT NULL auto_increment,
  `email` varchar(255) NOT NULL default '',
  `firstName` varchar(50) default NULL,
  `lastName` varchar(50) default NULL,
  `name` varchar(20) NOT NULL default '',
  `password` varchar(10) default NULL,
  PRIMARY KEY  (`id`),
  UNIQUE KEY `email` (`email`),
  UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=18398 DEFAULT CHARSET=utf8

Thank you for the feedback.

I still can not repeat described behavior with 34000 rows/3M file. Could you please provide your configuration file. Also please indicate how much physical RAM do you have.

Hi,
I have 4GB ram on my machine.
At the time the problem occurred, mySQL was running without configuration file. I guess it was using the default properties...

Right now, I have following config. file in /etc/my.cnf
[mysqld]
#Set the default character set.
default-character-set=utf8
#Set the default collation.
default-collation=utf8_unicode_ci

Thank you for the feedback.

Unfortunately I can not repeat crash, so I close the report as "Can't repeat" If you try custom collation another time please generate and provide core file so we can find more information and reopen the report.