Bug #10572 Prep'd select concat(<param>,<col>), still wrong for utf8_bin in 4.1.9
Submitted: 12 May 2005 8:00 Modified: 22 Jun 2005 19:42
Reporter: Adrian Corston Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:4.1.9-standard-log OS:Linux (Linux version 2.4.21-27.0.2.EL)
Assigned to: CPU Architecture:Any

[12 May 2005 8:00] Adrian Corston
Description:
Bug 3796 still occurs for collation utf8_bin, even though it does not occur for latin1_swedish_ci.

See bug 3796 for a full description of the bug, which was reportedly fixed in 4.1.3.

How to repeat:
Example 1. utf8_bin - shows incorrect behaviour:
mysql> create table TEST(x varchar(20) collate utf8_bin not null default '');
Query OK, 0 rows affected (0.07 sec)
mysql> insert into TEST values('one');
Query OK, 1 row affected (0.00 sec)
mysql> insert into TEST values('two');
Query OK, 1 row affected (0.00 sec)
mysql> insert into TEST values('three');
Query OK, 1 row affected (0.00 sec)
mysql> select concat('!', x) from TEST;
+----------------+
| concat('!', x) |
+----------------+
| !one           |
| !onetwo        |
| !onetwothree   |
+----------------+
3 rows in set (0.00 sec)

Example 2. latin1_swedish_ci shows correct behaviour:
mysql> create table TEST2(x varchar(20) collate latin1_swedish_ci not null default '');
Query OK, 0 rows affected (0.04 sec)
mysql> insert into TEST2 values('one');
Query OK, 1 row affected (0.00 sec)
mysql> insert into TEST2 values('two');
Query OK, 1 row affected (0.00 sec)
mysql> insert into TEST2 values('three');
Query OK, 1 row affected (0.00 sec)
mysql> select concat('!', x) from TEST2;
+----------------+
| concat('!', x) |
+----------------+
| !one           |
| !two           |
| !three         |
+----------------+
3 rows in set (0.00 sec)

Suggested fix:
Revisit the fix for bug 3796 and see why it didn't fix the problem for the utf8_bin collation.
[12 May 2005 9:21] Hartmut Holzgraefe
I've tested this against both 4.1.9 and current 4.1.11 but could not reproduce the problem
[13 May 2005 0:02] Adrian Corston
Perhaps it's related to the system collation_* variables below.  I can provide a login if that would be useful to you.  I've attached our startup script, error log and my.conf.

mysql> show variables;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    678019
Current database: legacy_cc

+---------------------------------+------------------------------+
| Variable_name                   | Value                        |
+---------------------------------+------------------------------+
| back_log                        | 50                           |
| basedir                         | /                            |
| binlog_cache_size               | 32768                        |
| bulk_insert_buffer_size         | 8388608                      |
| character_set_client            | latin1                       |
| character_set_connection        | latin1                       |
| character_set_database          | utf8                         |
| character_set_results           | latin1                       |
| character_set_server            | utf8                         |
| character_set_system            | utf8                         |
| character_sets_dir              | /usr/share/mysql/charsets/   |
| collation_connection            | latin1_swedish_ci            |
| collation_database              | utf8_bin                     |
| collation_server                | utf8_bin                     |
| concurrent_insert               | ON                           |
| connect_timeout                 | 5                            |
| datadir                         | /abel/var/lib/mysql/         |
| date_format                     | %Y-%m-%d                     |
| datetime_format                 | %Y-%m-%d %H:%i:%s            |
| default_week_format             | 0                            |
| delay_key_write                 | ON                           |
| delayed_insert_limit            | 100                          |
| delayed_insert_timeout          | 300                          |
| delayed_queue_size              | 1000                         |
| expire_logs_days                | 0                            |
| flush                           | OFF                          |
| flush_time                      | 0                            |
| ft_boolean_syntax               | + -><()~*:""&|               |
| ft_max_word_len                 | 84                           |
| ft_min_word_len                 | 4                            |
| ft_query_expansion_limit        | 20                           |
| ft_stopword_file                | (built-in)                   |
| group_concat_max_len            | 1024                         |
| have_archive                    | NO                           |
| have_bdb                        | NO                           |
| have_compress                   | YES                          |
| have_crypt                      | YES                          |
| have_csv                        | NO                           |
| have_example_engine             | NO                           |
| have_geometry                   | YES                          |
| have_innodb                     | YES                          |
| have_isam                       | NO                           |
| have_ndbcluster                 | NO                           |
| have_openssl                    | NO                           |
| have_query_cache                | YES                          |
| have_raid                       | NO                           |
| have_rtree_keys                 | YES                          |
| have_symlink                    | YES                          |
| init_connect                    |                              |
| init_file                       |                              |
| init_slave                      |                              |
| innodb_additional_mem_pool_size | 1048576                      |
| innodb_autoextend_increment     | 8                            |
| innodb_buffer_pool_awe_mem_mb   | 0                            |
| innodb_buffer_pool_size         | 8388608                      |
| innodb_data_file_path           | ibdata1:10M:autoextend       |
| innodb_data_home_dir            |                              |
| innodb_fast_shutdown            | ON                           |
| innodb_file_io_threads          | 4                            |
| innodb_file_per_table           | OFF                          |
| innodb_flush_log_at_trx_commit  | 1                            |
| innodb_flush_method             |                              |
| innodb_force_recovery           | 0                            |
| innodb_lock_wait_timeout        | 50                           |
| innodb_locks_unsafe_for_binlog  | OFF                          |
| innodb_log_arch_dir             |                              |
| innodb_log_archive              | OFF                          |
| innodb_log_buffer_size          | 1048576                      |
| innodb_log_file_size            | 5242880                      |
| innodb_log_files_in_group       | 2                            |
| innodb_log_group_home_dir       | ./                           |
| innodb_max_dirty_pages_pct      | 90                           |
| innodb_max_purge_lag            | 0                            |
| innodb_mirrored_log_groups      | 1                            |
| innodb_open_files               | 300                          |
| innodb_table_locks              | ON                           |
| innodb_thread_concurrency       | 8                            |
| interactive_timeout             | 28800                        |
| join_buffer_size                | 131072                       |
| key_buffer_size                 | 8388600                      |
| key_cache_age_threshold         | 300                          |
| key_cache_block_size            | 1024                         |
| key_cache_division_limit        | 100                          |
| language                        | /usr/share/mysql/english/    |
| large_files_support             | ON                           |
| license                         | GPL                          |
| local_infile                    | ON                           |
| locked_in_memory                | OFF                          |
| log                             | OFF                          |
| log_bin                         | ON                           |
| log_error                       |                              |
| log_slave_updates               | OFF                          |
| log_slow_queries                | OFF                          |
| log_update                      | OFF                          |
| log_warnings                    | 1                            |
| long_query_time                 | 10                           |
| low_priority_updates            | OFF                          |
| lower_case_file_system          | OFF                          |
| lower_case_table_names          | 0                            |
| max_allowed_packet              | 1048576                      |
| max_binlog_cache_size           | 4294967295                   |
| max_binlog_size                 | 1073741824                   |
| max_connect_errors              | 10                           |
| max_connections                 | 100                          |
| max_delayed_threads             | 20                           |
| max_error_count                 | 64                           |
| max_heap_table_size             | 16777216                     |
| max_insert_delayed_threads      | 20                           |
| max_join_size                   | 4294967295                   |
| max_length_for_sort_data        | 1024                         |
| max_relay_log_size              | 0                            |
| max_seeks_for_key               | 4294967295                   |
| max_sort_length                 | 1024                         |
| max_tmp_tables                  | 32                           |
| max_user_connections            | 0                            |
| max_write_lock_count            | 4294967295                   |
| myisam_data_pointer_size        | 4                            |
| myisam_max_extra_sort_file_size | 2147483648                   |
| myisam_max_sort_file_size       | 2147483647                   |
| myisam_recover_options          | OFF                          |
| myisam_repair_threads           | 1                            |
| myisam_sort_buffer_size         | 8388608                      |
| net_buffer_length               | 16384                        |
| net_read_timeout                | 30                           |
| net_retry_count                 | 10                           |
| net_write_timeout               | 60                           |
| new                             | OFF                          |
| old_passwords                   | ON                           |
| open_files_limit                | 1024                         |
| pid_file                        | /abel/var/lib/mysql/abel.pid |
| port                            | 3306                         |
| preload_buffer_size             | 32768                        |
| protocol_version                | 10                           |
| query_alloc_block_size          | 8192                         |
| query_cache_limit               | 1048576                      |
| query_cache_min_res_unit        | 4096                         |
| query_cache_size                | 0                            |
| query_cache_type                | ON                           |
| query_cache_wlock_invalidate    | OFF                          |
| query_prealloc_size             | 8192                         |
| range_alloc_block_size          | 2048                         |
| read_buffer_size                | 131072                       |
| read_only                       | OFF                          |
| read_rnd_buffer_size            | 262144                       |
| relay_log_purge                 | ON                           |
| rpl_recovery_rank               | 0                            |
| secure_auth                     | OFF                          |
| server_id                       | 1                            |
| skip_external_locking           | ON                           |
| skip_networking                 | OFF                          |
| skip_show_database              | OFF                          |
| slave_net_timeout               | 3600                         |
| slow_launch_time                | 2                            |
| socket                          | /var/lib/mysql/mysql.sock    |
| sort_buffer_size                | 2097144                      |
| sql_mode                        |                              |
| storage_engine                  | MyISAM                       |
| sync_binlog                     | 0                            |
| sync_frm                        | ON                           |
| system_time_zone                | UTC                          |
| table_cache                     | 64                           |
| table_type                      | MyISAM                       |
| thread_cache_size               | 0                            |
| thread_stack                    | 126976                       |
| time_format                     | %H:%i:%s                     |
| time_zone                       | SYSTEM                       |
| tmp_table_size                  | 33554432                     |
| tmpdir                          | /var/tmp                     |
| transaction_alloc_block_size    | 8192                         |
| transaction_prealloc_size       | 4096                         |
| tx_isolation                    | REPEATABLE-READ              |
| version                         | 4.1.9-standard-log           |
| version_comment                 | Official MySQL RPM           |
| version_compile_machine         | i686                         |
| version_compile_os              | pc-linux-gnu                 |
| wait_timeout                    | 28800                        |
+---------------------------------+------------------------------+
mysql> show status;
+--------------------------+------------+
| Variable_name            | Value      |
+--------------------------+------------+
| Aborted_clients          | 266        |
| Aborted_connects         | 58         |
| Binlog_cache_disk_use    | 0          |
| Binlog_cache_use         | 0          |
| Bytes_received           | 1207086044 |
| Bytes_sent               | 925597265  |
| Com_admin_commands       | 9          |
| Com_alter_db             | 0          |
| Com_alter_table          | 8          |
| Com_analyze              | 0          |
| Com_backup_table         | 0          |
| Com_begin                | 0          |
| Com_change_db            | 255922     |
| Com_change_master        | 0          |
| Com_check                | 0          |
| Com_checksum             | 0          |
| Com_commit               | 0          |
| Com_create_db            | 0          |
| Com_create_function      | 0          |
| Com_create_index         | 0          |
| Com_create_table         | 9          |
| Com_dealloc_sql          | 0          |
| Com_delete               | 615        |
| Com_delete_multi         | 375        |
| Com_do                   | 0          |
| Com_drop_db              | 0          |
| Com_drop_function        | 0          |
| Com_drop_index           | 0          |
| Com_drop_table           | 0          |
| Com_drop_user            | 0          |
| Com_execute_sql          | 0          |
| Com_flush                | 1          |
| Com_grant                | 2          |
| Com_ha_close             | 0          |
| Com_ha_open              | 0          |
| Com_ha_read              | 0          |
| Com_help                 | 0          |
| Com_insert               | 2802754    |
| Com_insert_select        | 179        |
| Com_kill                 | 0          |
| Com_load                 | 3          |
| Com_load_master_data     | 0          |
| Com_load_master_table    | 0          |
| Com_lock_tables          | 1          |
| Com_optimize             | 0          |
| Com_preload_keys         | 0          |
| Com_prepare_sql          | 0          |
| Com_purge                | 0          |
| Com_purge_before_date    | 0          |
| Com_rename_table         | 0          |
| Com_repair               | 0          |
| Com_replace              | 21         |
| Com_replace_select       | 0          |
| Com_reset                | 0          |
| Com_restore_table        | 0          |
| Com_revoke               | 0          |
| Com_revoke_all           | 0          |
| Com_rollback             | 0          |
| Com_savepoint            | 0          |
| Com_select               | 1143950    |
| Com_set_option           | 423039     |
| Com_show_binlog_events   | 0          |
| Com_show_binlogs         | 27         |
| Com_show_charsets        | 407        |
| Com_show_collations      | 407        |
| Com_show_column_types    | 0          |
| Com_show_create_db       | 28         |
| Com_show_create_table    | 3          |
| Com_show_databases       | 210        |
| Com_show_errors          | 0          |
| Com_show_fields          | 1080       |
| Com_show_grants          | 20         |
| Com_show_innodb_status   | 0          |
| Com_show_keys            | 130        |
| Com_show_logs            | 0          |
| Com_show_master_status   | 0          |
| Com_show_new_master      | 0          |
| Com_show_open_tables     | 0          |
| Com_show_privileges      | 0          |
| Com_show_processlist     | 0          |
| Com_show_slave_hosts     | 9          |
| Com_show_slave_status    | 0          |
| Com_show_status          | 2          |
| Com_show_storage_engines | 0          |
| Com_show_tables          | 2321       |
| Com_show_variables       | 1142       |
| Com_show_warnings        | 0          |
| Com_slave_start          | 0          |
| Com_slave_stop           | 0          |
| Com_truncate             | 0          |
| Com_unlock_tables        | 1          |
| Com_update               | 1785417    |
| Com_update_multi         | 957888     |
| Connections              | 678044     |
| Created_tmp_disk_tables  | 10644      |
| Created_tmp_files        | 30         |
| Created_tmp_tables       | 1003983    |
| Delayed_errors           | 0          |
| Delayed_insert_threads   | 0          |
| Delayed_writes           | 0          |
| Flush_commands           | 1          |
| Handler_commit           | 0          |
| Handler_delete           | 239        |
| Handler_discover         | 0          |
| Handler_read_first       | 52247      |
| Handler_read_key         | 59612310   |
| Handler_read_next        | 1545296428 |
| Handler_read_prev        | 0          |
| Handler_read_rnd         | 4662981    |
| Handler_read_rnd_next    | 1019368875 |
| Handler_rollback         | 0          |
| Handler_update           | 51530976   |
| Handler_write            | 6289574    |
| Key_blocks_not_flushed   | 0          |
| Key_blocks_unused        | 0          |
| Key_blocks_used          | 7248       |
| Key_read_requests        | 583483799  |
| Key_reads                | 59046024   |
| Key_write_requests       | 10160088   |
| Key_writes               | 6577039    |
| Max_used_connections     | 67         |
| Not_flushed_delayed_rows | 0          |
| Open_files               | 105        |
| Open_streams             | 0          |
| Open_tables              | 64         |
| Opened_tables            | 2030       |
| Qcache_free_blocks       | 0          |
| Qcache_free_memory       | 0          |
| Qcache_hits              | 0          |
| Qcache_inserts           | 0          |
| Qcache_lowmem_prunes     | 0          |
| Qcache_not_cached        | 0          |
| Qcache_queries_in_cache  | 0          |
| Qcache_total_blocks      | 0          |
| Questions                | 8053997    |
| Rpl_status               | NULL       |
| Select_full_join         | 85773      |
| Select_full_range_join   | 0          |
| Select_range             | 35122      |
| Select_range_check       | 0          |
| Select_scan              | 1717408    |
| Slave_open_temp_tables   | 0          |
| Slave_running            | OFF        |
| Slow_launch_threads      | 72         |
| Slow_queries             | 1245       |
| Sort_merge_passes        | 13         |
| Sort_range               | 6          |
| Sort_rows                | 3556625    |
| Sort_scan                | 68452      |
| Table_locks_immediate    | 10019758   |
| Table_locks_waited       | 125222     |
| Threads_cached           | 0          |
| Threads_connected        | 3          |
| Threads_created          | 678043     |
| Threads_running          | 3          |
| Uptime                   | 1375051    |
+--------------------------+------------+
[13 May 2005 0:03] Adrian Corston
/etc/my.cnf configuration file

Attachment: my.cnf (application/octet-stream, text), 107 bytes.

[13 May 2005 1:48] Adrian Corston
When the parameter is converted to utf8 character set, the bug does not manifest, so it's definitely a charset issue:

mysql> select concat('!', x) from TEST;
+----------------+
| concat('!', x) |
+----------------+
| !one           |
| !onetwo        |
| !onetwothree   |
+----------------+
3 rows in set (0.00 sec)

mysql> select concat(_utf8 '!', x) from TEST;
+----------------------+
| concat(_utf8 '!', x) |
+----------------------+
| !one                 |
| !two                 |
| !three               |
+----------------------+
3 rows in set (0.00 sec)
[13 Jun 2005 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[22 Jun 2005 19:42] MySQL Verification Team
I was unable to repeat the behavior reported with latest BK source
server and using your my.cnf file.
[23 Jun 2005 1:00] Adrian Corston
Please close the bug then.  If anyone experiences the bug they can use the _utf8 fix I wrote about earlier.  Thankyou!