Bug #9594 | Fulltext boolean searches with wildcard characters at the end does not work | ||
---|---|---|---|
Submitted: | 3 Apr 2005 18:53 | Modified: | 16 Sep 2005 12:44 |
Reporter: | Joyce Gross | Email Updates: | |
Status: | Not a Bug | Impact on me: | |
Category: | MySQL Server | Severity: | S2 (Serious) |
Version: | 4.1.10a | OS: | Solaris (Solaris 2.9) |
Assigned to: | CPU Architecture: | Any |
[3 Apr 2005 18:53]
Joyce Gross
[3 Apr 2005 20:21]
Joyce Gross
I should also mention that creating new fulltext indexes in mysql 4.1.10a takes several times longer than it did in 4.0.5. Is this normal? For example, the same data that took 5 hours to index in 4.0.5 now takes up to 50 hours. I've done two loads (with indexing) of this same data in mysql 4.1.10a--the first one took 16 hours, the second one is still going after 35 hours; I'm guessing it will be done after about 50 hours. In mysql 4.0.5 it took 5 hours.
[5 Apr 2005 17:32]
Sergei Golubchik
phrase search ("...") never supported truncation (*) operation. Sometimes it looked like it did, but even a slight change in the query would result in wrong results. It was a bug that was fixed. As for "slow". This looks like a bug. It should not be slower. Could you show the output of SHOW STATUS SHOW VARIABLES then, for the table that is slow to build fulltextindex on: SHOW TABLE STATUS LIKE 'tablename' SHOW CREATE TABLE tablename and, while this slow CREATE FULLTEXT INDEX is working (issue the query from another connection): SHOW PROCESSLIST
[11 Apr 2005 22:43]
Joyce Gross
Here is the output of the various commands, as requested. Note that I have been playing with the my.cnf file as it seems to have an effect on how fast indexing happens. The configuration below is similar to that in the my-medium.cnf file that came with the server, with some changes. When I used settings more similar to those in the my-large.cnf file, indexing took much longer. The solaris box I use is somewhat old... it has four 400MHz processors and 1.25 GB RAM. But with mysql 4.0.5 it used to do the indexing on the same data much more quickly than it does now. mysql> show status; +--------------------------+------------+ | Variable_name | Value | +--------------------------+------------+ | Aborted_clients | 4010 | | Aborted_connects | 1874 | | Binlog_cache_disk_use | 0 | | Binlog_cache_use | 0 | | Bytes_received | 55014009 | | Bytes_sent | 2303798303 | | Com_admin_commands | 913 | | Com_alter_db | 0 | | Com_alter_table | 12 | | Com_analyze | 0 | | Com_backup_table | 0 | | Com_begin | 0 | | Com_change_db | 5056 | | Com_change_master | 0 | | Com_check | 0 | | Com_checksum | 0 | | Com_commit | 0 | | Com_create_db | 0 | | Com_create_function | 0 | | Com_create_index | 40 | | Com_create_table | 35 | | Com_dealloc_sql | 0 | | Com_delete | 101 | | Com_delete_multi | 0 | | Com_do | 0 | | Com_drop_db | 0 | | Com_drop_function | 0 | | Com_drop_index | 2 | | Com_drop_table | 35 | | Com_drop_user | 0 | | Com_execute_sql | 0 | | Com_flush | 0 | | Com_grant | 0 | | Com_ha_close | 0 | | Com_ha_open | 0 | | Com_ha_read | 0 | | Com_help | 0 | | Com_insert | 21449 | | Com_insert_select | 2 | | Com_kill | 0 | | Com_load | 136 | | Com_load_master_data | 0 | | Com_load_master_table | 0 | | Com_lock_tables | 0 | | Com_optimize | 0 | | Com_preload_keys | 0 | | Com_prepare_sql | 0 | | Com_purge | 0 | | Com_purge_before_date | 0 | | Com_rename_table | 1 | | Com_repair | 0 | | Com_replace | 0 | | Com_replace_select | 0 | | Com_reset | 0 | | Com_restore_table | 0 | | Com_revoke | 0 | | Com_revoke_all | 0 | | Com_rollback | 0 | | Com_savepoint | 0 | | Com_select | 266314 | | Com_set_option | 2254 | | Com_show_binlog_events | 0 | | Com_show_binlogs | 0 | | Com_show_charsets | 0 | | Com_show_collations | 744 | | Com_show_column_types | 0 | | Com_show_create_db | 0 | | Com_show_create_table | 0 | | Com_show_databases | 0 | | Com_show_errors | 0 | | Com_show_fields | 46 | | Com_show_grants | 0 | | Com_show_innodb_status | 0 | | Com_show_keys | 0 | | Com_show_logs | 0 | | Com_show_master_status | 0 | | Com_show_new_master | 0 | | Com_show_open_tables | 0 | | Com_show_privileges | 0 | | Com_show_processlist | 20 | | Com_show_slave_hosts | 0 | | Com_show_slave_status | 0 | | Com_show_status | 1 | | Com_show_storage_engines | 0 | | Com_show_tables | 2 | | Com_show_variables | 744 | | Com_show_warnings | 0 | | Com_slave_start | 0 | | Com_slave_stop | 0 | | Com_truncate | 10 | | Com_unlock_tables | 0 | | Com_update | 21890 | | Com_update_multi | 39 | | Connections | 284737 | | Created_tmp_disk_tables | 57 | | Created_tmp_files | 563 | | Created_tmp_tables | 2196 | | Delayed_errors | 0 | | Delayed_insert_threads | 0 | | Delayed_writes | 0 | | Flush_commands | 1 | | Handler_commit | 0 | | Handler_delete | 20198 | | Handler_discover | 0 | | Handler_read_first | 1314 | | Handler_read_key | 6261528 | | Handler_read_next | 175172616 | | Handler_read_prev | 0 | | Handler_read_rnd | 2602586 | | Handler_read_rnd_next | 61111504 | | Handler_rollback | 0 | | Handler_update | 7835019 | | Handler_write | 17083291 | | Key_blocks_not_flushed | 0 | | Key_blocks_unused | 0 | | Key_blocks_used | 14347 | | Key_read_requests | 85505722 | | Key_reads | 223043 | | Key_write_requests | 18032917 | | Key_writes | 413992 | | Max_used_connections | 44 | | Not_flushed_delayed_rows | 0 | | Open_files | 124 | | Open_streams | 0 | | Open_tables | 64 | | Opened_tables | 545 | | Qcache_free_blocks | 0 | | Qcache_free_memory | 0 | | Qcache_hits | 0 | | Qcache_inserts | 0 | | Qcache_lowmem_prunes | 0 | | Qcache_not_cached | 0 | | Qcache_queries_in_cache | 0 | | Qcache_total_blocks | 0 | | Questions | 599795 | | Rpl_status | NULL | | Select_full_join | 31 | | Select_full_range_join | 0 | | Select_range | 56439 | | Select_range_check | 0 | | Select_scan | 4549 | | Slave_open_temp_tables | 0 | | Slave_running | OFF | | Slow_launch_threads | 0 | | Slow_queries | 246 | | Sort_merge_passes | 329 | | Sort_range | 22796 | | Sort_rows | 3219369 | | Sort_scan | 2839 | | Table_locks_immediate | 310738 | | Table_locks_waited | 30 | | Threads_cached | 0 | | Threads_connected | 3 | | Threads_created | 284736 | | Threads_running | 3 | | Uptime | 230443 | +--------------------------+------------+ 156 rows in set (0.01 sec) mysql> SHOW VARIABLES; +---------------------------------+-----------------------------------------------+ | Variable_name | Value | +---------------------------------+-----------------------------------------------+ | back_log | 50 | | basedir | /usr/local/mysql/ | | binlog_cache_size | 32768 | | bulk_insert_buffer_size | 8388608 | | character_set_client | latin1 | | character_set_connection | latin1 | | character_set_database | latin1 | | character_set_results | latin1 | | character_set_server | latin1 | | character_set_system | utf8 | | character_sets_dir | /usr/local/mysql/share/mysql/charsets/ | | collation_connection | latin1_swedish_ci | | collation_database | latin1_swedish_ci | | collation_server | latin1_swedish_ci | | concurrent_insert | ON | | connect_timeout | 5 | | datadir | /usr/local/mysql/data/ | | date_format | %Y-%m-%d | | datetime_format | %Y-%m-%d %H:%i:%s | | default_week_format | 0 | | delay_key_write | ON | | delayed_insert_limit | 100 | | delayed_insert_timeout | 300 | | delayed_queue_size | 1000 | | expire_logs_days | 0 | | flush | OFF | | flush_time | 0 | | ft_boolean_syntax | + -><()~*:""&| | | ft_max_word_len | 84 | | ft_min_word_len | 3 | | ft_query_expansion_limit | 20 | | ft_stopword_file | (built-in) | | group_concat_max_len | 1024 | | have_archive | NO | | have_bdb | NO | | have_compress | YES | | have_crypt | YES | | have_csv | NO | | have_example_engine | NO | | have_geometry | YES | | have_innodb | YES | | have_isam | NO | | have_ndbcluster | NO | | have_openssl | NO | | have_query_cache | YES | | have_raid | NO | | have_rtree_keys | YES | | have_symlink | YES | | init_connect | | | init_file | | | init_slave | | | innodb_additional_mem_pool_size | 1048576 | | innodb_autoextend_increment | 8 | | innodb_buffer_pool_awe_mem_mb | 0 | | innodb_buffer_pool_size | 8388608 | | innodb_data_file_path | ibdata1:10M:autoextend | | innodb_data_home_dir | | | innodb_fast_shutdown | ON | | innodb_file_io_threads | 4 | | innodb_file_per_table | OFF | | innodb_flush_log_at_trx_commit | 1 | | innodb_flush_method | | | innodb_force_recovery | 0 | | innodb_lock_wait_timeout | 50 | | innodb_locks_unsafe_for_binlog | OFF | | innodb_log_arch_dir | | | innodb_log_archive | OFF | | innodb_log_buffer_size | 1048576 | | innodb_log_file_size | 5242880 | | innodb_log_files_in_group | 2 | | innodb_log_group_home_dir | ./ | | innodb_max_dirty_pages_pct | 90 | | innodb_max_purge_lag | 0 | | innodb_mirrored_log_groups | 1 | | innodb_open_files | 300 | | innodb_table_locks | ON | | innodb_thread_concurrency | 8 | | interactive_timeout | 28800 | | join_buffer_size | 131072 | | key_buffer_size | 16777216 | | key_cache_age_threshold | 300 | | key_cache_block_size | 1024 | | key_cache_division_limit | 100 | | language | /usr/local/mysql/share/mysql/english/ | | large_files_support | ON | | license | GPL | | local_infile | ON | | locked_in_memory | OFF | | log | OFF | | log_bin | ON | | log_error | | | log_slave_updates | OFF | | log_slow_queries | ON | | log_update | OFF | | log_warnings | 1 | | long_query_time | 10 | | low_priority_updates | OFF | | lower_case_file_system | OFF | | lower_case_table_names | 0 | | max_allowed_packet | 10484736 | | max_binlog_cache_size | 4294967295 | | max_binlog_size | 1073741824 | | max_connect_errors | 10 | | max_connections | 500 | | max_delayed_threads | 20 | | max_error_count | 64 | | max_heap_table_size | 16777216 | | max_insert_delayed_threads | 20 | | max_join_size | 4294967295 | | max_length_for_sort_data | 1024 | | max_relay_log_size | 0 | | max_seeks_for_key | 4294967295 | | max_sort_length | 1024 | | max_tmp_tables | 32 | | max_user_connections | 0 | | max_write_lock_count | 4294967295 | | myisam_data_pointer_size | 4 | | myisam_max_extra_sort_file_size | 2147483648 | | myisam_max_sort_file_size | 2147483647 | | myisam_recover_options | OFF | | myisam_repair_threads | 1 | | myisam_sort_buffer_size | 8388608 | | net_buffer_length | 8192 | | net_read_timeout | 30 | | net_retry_count | 10 | | net_write_timeout | 60 | | new | OFF | | old_passwords | ON | | open_files_limit | 2510 | | pid_file | /usr/local/mysql/data/dlp.CS.Berkeley.EDU.pid | | port | 3306 | | preload_buffer_size | 32768 | | protocol_version | 10 | | query_alloc_block_size | 8192 | | query_cache_limit | 1048576 | | query_cache_min_res_unit | 4096 | | query_cache_size | 0 | | query_cache_type | ON | | query_cache_wlock_invalidate | OFF | | query_prealloc_size | 8192 | | range_alloc_block_size | 2048 | | read_buffer_size | 258048 | | read_only | OFF | | read_rnd_buffer_size | 520192 | | relay_log_purge | ON | | rpl_recovery_rank | 0 | | secure_auth | OFF | | server_id | 1 | | skip_external_locking | ON | | skip_networking | OFF | | skip_show_database | OFF | | slave_net_timeout | 3600 | | slow_launch_time | 2 | | socket | /tmp/mysql.sock | | sort_buffer_size | 524252 | | sql_mode | | | storage_engine | MyISAM | | sync_binlog | 0 | | sync_replication | 0 | | sync_replication_slave_id | 0 | | sync_replication_timeout | 0 | | sync_frm | ON | | system_time_zone | PDT | | table_cache | 64 | | table_type | MyISAM | | thread_cache_size | 0 | | thread_concurrency | 10 | | thread_stack | 196608 | | time_format | %H:%i:%s | | time_zone | SYSTEM | | tmp_table_size | 33554432 | | tmpdir | | | transaction_alloc_block_size | 8192 | | transaction_prealloc_size | 4096 | | tx_isolation | REPEATABLE-READ | | version | 4.1.10a-debug-log | | version_comment | Source distribution | | version_compile_machine | sparc | | version_compile_os | sun-solaris2.9 | | wait_timeout | 28800 | +---------------------------------+-----------------------------------------------+ 180 rows in set (0.01 sec) mysql> SHOW TABLE STATUS LIKE 'mvz_dump'; +----------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+ | Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment | +----------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+ | mvz_dump | MyISAM | 9 | Dynamic | 654313 | 507 | 332144844 | 4294967295 | 126808064 | 0 | 11211643 | 2005-04-09 17:01:11 | 2005-04-09 17:08:50 | 2005-04-09 19:02:00 | latin1_swedish_ci | NULL | | | +----------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+ 1 row in set (0.01 sec) Table Create Table mvz_dump CREATE TABLE `mvz_dump` (\n `Coll_Object_id` int(10) unsigned NOT NULL auto_increment,\n `Cat_Num` int(11) default NULL, \n `Accn_Num` int(11) default NULL,\n `Accn_Num_Suffix` char(1) default NULL,\n `Collection_Cde` varchar(15) default NULL,\n `class` v archar(20) default NULL,\n `ordr` varchar(30) default NULL,\n `family` varchar(30) default NULL,\n `term` varchar(110) default NULL,\n `term_id` int(11) default NULL,\n `subfamily` varchar(30) default NULL,\n `family_id` int(11) default NULL,\n `Id_Modifier` varchar(15 ) default NULL,\n `Id_Remarks` varchar(255) default NULL,\n `Type_Status` varchar(20) default NULL,\n `Coll_Name` varchar(100) default NULL,\n `Sex_Cde` varchar(25) default NULL,\n `Sex_Cde_Mod` varchar(20) default NULL,\n `Age_Class` varchar(21) default NULL,\n `Weigh t` decimal(15,10) default NULL,\n `Weight_Units` varchar(15) default NULL,\n `Repro_Data` varchar(255) default NULL,\n `Age_Remarks` va rchar(255) default NULL,\n `Stomach_Cont` varchar(255) default NULL,\n `Verbatim_Date` varchar(30) default NULL,\n `Taxir_Remarks` varc har(255) default NULL,\n `Collectors` varchar(255) default NULL,\n `Parts` varchar(255) default NULL,\n `Parts_Dispositions` text,\n ` Tissues` varchar(255) default NULL,\n `Tissue_Remarks` text,\n `Tiss_Dispositions` text,\n `Spec_Dispositions` varchar(255) default NUL L,\n `Other_IDs` text,\n `Verbatim_Pres_Date` varchar(30) default NULL,\n `Fat_Deposition` varchar(35) default NULL,\n `Colors` varcha r(255) default NULL,\n `Molt_Condition` varchar(255) default NULL,\n `Incubation_Stage` varchar(255) default NULL,\n `Egg_Nest_Combo` v archar(20) default NULL,\n `Scientific_Name` varchar(110) default NULL,\n `Citation_Text` varchar(255) default NULL,\n `Year` varchar(1 0) default NULL,\n `year_collected` int(11) default NULL,\n `Continent_Ocean` varchar(50) default NULL,\n `Country` varchar(50) default NULL,\n `State_Prov` varchar(75) default NULL,\n `County` varchar(50) default NULL,\n `Feature` varchar(50) default NULL,\n `Island` varchar(50) default NULL,\n `Island_Group` varchar(50) default NULL,\n `Spec_Locality` varchar(255) default NULL,\n `Minimum_Elevation` decimal(16,10) default NULL,\n `Maximum_Elevation` decimal(16,10) default NULL,\n `Orig_Elev_Units` char(2) default NULL,\n `Dec_Lat` decimal(15,10) default NULL,\n `Dec_Long` decimal(15,10) default NULL,\n `Township` int(11) default NULL,\n `Township_Direction` char(1 ) default NULL,\n `Range` int(11) default NULL,\n `Range_Direction` char(1) default NULL,\n `Section` int(11) default NULL,\n `Section _Part` varchar(11) default NULL,\n `Continent_num` int(11) default NULL,\n `Country_num` int(11) default NULL,\n `State_Prov_num` int(1 1) default NULL,\n `County_num` int(11) default NULL,\n `Feature_num` int(11) default NULL,\n `Island_num` int(11) default NULL,\n `Is land_Group_num` int(11) default NULL,\n `Coll_id` int(11) default NULL,\n `higher_geog` varchar(255) default NULL,\n `max_error_distanc e` decimal(8,3) default NULL,\n `max_error_units` char(2) default NULL,\n `datum` varchar(40) default NULL,\n PRIMARY KEY (`Coll_Objec t_id`),\n KEY `BBiddxd_01_ix` (`Cat_Num`),\n KEY `BBiddxd_02_ix` (`Accn_Num`),\n KEY `BBiddxd_03_ix` (`Collection_Cde`),\n KEY `BBiddx d_04_ix` (`class`),\n KEY `BBiddxd_05_ix` (`ordr`),\n KEY `BBiddxd_10_ix` (`Type_Status`),\n KEY `BBiddxd_11_ix` (`Continent_Ocean`),\n KEY `BBiddxd_12_ix` (`Country`),\n KEY `BBiddxd_17_ix` (`Island_Group`),\n KEY `BBiddxd_19_ix` (`Sex_Cde`),\n KEY `BBiddxd_21_ix` (`y ear_collected`),\n FULLTEXT KEY `BBiddxd_06_ix` (`family`),\n FULLTEXT KEY `BBiddxd_07_ix` (`term`),\n FULLTEXT KEY `BBiddxd_13_ix` (`S tate_Prov`),\n FULLTEXT KEY `BBiddxd_14_ix` (`County`),\n FULLTEXT KEY `BBiddxd_16_ix` (`Island`),\n FULLTEXT KEY `BBiddxd_18_ix` (`Col lectors`),\n FULLTEXT KEY `BBiddxd_20_ix` (`Scientific_Name`),\n FULLTEXT KEY `BBiddxd_22_ix` (`Spec_Locality`)\n) ENGINE=MyISAM DEFAULT CHARSET=latin1 mysql> show processlist; +-------+--------+-----------+-------+---------+------+-------------------+--------------------------------------------------------------- -------+ | Id | User | Host | db | Command | Time | State | Info | +-------+--------+-----------+-------+---------+------+-------------------+--------------------------------------------------------------- -------+ | 971 | joyceg | localhost | elib | Query | 1982 | Repair by sorting | CREATE FULLTEXT INDEX BBiddxd_20_ix on mvz_dumpB ( Scientific_ Name ) | | 73493 | joyceg | localhost | mysql | Query | 0 | NULL | show processlist | +-------+--------+-----------+-------+---------+------+-------------------+--------------------------------------------------------------- -------+ 2 rows in set (0.00 sec)
[11 May 2005 23:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[13 May 2005 5:40]
Joyce Gross
> If you are able to provide the information > that was originally requested, please do so... I provided the information on 12 Apr. Is something missing?
[13 May 2005 10:14]
Sergei Golubchik
no, you just forgot to change the status of the bug (from "need feedback" to "open") and it was auto-closed after a month of inactivity. I reopened it for you.
[16 Sep 2005 12:44]
Valeriy Kravchuk
Thank you for taking the time to write to us, but this is not a bug. Please double-check the documentation available at http://www.mysql.com/documentation/ and the instructions on how to report a bug at http://bugs.mysql.com/how-to-report.php Additional info: I changed synopsis to reflect the original problem reported. As described initially, it is not a bug, but a documented behaviour, as Sergei Golubchik explaind you. Please read http://dev.mysql.com/doc/mysql/en/fulltext-restrictions.html for details: "The argument to AGAINST() must be a constant string" You may add a feture request to add pattern-matching support for the fulltext search. As for slow indexing, please, try the newer version (4.1.14) and report about any unexpected behaviour you will see.