Bug #32178 server crash when select from i_s and concurrent partition management
Submitted: 8 Nov 2007 6:47 Modified: 14 Dec 2007 15:46
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Partitions Severity:S1 (Critical)
Version:5.1.23 OS:Any
Assigned to: Sergei Glukhov CPU Architecture:Any
Tags: bfsm_2007_11_15
Triage: D1 (Critical)

[8 Nov 2007 6:47] Shane Bester
Description:
This is a followup from Bug #28477

When running that testcase, no crashes occur.  But, if you select from information_schema.partitions table while the testcase runs, a crash occurs with the following stack trace:

mysqld.exe!ha_partition::get_dynamic_partition_info
mysqld.exe!store_schema_partitions_record
mysqld.exe!get_schema_partitions_record
mysqld.exe!get_all_tables
mysqld.exe!get_schema_tables_result
mysqld.exe!JOIN::exec
mysqld.exe!mysql_select
mysqld.exe!handle_select
mysqld.exe!execute_sqlcom_select
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_threadstart

In the error log, a message like this may appear:

071108  8:25:18  InnoDB: Started; log sequence number 0 7692698
071108  8:25:18  InnoDB: Error: table `test`.071108  8:25:18 [ERROR] Invalid (old?) table or database name 't1_test#P#p08'
`#mysql50#t1_test#P#p08` does not exist in the InnoDB internal
InnoDB: data dictionary though MySQL is trying to drop it.

How to repeat:
compile and run the attached testcase against a clean 5.1 server.
[8 Nov 2007 6:49] Shane Bester
see top of file for user/host/port and compile lines.

Attachment: bug32178.c (text/plain), 6.54 KiB.

[14 Nov 2007 12:24] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/37742

ChangeSet@1.2610, 2007-11-14 16:19:38+04:00, gluh@mysql.com +2 -0
  Bug#32178 server crash when select from i_s and concurrent partition management
  The crash happens because of the following problem:
  prep_alter_part_table() func uses table name lock
  but I_S table opening does not use it.
  In I_S during table processing we can get discrepancy
  between share->partition_info and total number of partitions
  which obtained from par file.
  
  The fix:
  check if partition info is synchronized
  with the information in par file.
  Total number of partitons in part_info should be
  equal to(or less than) ha_partition::m_tot_parts value.
  If information is inconsistent then write a message to
  PARTITION_COMMENT field.
[15 Nov 2007 12:06] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/37843

ChangeSet@1.2610, 2007-11-15 16:00:57+04:00, gluh@mysql.com +2 -0
  Bug#32178 server crash when select from i_s and concurrent partition management(2nd version)
  The crash happens because of the following problem:
  prep_alter_part_table() func uses table name lock
  but I_S table opening does not use it.
  In I_S during table processing we can get discrepancy
  between share->partition_info and total number of partitions
  which is obtained from par file.
  
  The fix:
    check if partition info is synchronized
    with the information in par file.
    Total number of partitons in part_info should be
    less than ha_partition::m_tot_parts value. 
    Otherwise we skip file->get_dynamic_partition_info() call
    (see store_schema_partitions_record() func).
[21 Nov 2007 17:18] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/38222

ChangeSet@1.2610, 2007-11-21 21:12:35+04:00, gluh@mysql.com +3 -0
  Bug#32178 server crash when select from i_s and concurrent partition management
  The crash happens because we change share->partition_info where 'share' is global struc
  (it affects other threads which use the same 'share').
  It causes discrepancy between 'share' and handler data. 
  The fix:
  Move share->partition_info update into WFRM_INSTALL_SHADOW part which is protected by OPEN_lock.
[23 Nov 2007 12:32] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/38359

ChangeSet@1.2642, 2007-11-23 16:27:05+04:00, gluh@mysql.com +2 -0
  Bug#32178 server crash when select from i_s and concurrent partition management
  The crash happens because we change share->partition_info where 'share' is global struct
  (it affects other threads which use the same 'share').
  It causes discrepancy between 'share' and handler data. 
  The fix:
  Move share->partition_info update into WFRM_INSTALL_SHADOW part which is protected by OPEN_lock.
[14 Dec 2007 8:18] Bugs System
Pushed into 5.1.23-rc
[14 Dec 2007 8:21] Bugs System
Pushed into 6.0.5-alpha
[14 Dec 2007 15:46] Jon Stephens
Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Documented in 5.1.23 and 6.0.5 changelogs as:

      
        Selecting from INFORMATION_SCHEMA.PARTITIONS
        while partition management statements (for example,
        ALTER TABLE ... ADD PARTITION) were executing
        caused the server to crash.
[27 Oct 2009 8:14] Wim Symons
I have the same issue on MySQL 5.1.40 AMD64 RPM packages running on CentOS 5.2.

My stacktrace:

091027  0:01:00 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=81
max_threads=321
threads_connected=67
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 718140 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x2aaadc253db0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x44d2bf20 thread_stack 0x20000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x8490ee]
/usr/sbin/mysqld(handle_segfault+0x322)[0x5bf6a2]
/lib64/libpthread.so.0[0x3108c0de70]
/usr/sbin/mysqld(_ZN12ha_partition26get_dynamic_partition_infoEP14PARTITION_INFOj+0x1b)[0x6a173b]
/usr/sbin/mysqld[0x6b92dd]
/usr/sbin/mysqld[0x6c2d45]
/usr/sbin/mysqld(_Z14get_all_tablesP3THDP10TABLE_LISTP4Item+0x871)[0x6c10d1]
/usr/sbin/mysqld(_Z24get_schema_tables_resultP4JOIN23enum_schema_table_state+0x1e0)[0x6b9b30]
/usr/sbin/mysqld(_ZN4JOIN4execEv+0x43e)[0x63566e]
/usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x16e)[0x63757e]
/usr/sbin/mysqld(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x169)[0x637e99]
/usr/sbin/mysqld[0x5c9ed4]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x4c1)[0x5cc991]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x1f1)[0x5d1e71]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xff0)[0x5d2e80]
/usr/sbin/mysqld(_Z10do_commandP3THD+0xe4)[0x5d3414]
/usr/sbin/mysqld(handle_one_connection+0x6f0)[0x5c6650]
/lib64/libpthread.so.0[0x3108c062f7]
/lib64/libc.so.6(clone+0x6d)[0x31080d1e3d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x15650e20 = select partition_name,partition_ordinal_position from information_schema.partitions where concat(table_schema,'.',table_name)='tmsng_norestore.ZoneFlowDataPer10s' and partition_description!='MAXVALUE' order by partition_ordinal_position asc
thd->thread_id=43067
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
091027 00:01:00 mysqld_safe Number of processes running now: 0
091027 00:01:00 mysqld_safe mysqld restarted