Bug #17253 free(): invalid pointer, mysqld got signal 11
Submitted: 9 Feb 2006 0:32 Modified: 16 Mar 2006 15:03
Reporter: sanjeev sagar Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:5.0.18, 5.0.15 OS:Linux (Linux 2.4.21-32)
Assigned to: CPU Architecture:Any

[9 Feb 2006 0:32] sanjeev sagar
Description:
Version: '5.0.15-standard-log'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Community Edition - Standard (GPL)
free(): invalid pointer 0xa39f890!
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=1073741824
read_buffer_size=104853504
max_used_connections=7
max_connections=100
threads_connected=6
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 3104367 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0xa2b8900
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x486b24, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x814e7f8
0x708e48
0x486eb0
0x8352bb9
0x81928be
0x8190294
0x82275e5
0x8227e5d
0x8227a9c
0x8182d55
0x8162160
0x8168d82
0x8160aa6
0x81605d1
0x815fab1
0x702de8
0x34e93a
0x702de8
0x34e93a
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xa2c67c0 = Select concat_ws(',',concat_ws(' ',x.nameFirst,x.nameLast),x.Client,x.cells,x.month) Results
from (
Select e.nameFirst, e.nameLast, extract(MONTH FROM t.minDate) month, b.name Client, count(c.id) cells
from Cell c, Plan p, Job j, Employee e, MMTask t, BusinessEntity b
WHERE p.campaignManagerEmployee_id = e.id
AND p.endClient = b.id
  AND p.id = j.plan_id
  AND j.id = c.job_id
  AND t.cell_id = c.id
  AND c.cellType = 3
  AND t.minDate > '2005-12-31'
  AND t.minDate < '2006-02-01'
  group by e.nameFirst, e.nameLast, b.name, extract(MONTH FROM t.minDate))x
thd->thread_id=19
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Even the explain on the query crashed the server.

0x814e7f8 handle_segfault + 356
0x708e48 (?)
0x486eb0 (?)
0x8352bb9 free_root + 137
0x81928be _Z14free_tmp_tableP3THDP8st_table + 146
0x8190294 _ZN4JOIN7destroyEv + 540
0x82275e5 _ZN18st_select_lex_unit7cleanupEv + 441
0x8227e5d _Z21mysql_derived_fillingP3THDP6st_lexP13st_table_list + 153
0x8227a9c _Z20mysql_handle_derivedP6st_lexPFiP3THDS0_P13st_table_listE + 84
0x8182d55 _Z20open_and_lock_tablesP3THDP13st_table_list + 277
0x8162160 _Z21mysql_execute_commandP3THD + 592
0x8168d82 _Z11mysql_parseP3THDPcj + 306
0x8160aa6 _Z16dispatch_command19enum_server_commandP3THDPcj + 1178
0x81605d1 _Z10do_commandP3THD + 129
0x815fab1 handle_one_connection + 569
0x702de8 (?)
0x34e93a (?)

So I removed the top select and run the inner select in the impression that derived table is creating the problem but server crashed for that inner select too. see below

060208 15:38:24 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.0.15-standard-log'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Community Edition - Standard (GPL)
free(): invalid pointer 0xa7df2a8!
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=1073741824
read_buffer_size=104853504
max_used_connections=1
max_connections=100
threads_connected=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 3104367 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0xa7b69f8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xfaeb24, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x814e7f8
0x22de48
0xfaeeb0
0x8352bb9
0x81928be
0x8190294
0x82277e8
0x81903a0
0x818c6ba
0x81621ad
0x8168d82
0x8160aa6
0x81605d1
0x815fab1
0x227de8
0x3f893a
0x227de8
0x3f893a
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xa7c3d30 = Select e.nameFirst, e.nameLast, extract(MONTH FROM t.minDate) month, b.name Client, count(c.id) cells         from Cell c, Plan p, Job j, Employee e, MMTask t, BusinessEntity b         WHERE p.campaignManagerEmployee_id = e.id AND p.endClient = b.id   AND p.id = j.plan_id         AND j.id = c.job_id   AND t.cell_id = c.id   AND c.cellType = 3   AND t.minDate > '2005-12-31'         AND t.minDate < '2006-02-01'         group by e.nameFirst, e.nameLast, b.name, extract(MONTH FROM t.minDate)
thd->thread_id=3
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

/tmp> resolve_stack_dump -s /tmp/mysqld.sym -n mysqld.stack
0x814e7f8 handle_segfault + 356
0x22de48 (?)
0xfaeeb0 (?)
0x8352bb9 free_root + 137
0x81928be _Z14free_tmp_tableP3THDP8st_table + 146
0x8190294 _ZN4JOIN7destroyEv + 540
0x82277e8 _ZN13st_select_lex7cleanupEv + 92
0x81903a0 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orderSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 260
0x818c6ba _Z13handle_selectP3THDP6st_lexP13select_resultm + 234
0x81621ad _Z21mysql_execute_commandP3THD + 669
0x8168d82 _Z11mysql_parseP3THDPcj + 306
0x8160aa6 _Z16dispatch_command19enum_server_commandP3THDPcj + 1178
0x81605d1 _Z10do_commandP3THD + 129
0x815fab1 handle_one_connection + 569
0x227de8 (?)
0x3f893a (?)

How to repeat:
Run the following queries

Select concat_ws(',',concat_ws(' ',x.nameFirst,x.nameLast),x.Client,x.cells,x.month) Results
from (
Select e.nameFirst, e.nameLast, extract(MONTH FROM t.minDate) month, b.name Client, count(c.id) cells
from Cell c, Plan p, Job j, Employee e, MMTask t, BusinessEntity b
WHERE p.campaignManagerEmployee_id = e.id
AND p.endClient = b.id
  AND p.id = j.plan_id
  AND j.id = c.job_id
  AND t.cell_id = c.id
  AND c.cellType = 3
  AND t.minDate > '2005-12-31'
  AND t.minDate < '2006-02-01'
  group by e.nameFirst, e.nameLast, b.name, extract(MONTH FROM t.minDate))x

or Explain on the same query

Also server crashed for inner select sql too
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xa7c3d30 = Select e.nameFirst, e.nameLast, extract(MONTH FROM t.minDate) month, b.name Client, count(c.id
) cells         from Cell c, Plan p, Job j, Employee e, MMTask t, BusinessEntity b         WHERE p.campaignManagerEmploy
ee_id = e.id AND p.endClient = b.id   AND p.id = j.plan_id         AND j.id = c.job_id   AND t.cell_id = c.id   AND c.ce
llType = 3   AND t.minDate > '2005-12-31'         AND t.minDate < '2006-02-01'         group by e.nameFirst, e.nameLast,
 b.name, extract(MONTH FROM t.minDate)
thd->thread_id=3

Suggested fix:
Don't know
[9 Feb 2006 10:32] Valeriy Kravchuk
Thank you for a problem report. Your 5.0.15 is old enough, so, please, try to repeat on 5.0.18. 

Anyway, the results of SHOW CREATE TABLE for all the tables used (Cell, Plan, Job, Employee, MMTask, and BusinessEntity) are needed. Please, send them.
[10 Feb 2006 0:52] sanjeev sagar
I've upgraded to 5.0.18 but it still didn't resolve the crash. please see below for messages and stack trace details

0x809f182 (?)
0x82dceb8 fsp_validate + 4708
0x82f97bc buf_page_release + 444
0x82f96a3 buf_page_release + 163
0x82a6b56 btr_search_build_page_hash_index + 1386
0x82a724c btr_search_build_page_hash_index + 3168
0x80ed080 _Z18resolve_const_itemP3THDPP4ItemS2_ + 700
0x80df3ce _Z5yylexPvS_ + 2910
0x816d051 _Z7yyparsePv + 2445
0x816cf19 _Z7yyparsePv + 2133
0x816d62b _Z7yyparsePv + 3943
0x816d166 _Z7yyparsePv + 2722
0x80d3b8c (?)
0x80b087e (?)
0x80b7378 (?)
0x80aefd3 (?)
0x80ae8f3 (?)
0x80ade44 (?)
0x82da66c fsp_reserve_free_extents + 292
0x8303faa page_copy_rec_list_end_to_created_page + 2022

Error File details:

        Select e.nameFirst, e.nameLast, extract(MONTH FROM t.minDate) month, b.name Client, count(c.id) cells
        from Cell c, Plan p, Job j, Employee e, MMTask t, BusinessEntity b
        WHERE p.campaignManagerEmployee_id = e.id AND p.endClient = b.id   AND p.id = j.plan_id
        AND j.id = c.job_id   AND t.cell_id = c.id   AND c.cellType = 3   AND t.minDate > '2005-12-31'
        AND t.minDate < '2006-02-01'
        group by e.nameFirst, e.nameLast, b.name, extract(MONTH FROM t.minDate))x

Tables DDL:

Create Table: CREATE TABLE `Cell` (
  `id` int(11) NOT NULL auto_increment,
  `cellType` int(11) NOT NULL default '0',
  `version` int(11) NOT NULL default '0',
  `inactive` tinyint(1) NOT NULL default '0',
  `isInternal` tinyint(1) default NULL,
  `description` varchar(255) default NULL,
  `notes` text,
  `cellSplashContent_id` int(11) default NULL,
  `state` int(11) NOT NULL default '0',
  `endDate` datetime default NULL,
  `startDate` datetime default NULL,
  `firstLive` datetime default NULL,
  `lastLive` datetime default NULL,
  `equivCells_id` int(11) NOT NULL default '0',
  `job_id` int(11) NOT NULL default '0',
  `molester_adminUser_id` int(11) default NULL,
  `welcomeEmail` varchar(255) default NULL,
  `targetURI` text,
  `hasGuardQuestion` tinyint(1) default '0',
  `guardQuestion_id` int(11) default NULL,
  `ad_id` int(11) default NULL,
  `copyText` text,
  `pointsText` varchar(255) default NULL,
  `parentCell_id` int(11) default NULL,
  `targetQuery` text,
  `emailSubject` varchar(255) default NULL,
  `emailFrom` varchar(255) default NULL,
  `block_id` int(11) default NULL,
  `changeTimestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  KEY `job_id` (`job_id`),
  KEY `cellSplashContent_id` (`cellSplashContent_id`),
  KEY `ad_id` (`ad_id`),
  KEY `equivCells_id` (`equivCells_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

Create Table: CREATE TABLE `Plan` (
  `id` int(11) NOT NULL auto_increment,
  `version` int(11) NOT NULL default '0',
  `inactive` tinyint(1) NOT NULL default '0',
  `isInternal` tinyint(1) default NULL,
  `endDate` datetime default NULL,
  `department` int(11) default NULL,
  `contract_id` int(11) default NULL,
  `endClient` int(11) default NULL,
  `purchaseOrder` varchar(255) default NULL,
  `accountExecutiveEmployee_id` int(11) default NULL,
  `agencyDiscount` int(11) default NULL,
  `campaignManagerEmployee_id` int(11) default NULL,
  `description` varchar(255) default NULL,
  `notes` text,
  `contact_id` int(11) default NULL,
  `targeting` varchar(255) default NULL,
  `duration` int(11) default NULL,
  `proposedLaunchDate` datetime default NULL,
  `changeTimestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  KEY `endClient` (`endClient`),
  KEY `contract_id` (`contract_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

Create Table: CREATE TABLE `Job` (
  `id` int(11) NOT NULL auto_increment,
  `version` int(11) NOT NULL default '0',
  `inactive` tinyint(1) NOT NULL default '0',
  `isInternal` tinyint(1) default NULL,
  `description` varchar(255) default NULL,
  `pricingModel` int(11) default NULL,
  `notes` text,
  `plan_id` int(11) default NULL,
  `changeTimestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  KEY `plan_id` (`plan_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

Create Table: CREATE TABLE `Employee` (
  `id` int(11) NOT NULL auto_increment,
  `version` int(11) NOT NULL default '0',
  `inactive` tinyint(1) NOT NULL default '0',
  `isInternal` tinyint(1) default NULL,
  `type` int(11) NOT NULL default '0',
  `email` varchar(255) default NULL,
  `location` varchar(255) default NULL,
  `nameFirst` varchar(255) default NULL,
  `nameLast` varchar(255) default NULL,
  `phone` varchar(255) default NULL,
  `changeTimestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

Create Table: CREATE TABLE `MMTask` (
  `id` int(11) NOT NULL auto_increment,
  `version` int(11) NOT NULL default '0',
  `inactive` tinyint(1) NOT NULL default '0',
  `isInternal` tinyint(1) default NULL,
  `minDate` datetime NOT NULL default '0000-00-00 00:00:00',
  `maxDate` datetime NOT NULL default '0000-00-00 00:00:00',
  `targetDeliveries` int(11) default NULL,
  `targetAttempts` int(11) default NULL,
  `maxRate` int(11) default NULL,
  `cell_id` int(11) NOT NULL default '0',
  `targetQuery` text,
  `status` int(11) NOT NULL default '0',
  `numAttempted` int(11) NOT NULL default '0',
  `numSent` int(11) NOT NULL default '0',
  `numFastBounced` int(11) NOT NULL default '0',
  `totalSequence` bigint(20) NOT NULL default '0',
  `purgeOnClose` tinyint(1) default '0',
  `changeTimestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  KEY `maxDate` (`maxDate`),
  KEY `FK87C9E3452786D518` (`cell_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

Create Table: CREATE TABLE `BusinessEntity` (
  `id` int(11) NOT NULL auto_increment,
  `version` int(11) NOT NULL default '0',
  `inactive` tinyint(1) NOT NULL default '0',
  `isInternal` tinyint(1) default NULL,
  `name` varchar(255) NOT NULL default '',
  `isRedemptionMerchant` tinyint(1) default NULL,
  `street1` varchar(255) default NULL,
  `street2` varchar(255) default NULL,
  `city` varchar(255) default NULL,
  `state_code` varchar(5) default NULL,
  `country_code` char(2) default NULL,
  `postal` varchar(255) default NULL,
  `ba_street1` varchar(255) default NULL,
  `ba_street2` varchar(255) default NULL,
  `ba_city` varchar(255) default NULL,
  `ba_state_code` varchar(5) default NULL,
  `ba_country_code` char(2) default NULL,
  `ba_postal` varchar(255) default NULL,
  `changeTimestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  UNIQUE KEY `name` (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
[16 Feb 2006 15:03] Valeriy Kravchuk
I was not able to repeat the problem you described with 5.0.19-BK:

mysql> Select e.nameFirst, e.nameLast, extract(MONTH FROM t.minDate) month,
    -> b.name Client, count(c.id) cells
    -> from Cell c, Plan p, Job j, Employee e, MMTask t, BusinessEntity b
    -> WHERE p.campaignManagerEmployee_id = e.id
    -> AND p.endClient = b.id
    -> AND p.id = j.plan_id
    -> AND j.id = c.job_id
    -> AND t.cell_id = c.id
    -> AND c.cellType = 3
    -> AND t.minDate > '2005-12-31'
    -> AND t.minDate < '2006-02-01'
    -> group by e.nameFirst, e.nameLast, b.name, extract(MONTH FROM t.minDate);
Empty set (0.01 sec)

mysql> select version();
+-----------+
| version() |
+-----------+
| 5.0.19    |
+-----------+
1 row in set (0.00 sec)

No crash. Have you created that tables in 5.0.18 or they left from the previous version of MySQL, 4.x.y? Can you upload some sample data for that tables that gives you the crash?

Please, send also EXPLAIN results for that statement in your environment.
[17 Mar 2006 0:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".