MySQL Bugs: #17951: myisampack --force --silent : adnormal end in Windows XP

Bug #17951	myisampack --force --silent : adnormal end in Windows XP
Submitted:	6 Mar 2006 5:09	Modified:	11 Dec 2006 4:12
Reporter:	Mark V	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Command-line Clients	Severity:	S2 (Serious)
Version:	5.0.18-nt	OS:	Windows (WIN-XP-Pro (latest patches))
Assigned to:	Iggy Galarza	CPU Architecture:	Any

Description:
Hi,
I've come across instances of myisampack randomly crashing under windows XP Pro.

I loop over a large number of tables (+7000) of varying sizes from 'small', Kb's, to 'large', Mb's.  
The MySQL server is turned off, and line in the batch file that calls myisampack is below (%2 is the table name that is passed to the batch file)

IF EXIST %2.MYD myisampack --force --silent %2

Reproducing this is tough since this seems to be an intermitent problem.  
Further, it's not clear how to trap an error from myisampack within a batch file, if you have any suggestions I'd be happy to incorporate them, and this might help identify table characteristics that trigger such crashes.

In the meantime I have attached two of the MS generagted crash/log files - I tried to copy what looked like another relevant file but this was locked while the MS crash reporting tool dialog was displayed, and then deleted.

I'm not a MySQL or programming expert but am happy to try and work out when this is occuring, e.g. knowing a myisampack error number to trap (using the windows command prompt) would be useful.

Hope that helps and thanks for all your effort.

 

How to repeat:
Seems to be random?

File generated by MS crash reporting tool: #2

Attachment: c15f_appcompat.txt (text/plain), 10.73 KiB.

Thank you for a problem report. I had changed category to a more appropriate one. Can you upload the files for one of the smallest tables that gices you this crash of myisampack?

Thanks for the response.  The batch process hasn't finished running.
I've kept a log file, so hope to work out exactly which tables gave the problem - If I can I'll upload a file.  

I will need to run this batch file regularly, so ideally I'd like to check, after each call of myisampack, wether it crashed or not.  Hence my question about whether myisampack returns or sets a rturn value (or the errorlevel environment variable) that can be trapped in a windows batch file?  Once I know the codes that can be returned I'll be able to check after each use of myisampack, and log the problem cases.
Regards
Mark

Hi,
the batch job has finished.

I ran 

sed -n "/error/p;" <log.txt

This returned:
myisamchk: error: Can't create new tempfile: '631100_quotes.TMD'
MyISAM-table '631100_quotes' is not fixed because of errors
myisamchk: error: Can't create new tempfile: '631100_quotes.TMD'
MyISAM-table '631100_quotes.MYI' is not fixed because of errors
myisamchk: error: Can't create new tempfile: '73179P_replaytradesquotes.TMD'
MyISAM-table '73179P_replaytradesquotes' is not fixed because of errors
myisamchk: error: Can't create new tempfile: '654902_replaytradesquotes.TMD'
MyISAM-table '654902_replaytradesquotes' is not fixed because of errors
myisamchk: error: Can't create new tempfile: '500631_replaytradesquotes.TMD'
MyISAM-table '500631_replaytradesquotes' is not fixed because of errors
myisamchk: error: Can't create new tempfile: '416196_replaytradesquotes.TMD'
MyISAM-table '416196_replaytradesquotes' is not fixed because of errors
myisamchk: error: Can't create new tempfile: '11373M_replaytradesquotes.TMD'
MyISAM-table '11373M_replaytradesquotes' is not fixed because of errors
myisamchk: error: Can't create new tempfile: '06652V_replaytradesquotes.TMD'
MyISAM-table '06652V_replaytradesquotes' is not fixed because of errors

The table count is the number of times I saw the myisampack crash reporting tool dialog.
Note that _after_ myisampack I run the following myisamchk:

IF EXIST %2.MYD myisamchk --recover --sort-index --analyze %2

Hence it seems plausible that these are infact the tables that caused myisampack to crash.

If I run myisampack --force --silent on these I get:

myisampack: Unknown error 144
416196_replaytradesquotes gave error 144 on open

I will puload the smallest of the tables.

Regards
Mark

I've chosen one other table and can reproduce the crash of myisampack:

Using the files I uploaded:

>del 416196_replaytradesquotes.tmd
>myisamchk --recover 416196_replaytradesquotes
>ECHO %ERRORLEVEL%
'0'
>myisampack --force --silent 416196_replaytradesquotes
(crashes)
>ECHO %ERRORLEVEL%
-1073741819

The same result occurs if I use --safe-recover

I've upgraded the severity given that myisamchk --safe-recover or --recover both report success in repairing the table, and yet myisampack still crashes.  This suggests there is no work around - apart from not using myisampack.

Hope that helps.
Mark

Output of my session with 5.0.19-bk. Uploaded table prefix is 06652v_, but table prefix quoted is 416196_

Attachment: mylog.txt (text/plain), 4.87 KiB.

With 5.0.18-nt, it crashed with The instruction at "0x77fcd989" referenced memory at "0x00000000"

Attachment: mylog2.txt (text/plain), 5.20 KiB.

Verified to crash on Windows using 5.0.18 and 5.0.19.
Linux version didn't crash:

[sbester@fc4 bin]$ ./myisampack ../data/test/06652v_replaytradesquotes
Compressing ../data/test/06652v_replaytradesquotes.MYD: (44037 records)
- Calculating statistics
- Compressing file
78.13%     
Remember to run myisamchk -rq on compressed tables

I'm able to duplicate this 'myisampack' crash for MySQL v5.0.18 on 
Sun OS v5.10 CPU:i86pc 64bit.  The program 'segmentation' faults 

I get segmentation fault on the same table consistently.

I not able to uploaded my table (since I'm not the owner of this bug).
or cmd Line: myiasmpack -fw mdl_table_data_u300173601

This is after compressing ~1300 tables successfully.

****************************************************
cmd line: myisampack -fvvw mdl_table_data_u300173601
******* Output Prior to seg fault  *****************

column:  73  type:  0  pack:  0  zero:    0  lbits:  0  tree: 36  length:    5
column:  74  type:  0  pack:  0  zero:    0  lbits:  0  tree: 37  length:    3

tree:  1  elements:    6  char_bits:  3  offset_bits:  2  bufflen:    54  codelen:  5
tree:  2  elements:  256  char_bits:  8  offset_bits:  8  min_chr:     0  codelen:  9
tree:  3  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     2  codelen:  0
tree:  4  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     2  codelen:  0
tree:  5  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     2  codelen:  0
tree:  6  elements:    3  char_bits:  7  offset_bits:  2  min_chr:     1  codelen:  2
tree:  7  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     2  codelen:  0
tree:  8  elements:   35  char_bits:  6  offset_bits:  5  min_chr:     0  codelen: 16
tree:  9  elements:  256  char_bits:  8  offset_bits:  9  min_chr:     0  codelen: 13
tree: 10  elements:  249  char_bits:  8  offset_bits:  9  min_chr:     0  codelen: 12
tree: 11  elements: 2949  char_bits: 12  offset_bits: 13  bufflen: 14745  codelen: 15
tree: 12  elements:    7  char_bits:  3  offset_bits:  2  bufflen:    21  codelen:  5
tree: 13  elements: 2341  char_bits: 12  offset_bits: 12  bufflen: 11705  codelen: 15
tree: 14  elements:   93  char_bits:  8  offset_bits:  7  min_chr:     0  codelen: 16
tree: 15  elements:  255  char_bits:  8  offset_bits:  8  min_chr:     0  codelen: 16
tree: 16  elements:   42  char_bits:  6  offset_bits:  6  bufflen:   126  codelen:  9
tree: 17  elements:  808  char_bits: 10  offset_bits: 10  bufflen:  4040  codelen: 12
tree: 18  elements:  993  char_bits: 10  offset_bits: 10  bufflen:  2979  codelen: 13
tree: 19  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     5  codelen:  0
tree: 20  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     3  codelen:  0
tree: 21  elements:   63  char_bits:  6  offset_bits:  6  bufflen:   315  codelen:  8
tree: 22  elements:   32  char_bits:  5  offset_bits:  5  bufflen:    96  codelen:  7
tree: 23  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     5  codelen:  0
tree: 24  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     3  codelen:  0
tree: 25  elements:   28  char_bits:  6  offset_bits:  6  min_chr:     0  codelen: 14
tree: 26  elements:  104  char_bits:  7  offset_bits:  7  bufflen:   312  codelen: 14
tree: 27  elements:   95  char_bits:  7  offset_bits:  7  bufflen:   285  codelen: 14
tree: 28  elements:   17  char_bits:  5  offset_bits:  4  bufflen:    51  codelen:  9
tree: 29  elements:   82  char_bits:  7  offset_bits:  8  bufflen:   246  codelen: 13
tree: 30  elements:  713  char_bits: 10  offset_bits: 11  bufflen:  2139  codelen: 14
tree: 31  elements:    3  char_bits:  6  offset_bits:  2  min_chr:    32  codelen:  2
tree: 32  elements:    1  char_bits:  1  offset_bits:  2  bufflen:     5  codelen:  0
tree: 33  elements:    8  char_bits:  3  offset_bits:  2  min_chr:     1  codelen:  7
tree: 34  elements:   14  char_bits:  4  offset_bits:  5  bufflen:    42  codelen: 12
tree: 35  elements:    4  char_bits:  2  offset_bits:  2  min_chr:     0  codelen:  3
tree: 36  elements:  256  char_bits:  8  offset_bits:  8  min_chr:     0  codelen: 15
tree: 37  elements:  102  char_bits:  8  offset_bits:  7  min_chr:     0  codelen: 10
tree: 38  elements: 1573  char_bits: 11  offset_bits: 11  bufflen:  7865  codelen: 14

- Compressing file
wrote 18227 records.
Min record length:     30   Max length:     78   Mean total length:     49
Segmentation Fault(coredump)

Able to duplicate myisampack "crash" with Mysql v5.0.24a
Sun OS 5.10 cpu:i86pc 64bit.
Error details are the same as for MySQL 5.0.18

I'm using the same table from my previous comment.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/16266

ChangeSet@1.2297, 2006-11-30 21:04:02-05:00, iggy@rolltop.ignatz42.dyndns.org +1 -0
  Bug#17951: myisampack --force --silent : abnormal end in Windows XP
  -myisampack wrote to a block on the heap that it did not allocate.

Noted in 5.0.32, 5.1.15 changelogs.

myisampack wrote to unallocated memory, causing a crash.