Bug #61358 "Out of memory" error when reading ARCHIVE table
Submitted: 30 May 2011 19:42 Modified: 9 Aug 2011 1:04
Reporter: Ask Hansen (Basic Quality Contributor) Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server: Archive storage engine Severity:S2 (Serious)
Version:5.5.10 OS:Linux (x86)
Assigned to: CPU Architecture:Any
Tags: qc

[30 May 2011 19:42] Ask Hansen
Description:
An ARCHIVE table with historical data from the NTP Pool project (www.pool.ntp.org) has gone corrupt without notice (until now when I tried to read it).

Reading the table makes mysqld return an "out of memory" error:

mysql> select count(*) from log_scores_archive where id  > 1;

ERROR 1041 (HY000): Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use more memory or you can add more swap space.

If I run 'repair table ...' it generates an .ARN file but eventually says it can't repair the table.   If I try again (or just do a select on the table again) the .ARN file grows by another gigabyte (the original file is about 1GB).

I tried using the archive_reader tool to get the data out, but it crashes with the backtrace below:

Starting program: /Users/ask/bin/archive_reader -b  log_scores_archive.ARZ  backup2.ARZ
Version 3
	Minor version 1
	Start position 8796
	Block size 16384
	Rows 87741732
	Autoincrement 0
	Check Point 1091516491
	Forced Flushes 1
	Longest Row 45
	Shortest Row 37
	State clean
	FRM stored at 78
	Comment stored at 8796
	Data starts at 8796
	FRM length 8718
	Comment length 0
		

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000100300000
0x00007fffffe00bee in __memcpy ()
(gdb) 
(gdb) bt
#0  0x00007fffffe00bee in __memcpy ()
#1  0x00000001001497d9 in fill_window ()
#2  0x000000010014af5c in deflate_slow ()
#3  0x000000010014a31e in deflate ()
#4  0x00000001000021e5 in azwrite ()
#5  0x0000000100001a08 in main ()

Any ideas for how I can extract the data?  I have daily backups going back months, but this data is only stored in this file and it appears the corruption happened a good while ago.

How to repeat:
.ARZ and .frm files at

http://mirrors.develooper.com/ask/mysql-archive/
http://mirrors.develooper.com/ask/mysql-archive/log_scores_archive.frm
http://mirrors.develooper.com/ask/mysql-archive/log_scores_archive.ARZ

Suggested fix:
1) Make mysqld more careful in not corrupting the data (or noticing it more easily).

2) Make 'repair table archive_table'  (and/or archive_reader) more robust (and the latter installed by default).
[31 May 2011 5:18] Alexey Kishkin
which mysql version do you use?
[31 May 2011 5:23] Ask Hansen
Hi Alexey,

5.5.10, Linux i386.

I tried reading this with 5.5.10 on Linux x86_64, with 5.1.57 on 64-bit darwin (OS X) and with archive_reader from 5.5.12 on 64-bit OS X.
[8 Jun 2011 18:57] Sveta Smirnova
Thank you for the feedback.

I can not repeat described behavior with data provided. Please send us your configuration file and output of `free`
[8 Jul 2011 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[9 Aug 2011 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".