Bug #24607 | MyISAM pointer size determined incorrectly | ||
---|---|---|---|
Submitted: | 27 Nov 2006 6:52 | Modified: | 7 Feb 2007 12:16 |
Reporter: | Yoshiaki Tajika (Basic Quality Contributor) | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Server: MyISAM storage engine | Severity: | S2 (Serious) |
Version: | 5.0.32-BK, 5.0.27 | OS: | Linux (Linux) |
Assigned to: | Ingo Strüwing | CPU Architecture: | Any |
[27 Nov 2006 6:52]
Yoshiaki Tajika
[27 Nov 2006 8:13]
Valeriy Kravchuk
Thank you for a bug report. In current 5.0.32-BK sources we still have: uint mi_get_pointer_length(ulonglong file_length, uint def) { DBUG_ASSERT(def >= 2 && def <= 7); if (file_length) /* If not default */ { #ifdef NOT_YET_READY_FOR_8_BYTE_POINTERS if (file_length >= (longlong) 1 << 56) def=8; #endif if (file_length >= (longlong) 1 << 48) def=7; if (file_length >= (longlong) 1 << 40) def=6; else if (file_length >= (longlong) 1 << 32) def=5; else if (file_length >= (1L << 24)) def=4; else if (file_length >= (1L << 16)) def=3; else def=2; } return def; } So, there is obvious bug for 2 cases you noted.
[30 Dec 2006 21:55]
Ingo Strüwing
Hi. I will fix this bug. But it will change nothing. Still you will not be able to create tables >= 256T by using MAX_ROWS and AVG_ROW_LENGTH. The problem is that the .frm file has only four bytes for MAX_ROWS. So it is limited to 4G-1. Rows can have a maximum length of 64K-1. So the product is somewhere below 256T. The only way to get tables bigger than this is to SET GLOBAL myisam_data_pointer_size= 7 and *not* to specify MAX_ROWS and AVG_ROW_LENGTH. If you want to raise the limit of an existing table, do SET GLOBAL myisam_data_pointer_size= 7 and ALTER TABLE table_name MAX_ROWS= 0. A pointer size of 7 allows for data files up to 65535T. MAX_ROWS= 0 means unlimited row count. The rows are counted with 8-byte variables so it is possible to go beyond the highest possible limit. You can check the success of this operation with SHOW TABLE STATUS LIKE 'table_name'. Regards Ingo
[2 Jan 2007 14:55]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/17523 ChangeSet@1.2590, 2007-01-02 15:55:06+01:00, istruewing@chilla.local +1 -0 Bug#24607 - MyISAM pointer size determined incorrectly The function mi_get_pointer_length() computed too small pointer size for very large tables. Inserted missing 'else' between the branches for very large tables. No test case. The effect is not easy to show. When creating a table with AVG_ROW_LENGTH/MAX_ROWS, file_length cannot be big enough. MAX_ROWS is limited by a 4-byte space in the .frm file. Hence file_length is limited to < 256T. The result of mi_get_pointer_length() is used internally in myisampack and in the server for sorting and optimizing. These would require an existing table >= 256T to provoke an failure due to the incorrectly computed pointer length.
[5 Jan 2007 2:49]
Yoshiaki Tajika
Hi, according to the previous comment below, > These would require an existing table >= 256T to provoke an > failure due to the incorrectly computed pointer length. do you need some correction of code anywhere other than mi_get_pointer_length()?
[5 Jan 2007 9:19]
Ingo Strüwing
Hi, no, I don't think so. Unless one wants to fix the limitation of MAX_ROWS. But this could introduce all sorts of incompatibilities with former versions. Meanwhile I noticed that my explanations regarding the impossibility of creating tables >= 256T was wrong. I said that records cannot be bigger than 64K-1 bytes. This is wrong for records with blobs. The limit exists only for the sum of the non-blob parts of a record, including the blob length field and an in-memory pointer. The blob data itself don't count for the record length limit. Consequentially one can specify AVG_ROW_LENGTH > 65535. I will commit a new patch with a test case soon. Regards Ingo
[5 Jan 2007 9:27]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/17655 ChangeSet@1.2590, 2007-01-05 10:26:51+01:00, istruewing@chilla.local +3 -0 Bug#24607 - MyISAM pointer size determined incorrectly The function mi_get_pointer_length() computed too small pointer size for very large tables. Inserted missing 'else' between the branches for very large tables.
[24 Jan 2007 12:17]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/18698 ChangeSet@1.2593, 2007-01-24 13:17:01+01:00, istruewing@chilla.local +2 -0 Bug#24607 - MyISAM pointer size determined incorrectly Fixed test. On 32-bit machines which compile without -DBIG_TABLES, MAX_ROWS is truncated to a 32-bit value. Using a value below 4G is portable.
[24 Jan 2007 12:30]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/18700 ChangeSet@1.2363, 2007-01-24 13:29:46+01:00, istruewing@chilla.local +1 -0 Bug#24607 - MyISAM pointer size determined incorrectly After merge fix
[25 Jan 2007 8:23]
Ingo Strüwing
Queued to 5.1-engines, 5.0-engines, 4.1-engines.
[25 Jan 2007 9:30]
Sergey Vojtovich
Pushed to trees currently marked as 5.0.36 and 5.1.15.
[5 Feb 2007 11:42]
Sergey Vojtovich
Pushed to tree currently marked as 4.1.23.
[7 Feb 2007 12:16]
MC Brown
Closed. No documentation needed, and no additions to the changelog have been made.