MySQL Bugs: #81781: Memory table with massive compound b-tree primary key

Bug #81781	Memory table with massive compound b-tree primary key
Submitted:	8 Jun 2016 20:40	Modified:	9 Jun 2016 13:32
Reporter:	Daniel Gary	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	MySQL Server: Memory storage engine	Severity:	S3 (Non-critical)
Version:	5.6.30	OS:	Ubuntu
Assigned to:		CPU Architecture:	Any

Description:
Been having an issue with a b-tree index on a rather large Memory table where the index fills the available max_heap well before it should.
max_heap_table_size and tmp_table_size both set to 32GB

just over 6GB of data but we use a covering index so the index is rather large
works fine with a hash index but we do a lot of range queries across dates

only thing I could find that matched is a thread from mariadb
http://lists.askmonty.org/pipermail/commits/2012-April/003198.html
https://bugs.launchpad.net/maria/+bug/794680
and it certainly seems to line up space wise from the testing I've done

checking the stock code and percona's it looks like that change in maria has not been applied here

How to repeat:
Make a memory table with a b-tree index larger than 4GB

Suggested fix:
=== modified file 'storage/heap/hp_write.c'
@@ -116,7 +116,7 @@
 		    uchar *recpos)
 {
   heap_rb_param custom_arg;
-  uint old_allocated;
+  size_t old_allocated;
 
   custom_arg.keyseg= keyinfo->seg;
   custom_arg.key_length= hp_rb_make_key(keyinfo, info->recbuf, record, recpos);

Hello Daniel Gary,

Thank you for the bug report.
This sounds like duplicate of Bug #44138.
Thank you for your interest in MySQL.

Thanks,
Umesh

That it does Umesh, do you know if there is ever going to be any action on 44138? It was filed in 2009 and last updated in 2013, considering the impact and the ease of patching the amount of time this bug has sat untouched is rather frightening.

Talking to some of our team this bug has likely been impacting them for years and they've simply assumed they were hitting a limit to the size of a Memory table and worked around it but as the dataset grows that becomes more and more unrealistic to achieve. I'll be setting up a MariaDB install for them in the meantime to use for batch processing but I'd prefer to avoid yet-another-server-syndrome.