MySQL Bugs: #66826: A strategy to make a core-dump quicker

Bug #66826	A strategy to make a core-dump quicker
Submitted:	15 Sep 2012 10:52	Modified:	22 Dec 2012 13:20
Reporter:	xiaobin lin (OCA)	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: Errors	Severity:	S4 (Feature request)
Version:	5.5+	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	Corefile, free buffer pool

Description:
In the production environment, we want to use core file to diagnose when the mysqld crash.

But another question is that, while the InnoDB buffer pool is up to at least 24G, the period of writting a core-file may take some time. We have occured that the period uses more than one minute.

This prevents us from configuring to write coredump.
And when coredump occurs, no core file, much information is lost.

How to repeat:
As above

Suggested fix:
I think if we free the buffer pool before writing the core-file, it may take much less time, for the reason of not need to dump the buffer pool information.

Perhaps it will lost some information, but in many cases, we are just intresting with the stack information.

The patch bellow is for the concept above.

Based on 5.5.27  (revno: 3900)

Attachment: free_bp_before_core_dump.5.5.diff (application/octet-stream, text), 2.72 KiB.

Just exclude the buffer pool range from the core (see MADV_DONTDUMP in madvise(2)).

It is a useful feature.  I think calling free() in signal handler would be risky though, so final solution should be discussed more..

with buffer pools in the hundreds of gigabytes these days, this feature becomes more important.

low-hang-fruit, based on mysql-5.6.16: https://launchpadlibrarian.net/167969706/innodb_dump_full_corefile.patch

I think that omitting the buffer pool from the core dump should be optional (which it is in the patch). A few times I have conducted some post-mortem analysis based on the buffer pool contents.

Having bumped into this type of problem a few times I'd agree with Shane. With large servers making the disk space available for a full dump can be tricky to start with so making it possible for the behaviour to be a reduced dump (with no buffer pool) first/by default seems like a good idea. If you really need the full dump that's because the smaller one or the stack trace without a coredump has been insufficient so by then you'll be willing to make space available and take the hit that implies.

I too would like this. The ticket was made for 5.5 and I'm guessing (since I hadn't seen this bug report before) that there's little chance of it making it into the 5.7 release due to time constraints. However, it would be nice if the patch is not too intrusive.

Would be great to have an option for this. Not only does it take too long to write a huge buffer pool in the core dump back to disk, but huge core dump files increase the chance of running out of disk space.

Related: Bug#91455

Bug #91455 marked as duplicate of this one