MySQL Bugs: #61090: please use the VALGRIND_DO_QUICK_LEAK

Bug #61090	please use the VALGRIND_DO_QUICK_LEAK_CHECK macro in valgrind builds
Submitted:	8 May 2011 0:30	Modified:	20 May 2013 17:43
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: General	Severity:	S4 (Feature request)
Version:	5.5, 5.6	OS:	Linux
Assigned to:		CPU Architecture:	Any
Tags:	Leak, memcheck, memory leak, valgrind, VALGRIND_DO_LEAK_CHECK, VALGRIND_DO_QUICK_LEAK_CHECK

Description:
In long-running tests with millions of queries, a memory leak is only seen after shutting down valgrind and reading the leak report.  It's near impossible to guess which query caused the leak.

This is a feature to make valgrind report memory leaks after each query is run, or at least when a thread terminates.  Or, when some "debug" variable is set for a thread, then report the memory leaks..

It could even be possible to print out the query causing the leak in valgrind outputs.

How to repeat:
feature.

Suggested fix:
Include memcheck.h in the valgrind builds, and implement these macros in a suitable place in the code:

http://valgrind.org/docs/manual/mc-manual.html

"VALGRIND_DO_LEAK_CHECK: does a full memory leak check (like --leak-check=full) right now. This is useful for incrementally checking for leaks between arbitrary places in the program's execution. It has no return value.

VALGRIND_DO_QUICK_LEAK_CHECK: like VALGRIND_DO_LEAK_CHECK, except it produces only a leak summary (like --leak-check=summary). It has no return value.

VALGRIND_COUNT_LEAKS: fills in the four arguments with the number of bytes of memory found by the previous leak check to be leaked (i.e. the sum of direct leaks and indirect leaks), dubious, reachable and suppressed. This is useful in test harness code, after calling VALGRIND_DO_LEAK_CHECK or VALGRIND_DO_QUICK_LEAK_CHECK."

Unfortunately, this won't work in practice because we can cache a lot of things between statements. For example, when a statement causes a table to be loaded into the table cache, or when a SP/PS is saved for later execution. These would all be flagged as memory leaks.

This would work better in the context of a test case, where the test case must clean up by itself at the end. Fishing a comment from Bug#43418:

Before any test case is run, we would call a leak check and count the number of leaks (VALGRIND_COUNT_LEAKS) so that this can be compared with the count after each test case run. If the numbers differ, there is probably a memory leak somewhere..

I still think that in some cases it's still better to get false positives immediately than wait for more correct results till the end of big test and then guess what exact step caused the real leak.

i will try this myself. maybe some supressions will work to get rid of the expected allocations that are not freed?  if it proves somewhat useful, i will post my results here.

Should work somewhat fine for simple cases, just ensure to test it on a valgrind build. Also, this can perhaps be a UDF which maps the valgrind leaks functions, this way it can be easily controlled when leak checking is done.

I ended up using tcmalloc and discovered the memory consumption was due to the creation of thousands of differently named @user variables.  solution in my case was to run mysql_change_user() every 100 million queries to keep the memory footprint down.  

so for the meanwhile, this FR doesn't need any more attention... however, it will still be useful to detect tiny memory leaks from queries, where total consumption is very small and tcmalloc cannot really help.

What I shall like to do is check a value for VALGRIND_COUNT_ERRORS after each SQL has run, and if this has increased since the last previous query was run, then we can print the query to the mysql error log, or to the valgrind log using VALGRIND_PRINTF.  Or print all queries/processlist for example.

windows crt has similar functions that can make detecting memory leaks alot easier.
http://msdn.microsoft.com/en-us/library/x98tx3cf.aspx

So can we #define _CRTDBG_MAP_ALLOC on windows debug builds?

I've done this on windows platform, for my own testing.
Next generation of this I hope will be done in performance_schema era.

Hence, I close this feature, as valgrind is obsolete in my eyes.
It is simply too slow to use to track memory leaks for zillions of queries.

this is implemented mysql 5.7.2 with valgrind builds when running with 
option -T4096 under valgrind.

Nothing to do here, apparently. Closing with no changelog entry.