Bug #24386 Performance degradation caused by instrumentation in mutex_struct
Submitted: 17 Nov 2006 11:53 Modified: 25 Jan 2007 16:09
Reporter: Marko Mäkelä
Status: Closed
Category:Server: InnoDB Severity:S2 (Serious)
Version:5.0, 5.1 OS:Any (all)
Assigned to: Marko Mäkelä Target Version:

[17 Nov 2006 11:53] Marko Mäkelä
Description:
Someone (probably Vadim Tkachenko) added a number of fields to mutex_t in order
to implement SHOW MUTEX STATUS. Especially the added mutex->count_using++
in mutex_enter_func() may contribute to the contention originally reported in Bug #22868.

How to repeat:
Run InnoDB on a multiprocessor system with multiple client connections. Observe the poor
scalability.

Suggested fix:
Enclose all added fields except count_os_wait in #ifdef UNIV_DEBUG and adapt the code.

To reduce the memory footprint further (to make mutex_t fit in fewer cache lines), also
remove mutex->level, mutex->magic_n and possibly mutex->lock_word from non-debug builds.
[20 Nov 2006 14:50] Sergei Golubchik
What speedup do you get with your patch ?
[20 Nov 2006 14:57] Heikki Tuuri
The patch at least saves about 80 bytes per buffer pool frame (in the newest 5.0 tree),
that is, 0.5 % of main memory. We do not yet have results if it affects scalability. Since
a mutex acquisition with Vadim's code caused two cache lines to be shipped, it might have
caused memory bus saturation.
[21 Dec 2006 21:02] Paul DuBois
Noted in 5.0.32 changelog.

The InnoDB mutex structure was simplified to reduce memory load.

Resetting report to In Progress pending decision about whether
to merge this patch into InnoDB 5.1. (The MySQL 5.0 patch has
been null-merged into MySQL 5.1.)
[30 Dec 2006 12:45] Vadim Tkachenko
Marko,

It would be interesting to look at benchmarks in real enviroment and under
profiler tool like Intel Vtune.

Yes, I made that change, but I don't think it is the reason of bad scalability.
[30 Dec 2006 16:18] Heikki Tuuri
Vadim,

looks like this had nothing to do with the thread thrashing problems. But Marko's patch is
likely to save some computer memory system bandwidth, since there is less ping-pong of
cache lines between processors after Marko's patch.

Best regards,

Heikki
[25 Jan 2007 16:09] Paul DuBois
No further documentation needed.