MySQL Bugs: #38983: regression in test coverage

Bug #38983	regression in test coverage
Submitted:	23 Aug 2008 14:28	Modified:	9 Dec 2009 22:37
Reporter:	Daniel Fischer	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Tests	Severity:	S7 (Test Cases)
Version:	5.0	OS:	Any
Assigned to:	Patrick Crews	CPU Architecture:	Any

Description:
In May, test coverage for the MySQL 5.0 server of 69.2% was achieved by the main regression test suite only.

In August, only three months later, the main regression test suite *plus* the funcs1 and funcs2 test suites only achieve combined test coverage of 68.3%.

In the same time frame, test coverage in MySQL 5.1 has only gone down 0.1 percent points, from 69.2% to 69.1%. MySQL 6.0 has gone down from 67.2% to 66.9%.

This should be treated as a regression for the following two reasons:
(a) We should not be adding code without test cases at all.
(b) We should especially not be adding untested code to our stable branch.

How to repeat:
Build with profiling support, run the "test-bt" make target, generate a coverage report with gcov/lcov.

An issue with MTRv2, bug#46327  would cause lower test coverage to be recorded. In addition, there were issues with which trees PB2 computes coverage and lcov reports on.

Right now all trees show a consistent percentage, with the exception of the 6.0-codebase trees, for which the root cause of the problem is still being sought.

Once we have stable gcov/lcov calculations and reports, we will start tracking longer-term trends and improvements.

Patrick,

Once we are done with the analysis of the 6.0 coverage situation, please put a comment to this bug about what happened and then include the latest coverage percentages. You can then close the bug altogether, since we will try to improve code coverage in the longer term, and a bug is not the appropriate venue for this. Thank you.

Patrick,

Once we are done with the analysis of the 6.0 coverage situation, please put a comment to this bug about what happened and then include the latest coverage percentages. You can then close the bug altogether, since we will try to improve code coverage in the longer term, and a bug is not the appropriate venue for this. Thank you.

This seems to be an issue with NDB coverage not being registered / the system isn't exiting properly so that the gcov numbers can be updated and stored.

To be handled by the NDB team / MTR maintenance