Bug #38908 mtr should refuse to run if there are not enough disk/memory to run tests

• Submitted: 20 Aug 2008 10:50
• Modified: 9 Jan 2015 14:34
• Reporter: Sven Sandberg
• Status: Won't fix
• Category: Tools: MTR / mysql-test-run
• Severity: S7 (Test Cases)
• Version: 5.1
• OS: Any
• Assigned to: Bjørn Munch
• CPU Architecture: Any
• Tags: 5.1-rpl-green, 6.0-rpl-green, disk overflow, feature request, mtr, out of memory, resource

[20 Aug 2008 10:50] Sven Sandberg

Description:
When the disk on a pb host is full or nearly full, it causes random errors in tests. These errors can be difficult to debug.

How to repeat:
See, e.g., BUG#37715, or the failure in main.upgrade on:

https://intranet.mysql.com/secure/pushbuild/showpush.pl?dir=bzr_mysql-6.0-rpl-testfixes-me... win2003-x86/ps_row

Suggested fix:
Make mtr check the amount of free disk space on startup. If it uses temporary directories that may reside on another partition, check the amount of free disk space there too. If there is not enough space, refuse to run. The box in pushbuild should be black (host problem), not red (test failure). That will save lots of developer time as it indicates the right source of the error.

Note that there must be enough disk space even if other instances of mtr are running in parallel. I would suggest at least 1 GB per instance. Disk space is cheap, and 1 GB should be enough for some time. Note that saved cores can grow quite quickly, unless cores are compressed as soon as they are found.

[22 Aug 2008 11:53] Sven Sandberg

Just a note: it is probably best to check disk space between each test, or at least for each server restart, since saved cores and possibly other debug info can eat up disk space.

[25 Aug 2008 15:51] Magnus Blåudd

Even if we would check this, it is still possible that another process on the test host takes up all the disk space.

Use user quotas and or test machines that are not being used by anyone else.

[8 Oct 2008 9:51] Magnus Blåudd

We will never fix this in mtr.pl

[22 Oct 2008 6:20] Lars Thalmann

Re-opening this so that Lars H and Bjorn can have a look at 
this problem and decide what we should do about it.

[21 Jan 2009 11:42] Alfranio Junior

Several failures are happening due to the lack of resources in the servers, most of them happening in the embedded mode. You can find below a few examples:

https://intranet.mysql.com/secure/pushbuild/showpush.pl?dir=bzr_mysql-6.0-rpl&order=147 [sol10-amd64-a/embedded]

main.collation_connection_basic
http://tinyurl.com/7m8hld

main.drop
http://tinyurl.com/a8sbeh

main.big_tables_basic
http://tinyurl.com/7k3dkp

main.null
http://tinyurl.com/76pbep

main.null_key
http://tinyurl.com/7zpxkz

main.innodb_data_home_dir_basic
http://tinyurl.com/8dhs3

main.innodb_log_file_size_basic
http://tinyurl.com/9evw7r

[21 Jan 2009 12:09] Alfranio Junior

Please, take also a look at BUG#40156, BUG#40400.

You can find more examples of failures due to the lack of disk/memory space:
BUG#40114, BUG#40109, BUG#38970, BUG#37779, BUG#40133, BUG#40155.