MySQL Bugs: #47095: Can't open_files_limit really be larger than 65535?

Bug #47095	Can't open_files_limit really be larger than 65535?
Submitted:	3 Sep 2009 10:31	Modified:	20 Jun 2010 22:51
Reporter:	Valeriy Kravchuk	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Options	Severity:	S3 (Non-critical)
Version:	5.0.84sp1	OS:	Any
Assigned to:	Georgi Kodinov	CPU Architecture:	Any

Description:
Our manual, http://dev.mysql.com/doc/refman/5.1/en/server-options.html#option_mysqld_open-files-limit, says:

"Range 	0-65535"

This is NOT true in case of Linux at least. The value is ulong (4 byte) internally, why 64K limit then? It all depends on OS actually, it seems.

How to repeat:
suse:/home2/openxs/dbs/5.1 # ulimit -n 100000
suse:/home2/openxs/dbs/5.1 # ulimit -n
100000
suse:/home2/openxs/dbs/5.1 # bin/mysqld_safe --user=openxs --open_files_limit=68000 &
[1] 19625
suse:/home2/openxs/dbs/5.1 # 091209 21:20:32 mysqld_safe Logging to '/home2/openxs/dbs/5.1/var/suse.err'.
091209 21:20:33 mysqld_safe Starting mysqld daemon with databases from /home2/openxs/dbs/5.1/var

suse:/home2/openxs/dbs/5.1 # exit
exit
openxs@suse:/home2/openxs/dbs/5.1> bin/mysql -uroot test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.39-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show global variables like '%open%';
+-------------------+----------+
| Variable_name     | Value    |
+-------------------+----------+
| have_openssl      | DISABLED |
| innodb_open_files | 300      |
| open_files_limit  | 100000   |
| table_open_cache  | 64       |
+-------------------+----------+
4 rows in set (0.00 sec)

Suggested fix:
Fix the manual, at least by making a note about Linux and what really limits this value.

The manual is correct: you can only set open_files_limit between 0 and 65536!
BUT: if not set, it will use the current limit from getrlimit (like 'ulimit -n')
So the bug is really that the variable open_files_limit cannot explicitly be set above 65536.

If one looks at the given bug report it clearly states 'ulimit -n 100000' and then uses mysqld_safe, which does not forward the --open_files_limit (should be --open-files-limit ?), hence is the global variable open_files_limit actually set to 100000 and not 68000! The test should not do ulimit -n at all, since that is exactly what open_files_limit is for.

If the open_files_limit is lower than what getrlimit returns, it will not be decreased.

I cannot really see why we have the startup option open_files_limit at all, since in mysqld_safe.sh it does 'ulimit -n <open_files_limit>' before starting mysqld.

On windows, also consider bug#24509 which limits it to 2048.

After some internal discussion I came to the conclusion that we have a bug in server here. open_files_limit should NOT be limited to 64K in any way, as (according to this my report), it can be made much higher anyway by setting ulimit (and Linux kernel parameters, if needed).

So, mysqld_safe also should NOT just ignore values for open_files_limit that are more than 64K (as it happens). The real limit depends on OS, and only on OS.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/105218

3448 Georgi Kodinov	2010-04-08
      Bug #47095: Can't open_files_limit really be larger than 65535?
      
      Several problems addressed:
      
      1. The maximum value for --open_files_limit on non-windows boxes
      is now raised to UINT_MAX (the maximum possible without significant
      changes in the code). The maximum value on windows is kept to be
      2048 due to a known limitation (bug 24509).
      
      2. mysqld_safe now supports --open_files_limit=xx in addition to 
      --open-files-limit=xx
      
      3. mysqld_safe always passes through --open[_-]files[_-]limit
      to the underlying mysqld. It used to pass it through only if it 
      the user running the script has access to the root directory or
      there was an --user argument specified.
      
      4. Fixed a prototype in my_file.c to match its counterpart in 
      the other #ifdef branch.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/105302

3449 Georgi Kodinov	2010-04-09
      Bug #47095: Can't open_files_limit really be larger than 65535?
      
      Several problems addressed:
      
      1. The maximum value for --open_files_limit on non-windows boxes
      is now raised to UINT_MAX (the maximum possible without significant
      changes in the code). The maximum value on windows is kept to be
      2048 due to a known limitation (bug 24509).
      
      2. mysqld_safe now supports --open_files_limit=xx in addition to 
      --open-files-limit=xx
      
      3. mysqld_safe always passes through --open[_-]files[_-]limit
      to the underlying mysqld. It used to pass it through only if it 
      the user running the script has access to the root directory or
      there was an --user argument specified.
      
      4. Fixed a prototype in my_file.c to match its counterpart in 
      the other #ifdef branch.

Pushed into 5.1.47 (revid:joro@sun.com-20100505145753-ivlt4hclbrjy8eye) (version source revid:kristofer.pettersson@sun.com-20100416145616-vdcdyz9eu7j86lp7) (merge vers: 5.1.47) (pib:16)

Noted in 5.1.47 changelog.

mysqld_safe did not always pass --open-files-limit through to mysqld.
mysqld_safe did not treat dashes and underscores as equivalent in
option names. 

Setting report to Need Merge pending further pushes.

Pushed into mysql-next-mr (revid:alik@sun.com-20100524190136-egaq7e8zgkwb9aqi) (version source revid:alik@sun.com-20100422150750-vp0n37kp9ywq5ghf) (pib:16)

Pushed into 6.0.14-alpha (revid:alik@sun.com-20100524190941-nuudpx60if25wsvx) (version source revid:alik@sun.com-20100422150658-fkhgnwwkyugtxrmu) (merge vers: 6.0.14-alpha) (pib:16)

Pushed into 5.5.5-m3 (revid:alik@sun.com-20100524185725-c8k5q7v60i5nix3t) (version source revid:alexey.kopytov@sun.com-20100411071742-a2o2anlcrj2bq14q) (merge vers: 5.5.4-m3) (pib:16)

Noted in 5.5.5, 6.0.14 changelogs.

Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)