Bug #24509 2048 file descriptor limit on windows needs increasing
Submitted: 22 Nov 2006 15:09 Modified: 15 Jun 2010 18:37
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
Category:MySQL Server: General Severity:S3 (Non-critical)
Version:5.0,5.1 OS:Microsoft Windows (Windows)
Assigned to: Vladislav Vaintroub CPU Architecture:Any
Tags: 2048, bfsm_2006_12_07, bfsm_2007_02_01, fd, file descriptor, limit

[22 Nov 2006 15:09] Shane Bester
Currently the number of fd's on windows is limited to 2048.
This is bad for performance, for many reasons, mostly windows servers
are stuck with a tiny table_cache or low number of concurrent connections,
compared with linux running on the same hardware.

How to repeat:
Run the server

mysqld-nt.exe --table_cache=1000 --max_connections=500 --open_files-limit=3000 --console

061122 17:04:27 [Warning] Could not increase number of max_open_files to more than 2048 (request: 2510)
061122 17:04:28  InnoDB: Started; log sequence number 0 43655
061122 17:04:28 [Note] mysqld-nt: ready for connections.
Version: '5.0.30-enterprise-gpl-nt'  socket: ''  port: 3306  MySQL Enterprise Server (GPL)

Suggested fix:
Not sure.  Perhaps break away from posix and use native win32 api's on windows?
[3 Dec 2006 15:49] MySQL Verification Team
Looking in the visual studio crt sources, we have this:

setmaxf.c -  _setmaxstdio()
int __cdecl _setmaxstdio (
 int maxnum
 void **newpiob;
 int i;
 int retval;

  * Make sure the request is reasonable.
 if ( (maxnum < _IOB_ENTRIES) || (maxnum > _NHANDLE_) )
  return -1;

Then, look how _NHANDLE_ is defined:

#define IOINFO_L2E          5
#define IOINFO_ARRAYS       64

Thus, ( 1 << 5 ) * 32 = 2048

There are also limits on the API CreateFile() functions too, (10000 by default, I believe)  They are extensible via registry editing, however.  
If a testcase is needed, I can try write one to test the theories.
[4 Dec 2006 6:08] MySQL Verification Team
Firstly, sorry if my analysis isn't 100% correct, I'm not a developer.. 

I looked at the code in my_open.c, and noticed that
Win32 function CreateFile() is used to open a file on windows.
Then, a c runtime handle is created using the _open_osfhandle() function.

But, when I tested this, the following happened:

SHOW TABLE STATUS on 3000 tables.  The mysqld-nt.exe 'open file' count reported by sysinternal's handle.exe program easily went up to over 5000 !!! However, errno 24 was being returned by the storage engine, because _open_osfhandle failed.

It looks like a HANDLE leak exists in my_sopen().  CreateFile() will succeed,
but _open_osfhandle() can fail at 2048.. In this case, the handle from CreateFile is never closed!!  Even FLUSH TABLES leaves a few thousand open HANDLE objects created by CreateFile() and never closed.  The result is ever increasing open win32 HANDLE's.
[10 Dec 2006 10:42] MySQL Verification Team
Tim, this raising from 2048 to 16384 isn't as simple as #ifdef.
It's a hard windows limit due to the fact of using posix-like
functions in some places.  I will open 2nd bug report about a 
handle leak when that 2048 limit is hit.
[10 Dec 2006 10:44] MySQL Verification Team
btw, I did tests using a small app, 100% win32 api, and found win2003 standard easily opens 45000+ files at a time, and win2003 enterprise went even higher than that.
[17 Jan 2007 17:17] Iggy Galarza
I spent some time reading the MSDN information on this problem and there is indeed a hard 2048 file limit when using the _open_osfhandle() function. 
[See http://msdn2.microsoft.com/en-us/library/kdfaxaay(VS.80).aspx]

In order to work around this, the Win32 native HANDLEs should be used instead of the C file handle.
[21 Feb 2007 14:40] MySQL Verification Team
2048 open posix file limit also exists on 64-bit windows even if binary is built as 64-bit. see attached testcase.
[21 Feb 2007 14:41] MySQL Verification Team

Attachment: fileposix.cpp (text/plain), 844 bytes.

[21 Feb 2007 14:47] MySQL Verification Team
On my 64-bit XP, the debug binary asserts of you try increase limit > 2048 using _setmaxstdio() call.

The only solution is to use 100% win32 handles for file i/o on windows, like innodb does.
[17 Jun 2008 23:37] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:


2657 Vladislav Vaintroub	2008-06-18
      Bug#24509 - 2048 file descriptor limit on windows needs increasing, also 
      WL#3049 - improved Windows I/O
      The patch replaces the use of the POSIX I/O interfaces in mysys on Windows with 
      the Win32 API calls (CreateFile, WriteFile, etc). The Windows HANDLE for the open 
      file is stored in the my_file_info struct, along with a flag for append mode 
      because the Windows API does not support opening files in append mode in all cases)
      The default max open files has been increased to 16384 and can be increased further
      by setting --max-open-files=<value> during the server start.
      Another major change in this patch that almost all Windows specific file IO code
      has been moved to a new file my_winfile.c, greatly reducing the amount of code 
      in #ifdef blocks within mysys, thus improving readability.
      Minor enhancements:
      - my_(f)stat() is changed to use __stati64 structure with  64 file size
      and timestamps. It will return correct file size now (C runtime implementation
      used to report outdated information)
      - my_lock on Windows is prepared to handle additional timeout parameter
      - after review : changed __WIN__ to _WIN32 in the new and changed code.
[7 Apr 2009 5:59] Sveta Smirnova
Bug #44122 was marked as duplicate of this one.
[8 Apr 2009 11:43] MySQL Verification Team
Bug http://bugs.mysql.com/bug.php?id=44147 has been marked as duplicate of this one.
[1 Dec 2009 15:34] Vladislav Vaintroub
Fixed in 5.5
there is no hard limit on the number of open files anymore.
[1 Dec 2009 19:44] Paul Dubois
Noted in 5.5.0 changelog.
[16 Dec 2009 3:11] Vladislav Vaintroub
Since I promised to inform interested parties, here is the announcement
fix is released in 5.5