MySQL Bugs: #44418: Error setting NTFS compression attribute: 5

Bug #44418	Error setting NTFS compression attribute: 5
Submitted:	23 Apr 2009 4:45	Modified:	4 Jun 2009 7:02
Reporter:	jack andrews	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S3 (Non-critical)
Version:	mysql-5.1-telco-7.0	OS:	Windows
Assigned to:	jack andrews	CPU Architecture:	Any

Description:
ndbd on windows repeatedly reports:
  Error setting NTFS compression attribute: 5

How to repeat:
.

error 5 is "ERROR_ACCESS_DENIED Access is denied. "

i believe this is a problem related to the fdwCreate parameter in CreateFile().  in the help for DeviceIoControl, it states that the "fdwCreate parameter must specify OPEN_EXISTING" (which is not always the case).  

how is compression handled on other file systems (eg. ZFS)?  is there a policy to follow?  should we add options to compress files?  or /some/ files?

this is the code (note it's only in DEBUG)

#ifdef DEBUG
  DWORD bytesReturned;
  USHORT compressionFormat= COMPRESSION_FORMAT_DEFAULT;
  if(DeviceIoControl(hFile, FSCTL_SET_COMPRESSION, &compressionFormat, sizeof(USHORT), NULL, 0, &bytesReturned, NULL)==0)
  {
	  ndbout_c("Error setting NTFS compression attribute: %d", GetLastError());
  }
#endif

After having looked at the code I don't really understand why we try to turn on compression at all. If we should do it, it should be turned on the same way as it is on other platforms.

Looking in PosixAsyncFile.cpp, I see this:

  if (flags & FsOpenReq::OM_GZ)
  {
    use_gz = 1;
  }

I.e if the OM_GZ bit is set, we write to the file using 'azwrite'

Would suggest removing the code that turns on compression from Win32AsyncFile.cpp

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75145

2879 jack andrews	2009-05-28
      Bug #44418  Error setting NTFS compression attribute: 5

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75182

2932 jack andrews	2009-05-28
      Bug #44418  Error setting NTFS compression attribute: 5
      removed DEBUG code that was causing files to be compressed
      on NTFS

Pushed into 5.1.34-ndb-7.0.7 (revid:jack@sun.com-20090528185428-1rhvyq874xjx8xup) (version source revid:jack@sun.com-20090528185428-1rhvyq874xjx8xup) (merge vers: 5.1.34-ndb-7.0.7) (pib:6)

We turn on compression when built as DEBUG only to get around limited disk space on Pushbuild Windows hosts.

They didn't have enough disk space to:
a) start the cluster
b) start a master and slave cluster for testing replication

or some other variant.

So the patch of turning on NTFS file compression in DEBUG builds was born - which let pushbuild run NDB tests.

Maybe this has been improved with PB2 Windows hosts.. but it does *dramatically* increase the disk space needed to run NDB tests (by a few gigabytes).

Documented bugfix in the NDB-7.0.7 changelog as follows:

        Debugging code causing ndbd to use file compression on NTFS
        filesystems failed with an error. (The code was removed.) This
        issue affected debug builds of MySQL Cluster on Windows platforms 
        only.