Bug #44418 Error setting NTFS compression attribute: 5
Submitted: 23 Apr 2009 4:45 Modified: 4 Jun 2009 7:02
Reporter: jack andrews Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:mysql-5.1-telco-7.0 OS:Microsoft Windows
Assigned to: jack andrews
Triage: Triaged: D4 (Minor) / R6 (Needs Assessment) / E6 (Needs Assessment)

[23 Apr 2009 4:45] jack andrews
Description:
ndbd on windows repeatedly reports:
  Error setting NTFS compression attribute: 5

How to repeat:
.
[25 May 2009 14:58] jack andrews
error 5 is "ERROR_ACCESS_DENIED Access is denied. "

i believe this is a problem related to the fdwCreate parameter in CreateFile().  in the help for DeviceIoControl, it states that the "fdwCreate parameter must specify OPEN_EXISTING" (which is not always the case).  

how is compression handled on other file systems (eg. ZFS)?  is there a policy to follow?  should we add options to compress files?  or /some/ files?

this is the code (note it's only in DEBUG)

#ifdef DEBUG
  DWORD bytesReturned;
  USHORT compressionFormat= COMPRESSION_FORMAT_DEFAULT;
  if(DeviceIoControl(hFile, FSCTL_SET_COMPRESSION, &compressionFormat, sizeof(USHORT), NULL, 0, &bytesReturned, NULL)==0)
  {
	  ndbout_c("Error setting NTFS compression attribute: %d", GetLastError());
  }
#endif
[26 May 2009 13:40] Magnus BlÄudd
After having looked at the code I don't really understand why we try to turn on compression at all. If we should do it, it should be turned on the same way as it is on other platforms.

Looking in PosixAsyncFile.cpp, I see this:

  if (flags & FsOpenReq::OM_GZ)
  {
    use_gz = 1;
  }

I.e if the OM_GZ bit is set, we write to the file using 'azwrite'

Would suggest removing the code that turns on compression from Win32AsyncFile.cpp
[28 May 2009 13:10] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75145

2879 jack andrews	2009-05-28
      Bug #44418  Error setting NTFS compression attribute: 5
[28 May 2009 18:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75182

2932 jack andrews	2009-05-28
      Bug #44418  Error setting NTFS compression attribute: 5
      removed DEBUG code that was causing files to be compressed
      on NTFS
[28 May 2009 18:59] Bugs System
Pushed into 5.1.34-ndb-7.0.7 (revid:jack@sun.com-20090528185428-1rhvyq874xjx8xup) (version source revid:jack@sun.com-20090528185428-1rhvyq874xjx8xup) (merge vers: 5.1.34-ndb-7.0.7) (pib:6)
[3 Jun 2009 0:39] Stewart Smith
We turn on compression when built as DEBUG only to get around limited disk space on Pushbuild Windows hosts.

They didn't have enough disk space to:
a) start the cluster
b) start a master and slave cluster for testing replication

or some other variant.

So the patch of turning on NTFS file compression in DEBUG builds was born - which let pushbuild run NDB tests.

Maybe this has been improved with PB2 Windows hosts.. but it does *dramatically* increase the disk space needed to run NDB tests (by a few gigabytes).
[4 Jun 2009 7:02] Jon Stephens
Documented bugfix in the NDB-7.0.7 changelog as follows:

        Debugging code causing ndbd to use file compression on NTFS
        filesystems failed with an error. (The code was removed.) This
        issue affected debug builds of MySQL Cluster on Windows platforms 
        only.