MySQL Bugs: #38292: MyISAM project code does not compile on Windows using Visual Studio 9.0

Bug #38292	MyISAM project code does not compile on Windows using Visual Studio 9.0
Submitted:	22 Jul 2008 18:16	Modified:	13 Feb 2009 15:38
Reporter:	Chuck Bell	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S3 (Non-critical)
Version:	6.0.6	OS:	Windows
Assigned to:		CPU Architecture:	Any

Description:
The latest code in mysql-6.0 does not compile on Windows using Visual Studio 2008 v9.0. The errors are in the myisam storage engine code and include:

1>------ Build started: Project: ZERO_CHECK, Configuration: Debug Win32 ------
1>Checking Build System
1>-- Configuring done
1>-- Generating done
1>-- Build files have been written to: C:/source/bzr/mysql-.0-wl-4073
...
31>------ Build started: Project: myisam, Configuration: Debug Win32 ------
31>Compiling...
31>myisam_backup_engine.cc
31>C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1040) : error C2733: second C linkage of overloaded function '_interlockedbittestandset' not allowed
31>        C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1040) : see declaration of '_interlockedbittestandset'
31>C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1041) : error C2733: second C linkage of overloaded function '_interlockedbittestandreset' not allowed
31>        C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1041) : see declaration of '_interlockedbittestandreset'
...
========== Build: 55 succeeded, 12 failed, 0 up-to-date, 1 skipped ==========

It appears that the Windows system include file 'intrin.h' is included twice. Once in include\atomic\generic-msvc.h:

#if (_MSC_VER >= 1400)
#include <intrin.h>
#else
 ...

and again in include\atomic\nolock.h:

#  ifdef HAVE_GCC_ATOMIC_BUILTINS
#    include "gcc_builtins.h"
#  elif __GNUC__
#    include "x86-gcc.h"
#  elif defined(_MSC_VER)
#    include "generic-msvc.h"
#  endif
#endif

One of these must be eliminated.

How to repeat:
Branch the latest code and build on Windows using VS9.

Suggested fix:
It seems the original intent was to include the intrin.h file IFF using VS 8 and higher (_MSVC >= 1400). However, this causes a probem on VS 9 (_MSVC == 1500). It also seems plausible the code in generic-msvc.h and nolock.h are related and should be made to match such as the following diff:

=== modified file 'include/atomic/nolock.h'
--- include/atomic/nolock.h     2008-07-09 07:12:43 +0000
+++ include/atomic/nolock.h     2008-07-22 18:01:06 +0000
@@ -27,7 +27,9 @@
 #  elif __GNUC__
 #    include "x86-gcc.h"
 #  elif defined(_MSC_VER)
-#    include "generic-msvc.h"
+#    if (_MSC_VER < 1400)
+#      include "generic-msvc.h"
+#    endif
 #  endif
 #endif

Patch suggestion sent to runtime team who has ok'd its use. Patch is:

@@ -1400,6 +1400,13 @@ ViewBaseObjectsIterator::create(THD *thd
   close_thread_tables(my_thd);
   delete my_thd;

+  /*
+    The mysys_var context is no longer valid and must be reset.
+  */
+  pthread_mutex_lock(&thd->LOCK_delete);
+  thd->mysys_var= NULL;
+  pthread_mutex_unlock(&thd->LOCK_delete);
+
   thd->store_globals();

Oops. Updated wrong bug. Please disregard last post. Reversing changes. Sorry.

I just tried again with a fresh pull from mysql-6.0-backup. It appears the patch is indeed applied. However, I still get the error:

5>C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1040) : error C2733: second C linkage of overloaded function '_interlockedbittestandset' not allowed
55>        C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1040) : see declaration of '_interlockedbittestandset'
55>C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1041) : error C2733: second C linkage of overloaded function '_interlockedbittestandreset' not allowed
55>        C:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1041) : see declaration of '_interlockedbittestandreset'
...

Here's is what I did:

win\configure WITH_INNOBASE_STORAGE_ENGINE WITH_PARTITION_STORAGE_ENGINE WITH_ARCHIVE_STORAGE_ENGINE WITH_BLACKHOLE_STORAGE_ENGINE WITH_EXAMPLE_STORAGE_ENGINE WITH_FEDERATED_STORAGE_ENGINE __NT__  EXTRA_DEBUG ENABLED_DEBUG_SYNC BACKUP_TEST "COMPILATION_COMMENT=MySQL-new-rpl clone"

cmake -G "Visual Studio 9 2008"

bzr branch [blah blah blah] mysql-6.0-bug-38292
cd mysql-6.0-38292

<open mysql.sln>
F7 -build all

I have VS9:
Microsoft Visual Studio 2008
Version 9.0.21022.8 RTM
Microsoft .NET Framework
Version 3.5

Installed Edition: Professional

Microsoft Visual Basic 2008   91605-130-0014324-60304
Microsoft Visual Basic 2008

Microsoft Visual C# 2008   91605-130-0014324-60304
Microsoft Visual C# 2008

Microsoft Visual C++ 2008   91605-130-0014324-60304
Microsoft Visual C++ 2008

Microsoft Visual Studio 2008 Tools for Office   91605-130-0014324-60304
Microsoft Visual Studio 2008 Tools for Office

Microsoft Visual Web Developer 2008   91605-130-0014324-60304
Microsoft Visual Web Developer 2008

I have cmake 2.6.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

This problem still exists. 

Build using VS2008 (v9.0) on Widows Vista 32-bit.

Chuck,

I tried to build v 6.0.7 in VS 2008 express:
========== Build: 61 succeeded, 0 failed, 1 up-to-date, 1 skipped ==========

perhaps it was fluctuation in 6.0.6 which has already been fixed...

This is still a problem. Please try the mysql-6.0-backup tree. 

I branched the mysql-6.0-tree:

c:\source\bzr>bzr branch mysql-6.0-backup mysql-6.0-bug-38292
Branched 2726 revision(s).

Here is the output from cmake:

c:\source\bzr\mysql-6.0-bug-38292>win\configure WITH_INNOBASE_STORAGE_ENGINE WIT
H_PARTITION_STORAGE_ENGINE WITH_ARCHIVE_STORAGE_ENGINE WITH_BLACKHOLE_STORAGE_EN
GINE WITH_EXAMPLE_STORAGE_ENGINE WITH_FEDERATED_STORAGE_ENGINE __NT__  EXTRA_DEB
UG ENABLED_DEBUG_SYNC BACKUP_TEST "COMPILATION_COMMENT=MySQL-new-rpl clone"

c:\source\bzr\mysql-6.0-bug-38292>cmake -G "Visual Studio 9 2008"
-- Check for working C compiler: cl
-- Check for working C compiler: cl -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: cl
-- Check for working CXX compiler: cl -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
CMake Warning (dev) at extra/CMakeLists.txt:22 (ADD_EXECUTABLE):
  Policy CMP0003 should be set before this line.  Add code such as

    if(COMMAND cmake_policy)
      cmake_policy(SET CMP0003 NEW)
    endif(COMMAND cmake_policy)

  as early as possible but after the most recent call to
  cmake_minimum_required or cmake_policy(VERSION).  This warning appears
  because target "comp_err" links to some libraries for which the linker must
  search:

    ws2_32

  and other libraries with known full path:

    C:/source/bzr/mysql-6.0-bug-38292/dbug/Debug/dbug.lib
    C:/source/bzr/mysql-6.0-bug-38292/mysys/Debug/mysys.lib
    C:/source/bzr/mysql-6.0-bug-38292/strings/Debug/strings.lib
    C:/source/bzr/mysql-6.0-bug-38292/zlib/Debug/zlib.lib

  CMake is adding directories in the second list to the linker search path in
  case they are needed to find libraries from the first list (for backwards
  compatibility with CMake 2.4).  Set policy CMP0003 to OLD or NEW to enable
  or disable this behavior explicitly.  Run "cmake --help-policy CMP0003" for
  more information.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: C:/source/bzr/mysql-6.0-bug-38292
c:\source\bzr\mysql-6.0-bug-38292>

Here are (some of) the warnings/errors - it's a long list.

0>c:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1040) : error C2733: second C linkage of overloaded function '_interlockedbittestandset' not allowed
40>        c:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1040) : see declaration of '_interlockedbittestandset'
40>c:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1041) : error C2733: second C linkage of overloaded function '_interlockedbittestandreset' not allowed
40>        c:\Program Files\Microsoft Visual Studio 9.0\VC\include\intrin.h(1041) : see declaration of '_interlockedbittestandreset'
...

Here is the patch that makes all the errors go away:

=== modified file 'include/atomic/nolock.h'
--- include/atomic/nolock.h	2008-07-09 07:12:43 +0000
+++ include/atomic/nolock.h	2008-07-22 18:16:53 +0000
@@ -27,7 +27,9 @@
 #  elif __GNUC__
 #    include "x86-gcc.h"
 #  elif defined(_MSC_VER)
-#    include "generic-msvc.h"
+#    if (_MSC_VER < 1400)
+#      include "generic-msvc.h"
+#    endif
 #  endif
 #endif
 
Here is the about box of my VS:

Microsoft Visual Studio 2008
Version 9.0.21022.8 RTM
Microsoft .NET Framework
Version 3.5

Installed Edition: Professional

...

Microsoft Visual C++ 2008

Here is the version of the OS:

c:\mysql\bin>systeminfo

Host Name:                 MYSQL_LAPTOP
OS Name:                   Microsoft® Windows Vista™ Ultimate
OS Version:                6.0.6001 Service Pack 1 Build 6001
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          Chuck
Registered Organization:
Product ID:                89580-OEM-7332132-00141
Original Install Date:     3/17/2008, 12:47:03 PM
System Boot Time:          10/31/2008, 8:33:21 AM
System Manufacturer:       Dell Inc.
System Model:              XPS M1530
System Type:               X86-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: x64 Family 6 Model 23 Stepping 6 GenuineIntel ~
2501 Mhz
BIOS Version:              Dell Inc. A09, 7/14/2008
Windows Directory:         C:\Windows
System Directory:          C:\Windows\system32
Boot Device:               \Device\HarddiskVolume3
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (GMT-05:00) Eastern Time (US & Canada)
Total Physical Memory:     3,581 MB
Available Physical Memory: 1,777 MB
Page File: Max Size:       7,347 MB
Page File: Available:      5,422 MB
Page File: In Use:         1,925 MB
Page File Location(s):     C:\pagefile.sys
...

Thank you for the feedback.

I tried with yesterday backup tree and could not repeat described errors. Please try in your environment again and inform us about result.

Yes, the problem still exists. Neither the mysql-6.0 nor any derivative (e.g. the mysql-6.0-backup tree) will compile on Windows Vista Ultimate 32-bit using Microsoft Visual Studio 2008 with the Microsoft Platform SDK for Windows Server 2003 R2 libraries installed.

The following diff fixes the problem (but I suspect it is incorrect):

=== modified file 'include/atomic/nolock.h'
--- include/atomic/nolock.h	2008-07-09 07:12:43 +0000
+++ include/atomic/nolock.h	2009-02-03 20:20:30 +0000
@@ -27,7 +27,9 @@
 #  elif __GNUC__
 #    include "x86-gcc.h"
 #  elif defined(_MSC_VER)
-#    include "generic-msvc.h"
+#    if (_MSC_VER < 1400)
+#      include "generic-msvc.h"
+#    endif
 #  endif
 #endif

Note that the problem is also present in the Falcon code. This patch (again, may not be correct) fixes the problem:

=== modified file 'storage/falcon/Interlock.h'
--- storage/falcon/Interlock.h	2008-12-11 12:35:08 +0000
+++ storage/falcon/Interlock.h	2009-02-03 20:22:06 +0000
@@ -46,7 +46,7 @@
   x64 on the other hand, always uses intrinsic version, even in debug build
 */
 #ifdef _M_IX86
-#if (_MSC_VER >= 1400)
+#if (_MSC_VER < 1400)
 #include <intrin.h>
 #else
 /* Visual Studio 2003 and earlier do not have prototypes for atomic intrinsics */

Hi,

I tried on Vista Ultimate with VS2008 Express Edition and I got the below error:

1>------ Build started: Project: GenServerSource, Configuration: Debug Win32 ------
1>Generating sql_yacc.h
1>bison.exe: option `--defines' doesn't allow an argument
1>Usage: bison.exe [-dhklntvyV] [-b file-prefix] [-o outfile] [-p name-prefix]
1>       [--debug] [--defines] [--fixed-output-files] [--no-lines]
1>       [--verbose] [--version] [--help] [--yacc]
1>       [--no-parser] [--token-table]
1>       [--file-prefix=prefix] [--name-prefix=prefix]
1>       [--output=outfile] grammar-file
1>Report bugs to bug-bison@gnu.org
1>Project : error PRJ0019: A tool returned an error code from "Generating sql_yacc.h"
1>Build log was saved at "file://c:\temp\mysql-6.0-backup\sql\GenServerSource.dir\Debug\BuildLog.htm"
1>GenServerSource - 1 error(s), 0 warning(s)
2>------ Build started: Project: mysqld, Configuration: Debug Win32 ------
2>Generating sql_yacc.h
2>bison.exe: option `--defines' doesn't allow an argument
2>Usage: bison.exe [-dhklntvyV] [-b file-prefix] [-o outfile] [-p name-prefix]
2>       [--debug] [--defines] [--fixed-output-files] [--no-lines]
2>       [--verbose] [--version] [--help] [--yacc]
2>       [--no-parser] [--token-table]
2>       [--file-prefix=prefix] [--name-prefix=prefix]
2>       [--output=outfile] grammar-file
2>Report bugs to bug-bison@gnu.org
2>Project : error PRJ0019: A tool returned an error code from "Generating sql_yacc.h"
2>Build log was saved at "file://c:\temp\mysql-6.0-backup\sql\mysqld.dir\Debug\BuildLog.htm"
2>mysqld - 1 error(s), 0 warning(s)
========== Build: 0 succeeded, 2 failed, 66 up-to-date, 0 skipped ==========

Looks different than yours.

That error comes from either not having bison installed or not having bison in the path (as well as perhaps other configuration issues).

I don't have that problem when Compiling with VS2005 Pro only with VS2008 Express Edition. Below the bison and cmake version installed.

Setting environment for using Microsoft Visual Studio 2005 x64 tools.

c:\dbs>bison --version
GNU Bison version 1.28

c:\dbs>cmake --version
cmake version 2.6-patch 2

c:\dbs>

I just built a new machine (actually, rebuilt from bare CD install of OS) using Microsoft Windows Vista Ultimate 32-bit, Visual Studio 2008 Professional + SP1, and Microsoft PSDK for Windows 2003 R2. The same build problem occurs. I must apply the following patch to get the code to compile.

=== modified file 'include/atomic/generic-msvc.h'
--- include/atomic/generic-msvc.h	2008-10-20 09:16:47 +0000
+++ include/atomic/generic-msvc.h	2009-02-03 20:19:11 +0000
@@ -31,7 +31,7 @@
 #ifdef _M_IX86
 
 #if (_MSC_VER >= 1500)
-#include <intrin.h>
+//#include <intrin.h>
 #else
 C_MODE_START
 /*Visual Studio 2003 and earlier do not have prototypes for atomic intrinsics*/

=== modified file 'include/atomic/nolock.h'
--- include/atomic/nolock.h	2008-07-09 07:12:43 +0000
+++ include/atomic/nolock.h	2009-02-03 20:20:30 +0000
@@ -27,7 +27,9 @@
 #  elif __GNUC__
 #    include "x86-gcc.h"
 #  elif defined(_MSC_VER)
-#    include "generic-msvc.h"
+#    if (_MSC_VER < 1400)
+#      include "generic-msvc.h"
+#    endif
 #  endif
 #endif
 

=== modified file 'storage/falcon/Interlock.h'
--- storage/falcon/Interlock.h	2008-12-11 12:35:08 +0000
+++ storage/falcon/Interlock.h	2009-02-03 20:22:06 +0000
@@ -46,7 +46,7 @@
   x64 on the other hand, always uses intrinsic version, even in debug build
 */
 #ifdef _M_IX86
-#if (_MSC_VER >= 1400)
+#if (_MSC_VER < 1400)
 #include <intrin.h>
 #else
 /* Visual Studio 2003 and earlier do not have prototypes for atomic intrinsics */

Aha! I see the problem.

The problem is due to the fact that I have links to the PSDK in my C++ directories config file.

This got there because I used to have VS Express and upgraded to VS Pro. Now that I've removed those links from the config file (and my restore scripts), the code compiles correctly without modification.

Problem solved.