Bug #71009 WORDS_BIGENDIAN not defined correctly for bigendian builds
Submitted: 26 Nov 2013 12:10 Modified: 5 Dec 2013 13:31
Reporter: Patrick Middleton Email Updates:
Status: Unsupported Impact on me:
None 
Category:MySQL Server: Compiling Severity:S2 (Serious)
Version:5.6.14 OS:MacOS (10.4 ppc+i386)
Assigned to: CPU Architecture:Any

[26 Nov 2013 12:10] Patrick Middleton
Description:
Building for MacOSX, cross-compiling for i386 and ppc using gcc4, bundled YASSL needs WORDS_BIGENDIAN to be defined for bigendian ppc.

config.h.cmake and Cmake 2.8-12 gets this wrong.
Code emitted into config.h/myconfig.h is this:

#ifdef __APPLE__
  #if __BIG_ENDIAN
    #define WORDS_BIGENDIAN 1
  #endif
#else
/* #undef WORDS_BIGENDIAN */
#endif

but the gcc toolchain predefines
_BIG_ENDIAN 1
__BIG_ENDIAN__ 1
or
_LITTLE_ENDIAN 1
__LITTLE_ENDIAN__ 1
as appropriate.

extra/yassl/taocrypt/include/types.hpp has this:

#if defined(WORDS_BIGENDIAN) || (defined(__MWERKS__) && !defined(__INTEL__))
    #define BIG_ENDIAN_ORDER
#endif

#ifndef BIG_ENDIAN_ORDER
    #define LITTLE_ENDIAN_ORDER
#endif

How to repeat:
Building 5.6.14GA on MacOSX  10.6.8 intel, cross compiling for 10.4u i386+ppc
and on MacOSX 10.4.11 ppc, cross compiling for 10.4u i386+ppc

rm -rf bld
mkdir bld
cd bld
cmake .. \
  -DWITH_PARTITION_STORAGE_ENGINE=OFF \
  -DCMAKE_OSX_ARCHITECTURES="ppc;i386" \
  -DCMAKE_OSX_SYSROOT="/Developer/SDKs/MacOSX10.4u.sdk" \
  -DCMAKE_OSX_DEPLOYMENT_TARGET="10.4" \
  -DCURSES_LIBRARY="/Developer/SDKs/MacOSX10.4u.sdk/usr/lib/libcurses.dylib" \
  -DCURSES_NCURSES_LIBRARY="/Developer/SDKs/MacOSX10.4u.sdk/usr/lib/libncurses.dylib" \
  -DCURSES_FORM_LIBRARY="/Developer/SDKs/MacOSX10.4u.sdk/usr/lib/libform.dylib"
  make install
cd ..

client/mysql built on i386 or ppc, for i386, runs and connects on i386
client/mysql built on i386 or ppc, for ppc, runs but does not connect on ppc or i386+rosetta

Suggested fix:
Patch against src for 5.6.14GA:

--- config.h.cmake.orig	2013-11-25 15:17:37.000000000 +0000
+++ config.h.cmake	2013-11-26 11:56:32.000000000 +0000
@@ -664,3 +664,12 @@
 #cmakedefine CPU_LEVEL1_DCACHE_LINESIZE @CPU_LEVEL1_DCACHE_LINESIZE@
 
 #endif
+
+#ifdef __GNUC__
+  #ifdef WORDS_BIGENDIAN
+    #undef WORDS_BIGENDIAN
+  #endif
+  #ifdef __BIG_ENDIAN__
+    #define WORDS_BIGENDIAN
+  #endif
+#endif

Although: 
for 5.7.2-m12, the equivalent section in config.h.cmake is this:

#cmakedefine WORDS_BIGENDIAN 1 

which I think will still be wrong for cross-compilation.
[26 Nov 2013 12:12] Patrick Middleton
Patch for config.h.cmake 5.6.14GA

Attachment: config.h.cmake.patch (application/octet-stream, text), 371 bytes.

[29 Nov 2013 18:01] MySQL Verification Team
Sorry, but as of 5.6, we do no longer support universal binaries for OS X. So, we do no longer support cross-compiling ...
[2 Dec 2013 17:24] Patrick Middleton
Ummmm.... I found this while cross-compiling.
What I found was that big-endian builds Do Not Work, and this is not to do with cross-compilation.
This is to do with WORDS_BIGENDIAN not being defined for bigendian builds, which leads to YASSL not building correctly. 
Try building natively on ppc, then use the mysql client you built to connect to a little-endian MySQL server.