Bug #78945 cmake does NOT download boost 1.59 when older version of boost is present
Submitted: 23 Oct 2015 14:34 Modified: 27 Oct 2015 11:34
Reporter: Valeriy Kravchuk Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Server: Compiling Severity:S3 (Non-critical)
Version:5.7.9 OS:Linux
Assigned to: CPU Architecture:Any
Tags: cmake, DOWNLOAD_BOOST, missing manual, WITH_BOOST

[23 Oct 2015 14:34] Valeriy Kravchuk
Description:
Manual, https://dev.mysql.com/doc/refman/5.7/en/source-configuration-options.html#option_cmake_wit..., clearly says:

"If the required Boost version is already there, no download is done. If the required Boost version changes, the newer version is downloaded."

But download does NOT happen in reality when there is older version of Boost in the directory specified:

[openxs@centos mysql-server]$ ls -l /home/openxs/boost/
total 81632
drwx------. 8 openxs openxs     4096 Aug 20 16:38 boost_1_58_0
-rw-rw-r--. 1 openxs openxs 83581760 Aug 20 16:37 boost_1_58_0.tar.gz
[openxs@centos mysql-server]$ git branch
  5.6
* 5.7
[openxs@centos mysql-server]$ cmake . -DCMAKE_INSTALL_PREFIX=/home/openxs/dbs/5.7 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DWITH_EMBEDDED_SERVER=OFF -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/home/openxs/boost
-- Running cmake version 2.8.12.2
-- Configuring with MAX_INDEXES = 64U
-- SIZEOF_VOIDP 8
-- MySQL 5.7.9
-- Packaging as: mysql-5.7.9-Linux-x86_64
-- Local boost zip /home/openxs/boost/boost_1_58_0.tar.gz
-- cd /home/openxs/boost; tar xfz /home/openxs/boost/boost_1_58_0.tar.gz
-- Found /home/openxs/boost/boost_1_58_0/boost/version.hpp
-- BOOST_VERSION_NUMBER is #define BOOST_VERSION 105800
CMake Warning at cmake/boost.cmake:256 (MESSAGE):
  Boost minor version found is 58 we need 59
Call Stack (most recent call first):
  CMakeLists.txt:435 (INCLUDE)

-- BOOST_INCLUDE_DIR /home/openxs/boost/boost_1_58_0
-- LOCAL_BOOST_DIR /home/openxs/boost
-- LOCAL_BOOST_ZIP /home/openxs/boost/boost_1_58_0.tar.gz
-- Could not find (the correct version of) boost.
-- MySQL currently requires boost_1_59_0

CMake Error at cmake/boost.cmake:76 (MESSAGE):
  You can download it with -DDOWNLOAD_BOOST=1 -DWITH_BOOST=<directory>

  This CMake script will look for boost in <directory>.  If it is not there,
  it will download and unpack it (in that directory) for you.

  If you are inside a firewall, you may need to use an http proxy:

  export http_proxy=http://example.com:80

Call Stack (most recent call first):
  cmake/boost.cmake:259 (COULD_NOT_FIND_BOOST)
  CMakeLists.txt:435 (INCLUDE)

-- Configuring incomplete, errors occurred!
See also "/home/openxs/git/mysql-server/CMakeFiles/CMakeOutput.log".
See also "/home/openxs/git/mysql-server/CMakeFiles/CMakeError.log".

How to repeat:
Get recent source code of MySQL 5.7.9 from GitHub and try to build it referring to the directory with Boost that worked for the previous version, like this:

cmake . -DCMAKE_INSTALL_PREFIX=/home/openxs/dbs/5.7 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DWITH_EMBEDDED_SERVER=OFF -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/home/openxs/boost

Note -DDOWNLOAD_BOOST=1 is present, so I ask to download what is missing clearly. Manual says nothing about checking for previous version being not there etc, so I expect the cmake command to just work. Instead, it spends time unpacking the boost library again, to find out it's of older version and then do nothing.

Suggested fix:
Please, make sure that -DDOWNLOAD_BOOST=1 does whatever is needed to download proper version in the directory specified by -DWITH_BOOST, and use it.

While this is not fixed, please, document the following workaround of removing older version from the directory listed in -DWITH_BOOST before calling cmake:

[openxs@centos mysql-server]$ rm -rf /home/openxs/boost/*
[25 Oct 2015 3:23] MySQL Verification Team
I couldn't repeat on Windows:

-- SIZEOF_VOIDP 4
-- MySQL 5.7.9
-- Packaging as: mysql-5.7.9-win32
-- Downloading boost_1_59_0.tar.gz to C:/
-- [download 0% complete]
-- [download 1% complete]
-- [download 2% complete]
-- [download 3% complete]
-- [download 4% complete]
-- [download 5% complete]
-- [download 6% complete]
-- [download 7% complete]
-- [download 8% complete]
-- [download 9% complete]
-- [download 10% complete]
-- [download 11% complete]
-- [download 12% complete]
-- [download 13% complete]
-- [download 14% complete]
-- [download 15% complete]
-- [download 16% complete]
-- [download 17% complete]
-- [download 18% complete]
-- [download 19% complete]
-- [download 20% complete]
-- [download 21% complete]
-- [download 22% complete]
-- [download 23% complete]
-- [download 24% complete]
-- [download 25% complete]
-- [download 26% complete]
-- [download 27% complete]
-- [download 28% complete]
-- [download 29% complete]
-- [download 30% complete]
-- [download 31% complete]
-- [download 32% complete]
-- [download 33% complete]
-- [download 34% complete]
-- [download 35% complete]
-- [download 36% complete]
-- [download 37% complete]
-- [download 38% complete]
-- [download 39% complete]
-- [download 40% complete]
-- [download 41% complete]
-- [download 42% complete]
-- [download 43% complete]
-- [download 44% complete]
-- [download 45% complete]
-- [download 46% complete]
-- [download 47% complete]
-- [download 48% complete]
-- [download 49% complete]
-- [download 50% complete]
-- [download 51% complete]
-- [download 52% complete]
-- [download 53% complete]
-- [download 54% complete]
-- [download 55% complete]
-- [download 56% complete]
-- [download 57% complete]
-- [download 58% complete]
-- [download 59% complete]
-- [download 60% complete]
-- [download 61% complete]
-- [download 62% complete]
-- [download 63% complete]
-- [download 64% complete]
-- [download 65% complete]
-- [download 66% complete]
-- [download 67% complete]
-- [download 68% complete]
-- [download 69% complete]
-- [download 70% complete]
-- [download 71% complete]
-- [download 72% complete]
-- [download 73% complete]
-- [download 74% complete]
-- [download 75% complete]
-- [download 76% complete]
-- [download 77% complete]
-- [download 78% complete]
-- [download 79% complete]
-- [download 80% complete]
-- [download 81% complete]
-- [download 82% complete]
-- [download 83% complete]
-- [download 84% complete]
-- [download 85% complete]
-- [download 86% complete]
-- [download 87% complete]
-- [download 88% complete]
-- [download 89% complete]
-- [download 90% complete]
-- [download 91% complete]
-- [download 92% complete]
-- [download 93% complete]
-- [download 94% complete]
-- [download 95% complete]
-- [download 96% complete]
-- [download 97% complete]
-- [download 98% complete]
-- [download 99% complete]
-- [download 100% complete]
-- cd C:/; tar xfz C://boost_1_59_0.tar.gz
-- Found C:/boost_1_59_0/boost/version.hpp
-- BOOST_VERSION_NUMBER is #define BOOST_VERSION 105900
-- BOOST_INCLUDE_DIR C:/boost_1_59_0
[27 Oct 2015 8:21] Valeriy Kravchuk
I checked on several Ubuntu (14.04, 15.04) boxes and on old Fedora Core. As soon as I have older .tar.gz:

openxs@ao756:~/git/mysql-server$ ls -l /home/openxs/boost
total 81628
drwx------ 8 openxs openxs     4096 пІп╬п╡ 27 10:18 boost_1_58_0
-rw-rw-r-- 1 openxs openxs 83581760 я│п╣я─ 22 20:01 boost_1_58_0.tar.gz

this command fails:

openxs@ao756:~/git/mysql-server$ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DWITH_EMBEDDED_SERVER=OFF -DCMAKE_INSTALL_PREFIX=/home/openxs/dbs/5.7 -DWITH_BOOST=/home/openxs/boost -DDOWNLOAD_BOOST=1
-- Running cmake version 2.8.12.2
-- Configuring with MAX_INDEXES = 64U
-- SIZEOF_VOIDP 8
-- MySQL 5.7.9
-- Packaging as: mysql-5.7.9-Linux-x86_64
-- Local boost zip /home/openxs/boost/boost_1_58_0.tar.gz
-- cd /home/openxs/boost; tar xfz /home/openxs/boost/boost_1_58_0.tar.gz
-- Found /home/openxs/boost/boost_1_58_0/boost/version.hpp
-- BOOST_VERSION_NUMBER is #define BOOST_VERSION 105800
CMake Warning at cmake/boost.cmake:256 (MESSAGE):
  Boost minor version found is 58 we need 59
Call Stack (most recent call first):
  CMakeLists.txt:435 (INCLUDE)

-- BOOST_INCLUDE_DIR /home/openxs/boost/boost_1_58_0
-- LOCAL_BOOST_DIR /home/openxs/boost
-- LOCAL_BOOST_ZIP /home/openxs/boost/boost_1_58_0.tar.gz
-- Could not find (the correct version of) boost.
-- MySQL currently requires boost_1_59_0

CMake Error at cmake/boost.cmake:76 (MESSAGE):
  You can download it with -DDOWNLOAD_BOOST=1 -DWITH_BOOST=<directory>

  This CMake script will look for boost in <directory>.  If it is not there,
  it will download and unpack it (in that directory) for you.

  If you are inside a firewall, you may need to use an http proxy:

  export http_proxy=http://example.com:80

Call Stack (most recent call first):
  cmake/boost.cmake:259 (COULD_NOT_FIND_BOOST)
  CMakeLists.txt:435 (INCLUDE)

-- Configuring incomplete, errors occurred!
See also "/home/openxs/git/mysql-server/CMakeFiles/CMakeOutput.log".
See also "/home/openxs/git/mysql-server/CMakeFiles/CMakeError.log".
[27 Oct 2015 10:22] MySQL Verification Team
I couldn't repeat on VirtalBox Ubuntu 14.04 too:

miguel@vbub14:~/mysql-5.7.9$ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DWITH_EMBEDDED_SERVER=OFF -DCMAKE_INSTALL_PREFIX=/home/miguel/dbs/5.7 -DWITH_BOOST=/home/miguel/boost -DDOWNLOAD_BOOST=1
-- Running cmake version 2.8.12.2
-- Could NOT find Git (missing:  GIT_EXECUTABLE) 
-- Configuring with MAX_INDEXES = 64U
-- The C compiler identification is GNU 4.8.4
<CUT>                                                                                                                  
-- MySQL 5.7.9                                                                                                                                                          
-- Packaging as: mysql-5.7.9-Linux-x86_64                                                                                                                                            
-- Downloading boost_1_59_0.tar.gz to /home/miguel/boost                                                                                                                             
-- [download 0% complete]                                                                                                                                                                    
-- [download 1% complete]                                                                                                                                                                        
-- [download 2% complete]                                                                                                                                                                        
-- [download 3% complete]                                                                                                                                                                              
-- [download 4% complete]                                                                                                                                                                              
-- [download 5% complete]
-- [download 6% complete]
-- [download 7% complete]
-- [download 8% complete]
-- [download 9% complete]
-- [download 10% complete]
-- [download 11% complete]
-- [download 12% complete]
-- [download 13% complete]
-- [download 14% complete]
-- [download 15% complete]
-- [download 16% complete]
-- [download 17% complete]
-- [download 18% complete]
-- [download 19% complete]
-- [download 20% complete]
-- [download 21% complete]
-- [download 22% complete]
-- [download 23% complete]
-- [download 24% complete]
-- [download 25% complete]
-- [download 26% complete]
-- [download 27% complete]
-- [download 28% complete]
-- [download 29% complete]
-- [download 30% complete]
-- [download 31% complete]
<cut>
-- [download 97% complete]
-- [download 98% complete]
-- [download 99% complete]
-- [download 100% complete]
-- cd /home/miguel/boost; tar xfz /home/miguel/boost/boost_1_59_0.tar.gz
-- Found /home/miguel/boost/boost_1_59_0/boost/version.hpp 
-- BOOST_VERSION_NUMBER is #define BOOST_VERSION 105900
-- BOOST_INCLUDE_DIR /home/miguel/boost/boost_1_59_0
-- Performing Test HAVE_LLVM_LIBCPP
-- Performing Test HAVE_LLVM_LIBCPP - Failed
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Looking for floor
-- Looking for floor - not found
-- Looking for floor in m
-- Looking for floor in m - found
-- Looking for gethostbyname_r
-- Looking for gethostbyname_r - found
-- Looking for bind
-- Looking for bind - found
-- Looking for crypt
-- Looking for crypt - not found
-- Looking for crypt in crypt
-- Looking for crypt in crypt - found
-- Looking for setsockopt
-- Looking for setsockopt - found
-- Looking for dlopen
-- Looking for dlopen - not found
-- Looking for dlopen in dl
-- Looking for dlopen in dl - found
-- Looking for sched_yield
-- Looking for sched_yield - found
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Looking for timer_create
-- Looking for timer_create - not found
-- Looking for timer_create in rt
-- Looking for timer_create in rt - found
-- Looking for include file alloca.h
-- Looking for include file alloca.h - found
-- Looking for include file arpa/inet.h
-- Looking for include file arpa/inet.h - found
-- Looking for include file dirent.h
-- Looking for include file dirent.h - found
-- Looking for include file dlfcn.h
-- Looking for include file dlfcn.h - found
-- Looking for include file execinfo.h
-- Looking for include file execinfo.h - found
-- Looking for include file fpu_control.h
-- Looking for include file fpu_control.h - found
-- Looking for include file grp.h
-- Looking for include file grp.h - found
-- Looking for include file ieeefp.h
-- Looking for include file ieeefp.h - not found
-- Looking for include file langinfo.h
-- Looking for include file langinfo.h - found
-- Looking for include file malloc.h
-- Looking for include file malloc.h - found
-- Looking for include file netinet/in.h
-- Looking for include file netinet/in.h - found
-- Looking for include file poll.h
-- Looking for include file poll.h - found
-- Looking for include file pwd.h
-- Looking for include file pwd.h - found
-- Looking for include file strings.h
<cut>
-- Looking for getegid
-- Looking for getegid - found
-- Looking for lstat
-- Looking for lstat - found
-- Looking for madvise
<CUT>
-- CMAKE_CXX_FLAGS_RELWITHDEBINFO: -O3 -g -fabi-version=2 -fno-omit-frame-pointer -fno-strict-aliasing -DDBUG_OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /home/miguel/mysql-5.7.9
miguel@vbub14:~/mysql-5.7.9$ ls /home/miguel/boost/
boost_1_57_0  boost_1_59_0  boost_1_59_0.tar.gz
miguel@vbub14:~/mysql-5.7.9$
[27 Oct 2015 10:23] MySQL Verification Team
Umesh,
Please check you too. Thanks.
[27 Oct 2015 11:20] Valeriy Kravchuk
tSo, the reason for the problem was CMakeCache.txt from older build of 5.7.8. I used to remove it only when cmake option changes, but it seems it must be cleaned up when version is upgraded. I hope this is documented somewhere (http://dev.mysql.com/doc/mysql-sourcebuild-excerpt/5.7/en/compilation-problems.html is more or less about this, but not exactly).

I'd still prefer cmake to care about Boost version change properly, no matter what cached CMakeCache.txt says, but it's up to you to decide on bug status.
[27 Oct 2015 11:34] MySQL Verification Team
Hello Valeriy,

Thank you for confirming.
Yes, it is always good to remove CMakeCache.txt when build fails(when again invoke cmake) and as you observed when CMakeCache.txt exists from older build etc

Thanks,
Umesh