| Bug #42969 | Please add a MANIFEST to each build | ||
|---|---|---|---|
| Submitted: | 18 Feb 2009 13:33 | Modified: | 14 Apr 2011 13:45 |
| Reporter: | Philip Stoev | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Packaging | Severity: | S3 (Non-critical) |
| Version: | 5.1,6.0 | OS: | Any |
| Assigned to: | Joerg Bruehe | CPU Architecture: | Any |
[31 Aug 2010 9:11]
Guilhem Bichot
Yes, having the output of "bzr version-info", or at least "bzr revision-info", in the MANIFEST would be useful. Use case: some QA dev downloads a non-debug snapshot of the server (from pb2 or other places), finds a bug; the server dev needs to know the bzr revision id of this snapshot in order to recreate the corresponding source tree (which she/he can then compile with debugging flags). The bug reporter can also be a community person who downloaded a nightly snapshot from somewhere.
[2 Nov 2010 10:16]
Axel Schwenke
there is related bug #56864 asking to put cmake build information into mysqlbug
[2 Nov 2010 10:57]
Jonathan Perkin
Joerg, we recently discussed including CMakeCache.txt which will provide most of this information, could you take a look? Thanks.
[9 Nov 2010 19:23]
Konstantin Osipov
mysqlbug was removed in 5.5. Thus, Davi's plea should not affect 5.1 or 5.0. Suggest to have a small script or just a text file containing the necessary information.
[2 Dec 2010 16:14]
Sveta Smirnova
Bug #58661 was marked as duplicate of this one.
[21 Dec 2010 10:11]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127367 3209 Joerg Bruehe 2010-12-21 Fix bug#42969 Please add a MANIFEST to each build This is work-in-progress, not yet ready for review: The current changeset is the first step only, but it must be committed in order to proceed ("bzr export" is needed). Step 1: Create "MANIFEST_source" during "make dist". @ cmake/make_dist.cmake.in Use the new "manifest.cmake" file during "make dist". @ cmake/manifest.cmake.in New file. It will contain macros to handle the "MANIFEST*" files during various steps of the build, the calls will be at other places. Step 1: Macro "CREATE_MANIFEST_SOURCE" only. @ sql/CMakeLists.txt Handle the (new) "cmake/manifest.cmake.in" file by running CONFIGURE_FILE() on it.
[21 Dec 2010 10:19]
Joerg Bruehe
As written in the changeset comment, this first patch is still incomplete - but I cannot proceed without committing it (the new file must become part of "bzr export"). There is no chance to use the same implementation in the autotools-besed 5.1 build and the cmake-based 5.5 build, so the target for this change is 5.5 and up.
[21 Jan 2011 20:41]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/129366 3209 Joerg Bruehe 2011-01-21 Fix bug#42969 Please add a MANIFEST to each build With this change, there will be new files "VERSION_src" and "VERSION_bin", which describe the source and the binaries. They will be contained in all packages. "VERSION_src" is also part of a source tarball. It gives the version as exact as possible, preferably by calling "bzr versioninfo" on the source tree. If that is not possible, it just contains the three level version number. "VERSION_bin" contains some info when and where the binaries were built, the options given to the compiler, and the flags controlling the included features. @ CMakeLists.txt For the new files describing the source and the build ("VERSION_src" and "VERSION_bin"), we need a new file "cmake/version_files.cmake.in" with the build rules. 1) This file must be configured with the current variables. 2) "VERSION_src" can be created during the cmake phase. 3) "VERSION_bin" must be created during the make phase only, because it contains information from files which will be written at the end of the cmake phase only. Therefore, it must be a custom target which is included in all "make" targets. 4) The resulting version files must be included in packages. @ cmake/make_dist.cmake.in Create a "VERSION_src" file during "make dist". In case it already exists from a preceding "cmake" run (which is quite likely), a new "make dist" must not modify it. @ cmake/version_files.cmake.in New file. It will contain macros to create the "VERSION_*" files during various steps of the build, the calls will be at other places. 1) For source: If running from a BZR tree, always create (update) "VERSION_src" by running "bzr version-info". Outside a BZR tree, create it only if missing, and put the three level version number into it. 2) "VERSION_bin" contains - date/time and host name of the build host, - information about the platform, if cmake provides it (2.6.0 does not, the manual is wrong), - information about the C and CXX compiler and the options given to them (these may vary by directory, take those of the "sql/" subdirectory), - the "WITH*" options (ON or OFF) from "CMakeCache.txt" (taking them from there rather than using "CONFIGURE_FILE()" avoids maintenance when features get added or removed). Several actions are more complicated than is nice, this was needed to make it work using cmake 2.6.0. When all builds move to newer versions, there will hopefully be room for improvement. @ support-files/mysql.spec.sh Add "VERSION_src" and "VERSION_bin" to the RPM contents.
[24 Jan 2011 11:32]
Jonathan Perkin
I don't like the names, they seem counter-intuitive and clash somewhat with the new VERSION file. How about BZR_INFO and BUILD_INFO?
[31 Jan 2011 19:33]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/130080 3209 Joerg Bruehe 2011-01-31 Fix bug#42969 Please add a MANIFEST to each build With this change, there will be new files "INFO_SRC" and "INFO_BIN", which describe the source and the binaries. They will be contained in all packages: - in "tar.gz" and derived packages, in "docs/", - in RPMs, in "/usr/share/doc/packages/MySQL-server". "INFO_SRC" is also part of a source tarball. It gives the version as exact as possible, preferably by calling "bzr version-info" on the source tree. If that is not possible, it just contains the three level version number. "INFO_BIN" contains some info when and where the binaries were built, the options given to the compiler, and the flags controlling the included features. (At the time of this writing, January 2011, cmake 2.6.4 does not provide some of this desired information in the extent promised by its documentation.) @ CMakeLists.txt For the new files describing the source and the build ("INFO_SRC" and "INFO_BIN"), we need a new file "cmake/info_macros.cmake.in" with the build rules. 1) This file must be configured with the current variables. 2) "INFO_SRC" can be created during the cmake phase. 3) "INFO_BIN" must be created during the make phase only, because it contains information from files which will be written at the end of the cmake phase only. Therefore, it must be a custom target which is included in all "make" targets. 4) The resulting version files must be included in packages. @ cmake/info_bin.cmake This is the file to create "INFO_BIN", by calling the "CREATE_INFO_BIN()" macro. It must be a separate file, so that the macro definitions can be included in other cmake scripts without that file inclusion causing a side effect, the macro call. That call would modify the source tree which should be trated read-only. @ cmake/info_macros.cmake.in This new file contains the macros to create the "INFO_*" files during various steps of the build, the calls will be at other places. 1) For source: If running from a BZR tree, always create (update) "INFO_SRC" by running "bzr version-info". Outside a BZR tree, try to take it from exported sources, and create it only if missing, in that case put the three level version number into it. 2) "INFO_BIN" contains (if provided) - date/time and host name of the build host, - information about the platform, (2.6.4 does not provide it, the manual is wrong), - information about the C and CXX compiler and the options given to them (Unix only), - the feature flags as reported by "cmake -L". @ cmake/make_dist.cmake.in Create a "VERSION_src" file during "make dist". In case it already exists from a preceding "cmake" run or tree export (which is quite likely), a new "make dist" must not modify it. @ support-files/mysql.spec.sh Add "INFO_SRC" and "INFO_BIN" to the RPM contents.
[1 Feb 2011 13:53]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/130127 3211 Joerg Bruehe 2011-02-01 Bug#42969 Please add a MANIFEST to each build Additional fix: cmake variables must be taken using the '@var@' notation and then be passed through "configure_file()", this was missing for the cmake_[host_]system*" ones.
[7 Feb 2011 12:24]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/130565 3212 Joerg Bruehe 2011-02-07 Bug #42969 Please add a MANIFEST to each build First attempt at a test that will verify the existence of both "INFO_SRC" and "INFO:_BIN", as well as (some of) their expected (invariant) contents. However, this may still need some directory name polishing when it is checked on all of our supported platforms. @ mysql-test/t/mysql.test The patterns searched must match the file creation in boh local developer and release builds. Path name construction may need changes.
[11 Feb 2011 11:52]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/131120 3209 Joerg Bruehe 2011-02-11 Fix bug#42969 Please add a MANIFEST to each build With this change, there will be new files "INFO_SRC" and "INFO_BIN", which describe the source and the binaries. They will be contained in all packages: - in "tar.gz" and derived packages, in "docs/", - in RPMs, in "/usr/share/doc/packages/MySQL-server". "INFO_SRC" is also part of a source tarball. It gives the version as exact as possible, preferably by calling "bzr version-info" on the source tree. If that is not possible, it just contains the three level version number. "INFO_BIN" contains some info when and where the binaries were built, the options given to the compiler, and the flags controlling the included features. The tests (test "mysql" in the main suite) are extended to verify the existence of both "INFO_SRC" and "INFO_BIN", as well as some of the expected contents. @ CMakeLists.txt For the new files describing the source and the build ("INFO_SRC" and "INFO_BIN"), we need a new file "cmake/info_macros.cmake.in" with the build rules. 1) This file must be configured with the current variables. 2) "INFO_SRC" can be created during the cmake phase, but this should be repeated with each "make" to protect against a developer doing only "make" after a "bzr pull" (or "bzr commit"). So have it both as a cmake rule and as a custom target. 3) "INFO_BIN" must be created during the make phase only, because it contains information from files which will be written at the end of the cmake phase only. Therefore, it must be a custom target which is included in all "make" targets. 4) The resulting "INFO_*" files must be included in packages. @ cmake/info_bin.cmake This is the file to create "INFO_BIN", by calling the "CREATE_INFO_BIN()" macro. It must be a separate file, so that the macro definitions can be included in other cmake scripts without that file inclusion causing a side effect, the macro call. That call would modify the source tree which should be trated read-only. @ cmake/info_macros.cmake.in This new file contains the macros to create the "INFO_*" files during various steps of the build, the calls will be at other places. 1) For source: If running from a BZR tree, always create (update) "INFO_SRC" by running "bzr version-info". Outside a BZR tree, try to take it from exported sources, and create it only if missing, in that case put the three level version number into it. 2) "INFO_BIN" contains - date/time and host name of the build host, - information about the platform, - information about the C and CXX compiler and the options given to them (Unix only), - the feature flags as reported by "cmake -L". @ cmake/info_src.cmake This is the file to create "INFO_SRC", by calling the "CREATE_INFO_SRC()" macro. It must be a separate file, so that the macro definitions can be included in other cmake scripts without that file inclusion causing a side effect, the macro call. That call would modify the source tree which should be trated read-only. @ cmake/make_dist.cmake.in Create a "VERSION_src" file during "make dist". In case it already exists from a preceding "cmake" run or tree export (which is quite likely), a new "make dist" must not modify it. @ mysql-test/r/mysql.result Add the expected result of checking the "INFO_*" files. @ mysql-test/t/mysql.test Check the "INFO_*" files. The path name to find them is not directly provided by MTR, so it is constructed by modifying "libdir". The names created must match the file placement in both local developer and release builds. tar.gz and RPM have a different storage layout, then even RedHat and SuSE differ, and development in BZR is yet another case. Allow for the different "INFO_BIN" contents on Unix vs Windows regarding the compiler. @ support-files/mysql.spec.sh Add "INFO_SRC" and "INFO_BIN" to the RPM contents.
[5 Apr 2011 12:39]
Joerg Bruehe
This has been approved (marked now), and pushed to the main trees. The fix takes effect for versions 5.5.11, 5.6.2, and their successors: All packages will contain both an "INFO_SRC" and an "INFO_BIN" file which provide the desired information. The placement varies by package format, they are put in the same directory as other documentation files.
[14 Apr 2011 13:45]
Paul DuBois
Noted in 5.5.11, 5.6.2 changelogs. MySQL distributions now include an INFO_SRC file that contains information about the source distribution, such as the MySQL version from which it was created. MySQL binary distributions additionally include an INFO_BIN file that contains information about how the distribution was built, such as compiler options and feature flags. In RPM packages, these files are located in the /usr/share/doc/packages/MySQL-server directory. In tar.gz and derived packages, they are located in the Docs directory under the location where the distribution is unpacked. CHANGESET - http://lists.mysql.com/commits/131120
Description: It would be really nice to have a MANIFEST file in each build (regardless of the method the build was produced) describing giving information on how the binary was built, similar to perl -V , however without actually hacking mysqld (a separate bug would be filed for that). Currently, mysqlbug does provide some information, however it is sometimes confusing, for example because both the GCC version that produced mysqld and the GCC version installed on the local machine are given (without indication which is which). How to repeat: Download a mysqld package from PB2 and then try to figure where it came from and how it was compiled... Suggested fix: Below is an example of the output provided by perl -V . In addition to that, we would need the output provided by bzr version-info and the like. Summary of my perl5 (revision 5 version 10 subversion 0) configuration: Platform: osname=linux, osvers=2.6.18-92.1.10.el5, archname=x86_64-linux-thread-multi uname='linux x86-7.fedora.phx.redhat.com 2.6.18-92.1.10.el5 #1 smp wed jul 2 3 03:56:11 edt 2008 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fex ceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -DPERL_ USE_SAFE_PUTENV -Dversion=5.10.0 -Dmyhostname=localhost -Dperladmin=root@localho st -Dcc=gcc -std=gnu99 -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr - Dsiteprefix=/usr/local -Dprivlib=/usr/lib/perl5/5.10.0 -Dsitelib=/usr/local/lib/ perl5/site_perl/5.10.0 -Dvendorlib=/usr/lib/perl5/vendor_perl/5.10.0 -Darchlib=/ usr/lib64/perl5/5.10.0/x86_64-linux-thread-multi -Dsitearch=/usr/local/lib64/per l5/site_perl/5.10.0/x86_64-linux-thread-multi -Dvendorarch=/usr/lib64/perl5/vend or_perl/5.10.0/x86_64-linux-thread-multi -Dinc_version_list=none -Darchname=x86_ 64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib - Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui _ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinpe rl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_p roto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_s etprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/us r/bin -Dotherlibdirs=/usr/local/lib/perl5/site_perl:/usr/local/lib64/perl5/site_ perl' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc -std=gnu99', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-s trict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BIT S=64 -I/usr/include/gdbm', optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-pr otector --param=ssp-buffer-size=4 -m64 -mtune=generic -DPERL_USE_SAFE_PUTENV', cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='4.3.2 20081105 (Red Hat 4.3.2-7)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize =8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc -std=gnu99', ldflags ='' libpth=/usr/local/lib64 /lib64 /usr/lib64 libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.9' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath, /usr/lib64/perl5/5.10.0/x86_64-linux-thread-multi/CORE' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOU RCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=gener ic -DPERL_USE_SAFE_PUTENV' Characteristics of this binary (from libperl): Compile-time options: DEBUGGING MULTIPLICITY PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_TRACK_MEMPOOL PERL_USE_SAFE_PUTENV USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Built under linux Compiled at Nov 28 2008 19:15:11 @INC: /usr/lib64/perl5/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/5.10.0 /usr/local/lib64/perl5/site_perl/5.10.0/x86_64-linux-thread-multi /usr/local/lib/perl5/site_perl/5.10.0 /usr/lib64/perl5/vendor_perl/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5/vendor_perl /usr/local/lib/perl5/site_perl/5.10.0 /usr/local/lib/perl5/site_perl /usr/local/lib64/perl5/site_perl/5.10.0/x86_64-linux-thread-multi /usr/local/lib64/perl5/site_perl/5.10.0 /usr/local/lib64/perl5/site_perl .