Bug #23427 incompatible ABI change in 5.0.26?
Submitted: 18 Oct 2006 17:10 Modified: 7 Nov 2006 20:32
Reporter: sean finney
Status: Closed
Category:Server Severity:S1 (Critical)
Version:5.0.26 OS:Linux (debian gnu/linux)
Assigned to: Magnus Blaudd Target Version:

[18 Oct 2006 17:10] sean finney
Description:
feel free to downgrade the severity and/or plonk me with a cluebat if i'm off on this, but
it seems that in 5.0.26 some incompatible API/ABI changes have been made.

for example, the st_mysql struct now has a new member current_stmt which
it did not have before, and the interface for some of the functions seems to have
changed.

if this is in fact the case, this will cause some really unpleasant symptoms, such as
random segfaults in programs linked against the shared library.

i'm hoping that this is a false alarm, but since this has happened in the past (5.0.24),
i'm suspecting it may have happened again.  i can provide the full output of the ABI
checking that i did if it would be helpful.

How to repeat:
install the latest version, and compare the header files to the previous version with a
tool like "icheck".  for example:

icheck --canonify -o foobar_old.ic /usr/include/mysql/mysql.h

upgrade to new version

icheck --canonify -o foobar_new.ic /usr/include/mysql/mysql.h

icheck --compare foobar_old.ic foobar_new.ic

Suggested fix:
ideally, back out the changes.

not-so-ideally, increment the SONAME for the shared library.
[19 Oct 2006 20:27] Chad MILLER
current_stmt was added in early June*, between releases for 5.0.23 and 5.0.24, as part of
a fix for Bug#16017.  We indeed should be very careful about breaking the ABI.
[19 Oct 2006 21:22] Sveta Smirnova
Thank you for the report.

Verified as described.

There is API change in MySQL 5.0.26 haven't been announced.
[20 Oct 2006 15:10] Magnus Blaudd
http://lists.mysql.com/commits/14068
[20 Oct 2006 16:41] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14075

ChangeSet@1.2593, 2006-10-20 16:41:03+02:00, msvensson@neptunus.(none) +5 -0
  WL#3462 Add testing of MySQL client ABI to prevent unintentional ABI breaks
  BUG#23427 incompatible ABI change in 5.0.26?
     - Use the icheck tool if avaliable and compare the current mysql.h to a version
  controlled reference file
[20 Oct 2006 23:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14100

ChangeSet@1.2292, 2006-10-20 17:17:24-04:00, cmiller@zippy.cornsilk.net +2 -0
  Bug #23427: incompatible ABI change in 5.0.26?
  
  Revert 1 June change enough to restore ABI compatibility with previous
  versions.
[21 Oct 2006 13:05] Olaf van der Spek
Why is this structure public at all?
I think you'd avoid lots of problems by making it private.
[24 Oct 2006 13:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14249

ChangeSet@1.2535, 2006-10-24 17:19:02+05:00, holyfoot@mysql.com +3 -0
  Bug #23427 (incompatible ABI change)
  
  the incompatibility was caused by current_stmt member added to the MYSQL
  structure.
  It's possible to move it to THD structure instead which saves ABI
[24 Oct 2006 20:32] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14296

ChangeSet@1.2593, 2006-10-24 20:32:07+02:00, msvensson@neptunus.(none) +5 -0
  WL#3462 Add testing of MySQL client ABI to prevent unintentional ABI breaks
  BUG#23427 incompatible ABI change in 5.0.26?
     - Use the icheck tool if avaliable and compare the current mysql.h to a version
  controlled reference file
[25 Oct 2006 12:32] Magnus Blaudd
The icheck patch is no good.
1. It doesn't fail the build when check fails. This worked before. 
2. The .ic file generated on my machine fails to compare on sapsrv1. Tested this before
and at that time it worked. Strange.
3. "make dist" on melody fails.

:(
[25 Oct 2006 23:46] Christian Hammers
Hi

Applying the patch http://lists.mysql.com/commits/14100 to 5.0 removes on ABI breakage but
icheck still produces some more warnings between the 5.0.24a and 5.0.26+patch version of
mysql.h.

Please tell me if this needs patching, too.

bye,

-christian- / Debian

ABI and API mismatch between:
  my_bool (MYSQL * mysql, enum enum_server_command, char const * header, unsigned long
int, char const * arg, unsigned long int, my_bool, MYSQL_STMT * stmt)
and:
  my_bool (MYSQL * mysql, enum enum_server_command, char const * header, unsigned long
int, char const * arg, unsigned long int, my_bool)
 in declaration:
my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const * header,
unsigned long int, char const * arg, unsigned long int, my_bool, M
YSQL_STMT * stmt);
 versus declaration:
my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const * header,
unsigned long int, char const * arg, unsigned long int, my_bool);
 in type defined at /usr/include/mysql/mysql.h:739:
struct __attribute__((aligned(__alignof__(void *)))) st_mysql_methods
  {
    my_bool (* read_query_result)(MYSQL * mysql);
    my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const *
header, unsigned long int, char const * arg, unsigned long int, my_boo
l, MYSQL_STMT * stmt);
    MYSQL_DATA * (* read_rows)(MYSQL * mysql, MYSQL_FIELD * mysql_fields, unsigned int);
    MYSQL_RES * (* use_result)(MYSQL * mysql);
    void (* fetch_lengths)(unsigned long int * to, MYSQL_ROW, unsigned int);
    void (* flush_use_result)(MYSQL * mysql);
    MYSQL_FIELD * (* list_fields)(MYSQL * mysql);
    my_bool (* read_prepare_result)(MYSQL * mysql, MYSQL_STMT * stmt);
    int (* stmt_execute)(MYSQL_STMT * stmt);
    int (* read_binary_rows)(MYSQL_STMT * stmt);
    int (* unbuffered_fetch)(MYSQL * mysql, char * * row);
    void (* free_embedded_thd)(MYSQL * mysql);
    char const * (* read_statistics)(MYSQL * mysql);
    my_bool (* next_result)(MYSQL * mysql);
    int (* read_change_user_result)(MYSQL * mysql, char * buff, char const * passwd);
    int (* read_rows_from_cursor)(MYSQL_STMT * stmt);
  };
 versus type defined at /usr/include/mysql/mysql.h:738:
struct __attribute__((aligned(__alignof__(void *)))) st_mysql_methods
  {
    my_bool (* read_query_result)(MYSQL * mysql);
    my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const *
header, unsigned long int, char const * arg, unsigned long int, my_boo
l);
    MYSQL_DATA * (* read_rows)(MYSQL * mysql, MYSQL_FIELD * mysql_fields, unsigned int);
    MYSQL_RES * (* use_result)(MYSQL * mysql);
    void (* fetch_lengths)(unsigned long int * to, MYSQL_ROW, unsigned int);
    void (* flush_use_result)(MYSQL * mysql);
    MYSQL_FIELD * (* list_fields)(MYSQL * mysql);
    my_bool (* read_prepare_result)(MYSQL * mysql, MYSQL_STMT * stmt);
    int (* stmt_execute)(MYSQL_STMT * stmt);
    int (* read_binary_rows)(MYSQL_STMT * stmt);
    int (* unbuffered_fetch)(MYSQL * mysql, char * * row);
    void (* free_embedded_thd)(MYSQL * mysql);
    char const * (* read_statistics)(MYSQL * mysql);
    my_bool (* next_result)(MYSQL * mysql);
    int (* read_change_user_result)(MYSQL * mysql, char * buff, char const * passwd);
    int (* read_rows_from_cursor)(MYSQL_STMT * stmt);
  };

API and ABI removal: identifier net_safe_read is gone
 was defined at /usr/include/mysql/mysql.h:839:
extern unsigned long int net_safe_read(MYSQL * mysql);

ABI is not compatible
API is not compatible
[26 Oct 2006 12:55] Magnus Blaudd
Christian how did you run icheck?

I have pushed an "icheck patch" to some of our build machines and it's detecting
differences between machines. :(

Need some improvement, we should not include the machine dependent code in the icheck
analysis.
[26 Oct 2006 18:38] Christian Hammers
I run manually after applying *only* the http://lists.mysql.com/commits/14100
patch from Chad Miller:
 icheck --canonical mysql-5.0.24a/include/mysql.h > icheck-5.0.24a
 icheck --canonical mysql-5.0.26/include/mysql.h > icheck-5.0.26
 icheck --compare icheck-5.0.24a icheck-5.0.26

After unpacking a source you can either put a mysql_version.h manually in there or compile
the package.

I used amd64 as architecture - it can be that for different architectures the header file
looks a bit different, I have no experience with icheck and cannot
tell how a decent automatic Makefile check could be done.
(Maybe by deleting all #include statments via sed so that only the mysql 
structures are left?)

If reproducing with icheck does not work for you, just verify manually if the structures
in mysql.h that icheck complained about, are indeed different or not, that should be
fairly easy and we would know if there is an open ABI change 
problem, or not.

bye,

-christian-
[26 Oct 2006 21:05] Magnus Blaudd
Removal of 'net_safe_read' should be safe. Anyone using that is doing weird things.

The addition of MYSQL_STMT* to the "advanced_command" function pointer in struct
st_mysql_method is only used by embedded MySQL Server and should thus also be safe.

The plan is to reduce these publicly visible structs into "typedef void * MYSQL"  so the
implementation of "struct mysql" is hidden in future versions of MySQL.

Thanks for your help!
[26 Oct 2006 22:21] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14449

ChangeSet@1.2346, 2006-10-26 22:20:43+02:00, msvensson@neptunus.(none) +2 -0
  Bug#23427  	incompatible ABI change in 5.0.26?
   - Improve icheck rules in Makefile.am
[7 Nov 2006 20:32] Paul DuBois
Noted in 5.0.27 (CS), 5.0.28 (ES), 5.1.13 changelogs.

MySQL 5.0.26 introduced an ABI incompatibility, which this release
reverts. Programs compiled against 5.0.26 are not compatible with any
other version and must be recompiled.
[13 Jun 2007 11:03] Suphap Inta
Suphap