MySQL Bugs: #23427: incompatible ABI change in 5.0.26?

Bug #23427	incompatible ABI change in 5.0.26?
Submitted:	18 Oct 2006 15:10	Modified:	7 Nov 2006 19:32
Reporter:	sean finney	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S1 (Critical)
Version:	5.0.26	OS:	Linux (debian gnu/linux)
Assigned to:	Magnus Blåudd	CPU Architecture:	Any

Description:
feel free to downgrade the severity and/or plonk me with a cluebat if i'm off on this, but it seems that in 5.0.26 some incompatible API/ABI changes have been made.

for example, the st_mysql struct now has a new member current_stmt which
it did not have before, and the interface for some of the functions seems to have changed.

if this is in fact the case, this will cause some really unpleasant symptoms, such as random segfaults in programs linked against the shared library.

i'm hoping that this is a false alarm, but since this has happened in the past (5.0.24), i'm suspecting it may have happened again.  i can provide the full output of the ABI checking that i did if it would be helpful.

How to repeat:
install the latest version, and compare the header files to the previous version with a tool like "icheck".  for example:

icheck --canonify -o foobar_old.ic /usr/include/mysql/mysql.h

upgrade to new version

icheck --canonify -o foobar_new.ic /usr/include/mysql/mysql.h

icheck --compare foobar_old.ic foobar_new.ic

Suggested fix:
ideally, back out the changes.

not-so-ideally, increment the SONAME for the shared library.

current_stmt was added in early June*, between releases for 5.0.23 and 5.0.24, as part of a fix for Bug#16017.  We indeed should be very careful about breaking the ABI.

Thank you for the report.

Verified as described.

There is API change in MySQL 5.0.26 haven't been announced.

http://lists.mysql.com/commits/14068

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14075

ChangeSet@1.2593, 2006-10-20 16:41:03+02:00, msvensson@neptunus.(none) +5 -0
  WL#3462 Add testing of MySQL client ABI to prevent unintentional ABI breaks
  BUG#23427 incompatible ABI change in 5.0.26?
     - Use the icheck tool if avaliable and compare the current mysql.h to a version
  controlled reference file

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14100

ChangeSet@1.2292, 2006-10-20 17:17:24-04:00, cmiller@zippy.cornsilk.net +2 -0
  Bug #23427: incompatible ABI change in 5.0.26?
  
  Revert 1 June change enough to restore ABI compatibility with previous
  versions.

Why is this structure public at all?
I think you'd avoid lots of problems by making it private.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14249

ChangeSet@1.2535, 2006-10-24 17:19:02+05:00, holyfoot@mysql.com +3 -0
  Bug #23427 (incompatible ABI change)
  
  the incompatibility was caused by current_stmt member added to the MYSQL
  structure.
  It's possible to move it to THD structure instead which saves ABI

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14296

ChangeSet@1.2593, 2006-10-24 20:32:07+02:00, msvensson@neptunus.(none) +5 -0
  WL#3462 Add testing of MySQL client ABI to prevent unintentional ABI breaks
  BUG#23427 incompatible ABI change in 5.0.26?
     - Use the icheck tool if avaliable and compare the current mysql.h to a version
  controlled reference file

The icheck patch is no good.
1. It doesn't fail the build when check fails. This worked before. 
2. The .ic file generated on my machine fails to compare on sapsrv1. Tested this before and at that time it worked. Strange.
3. "make dist" on melody fails.

:(

Hi

Applying the patch http://lists.mysql.com/commits/14100 to 5.0 removes on ABI breakage but icheck still produces some more warnings between the 5.0.24a and 5.0.26+patch version of mysql.h.

Please tell me if this needs patching, too.

bye,

-christian- / Debian

ABI and API mismatch between:
  my_bool (MYSQL * mysql, enum enum_server_command, char const * header, unsigned long int, char const * arg, unsigned long int, my_bool, MYSQL_STMT * stmt)
and:
  my_bool (MYSQL * mysql, enum enum_server_command, char const * header, unsigned long int, char const * arg, unsigned long int, my_bool)
 in declaration:
my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const * header, unsigned long int, char const * arg, unsigned long int, my_bool, M
YSQL_STMT * stmt);
 versus declaration:
my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const * header, unsigned long int, char const * arg, unsigned long int, my_bool);
 in type defined at /usr/include/mysql/mysql.h:739:
struct __attribute__((aligned(__alignof__(void *)))) st_mysql_methods
  {
    my_bool (* read_query_result)(MYSQL * mysql);
    my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const * header, unsigned long int, char const * arg, unsigned long int, my_boo
l, MYSQL_STMT * stmt);
    MYSQL_DATA * (* read_rows)(MYSQL * mysql, MYSQL_FIELD * mysql_fields, unsigned int);
    MYSQL_RES * (* use_result)(MYSQL * mysql);
    void (* fetch_lengths)(unsigned long int * to, MYSQL_ROW, unsigned int);
    void (* flush_use_result)(MYSQL * mysql);
    MYSQL_FIELD * (* list_fields)(MYSQL * mysql);
    my_bool (* read_prepare_result)(MYSQL * mysql, MYSQL_STMT * stmt);
    int (* stmt_execute)(MYSQL_STMT * stmt);
    int (* read_binary_rows)(MYSQL_STMT * stmt);
    int (* unbuffered_fetch)(MYSQL * mysql, char * * row);
    void (* free_embedded_thd)(MYSQL * mysql);
    char const * (* read_statistics)(MYSQL * mysql);
    my_bool (* next_result)(MYSQL * mysql);
    int (* read_change_user_result)(MYSQL * mysql, char * buff, char const * passwd);
    int (* read_rows_from_cursor)(MYSQL_STMT * stmt);
  };
 versus type defined at /usr/include/mysql/mysql.h:738:
struct __attribute__((aligned(__alignof__(void *)))) st_mysql_methods
  {
    my_bool (* read_query_result)(MYSQL * mysql);
    my_bool (* advanced_command)(MYSQL * mysql, enum enum_server_command, char const * header, unsigned long int, char const * arg, unsigned long int, my_boo
l);
    MYSQL_DATA * (* read_rows)(MYSQL * mysql, MYSQL_FIELD * mysql_fields, unsigned int);
    MYSQL_RES * (* use_result)(MYSQL * mysql);
    void (* fetch_lengths)(unsigned long int * to, MYSQL_ROW, unsigned int);
    void (* flush_use_result)(MYSQL * mysql);
    MYSQL_FIELD * (* list_fields)(MYSQL * mysql);
    my_bool (* read_prepare_result)(MYSQL * mysql, MYSQL_STMT * stmt);
    int (* stmt_execute)(MYSQL_STMT * stmt);
    int (* read_binary_rows)(MYSQL_STMT * stmt);
    int (* unbuffered_fetch)(MYSQL * mysql, char * * row);
    void (* free_embedded_thd)(MYSQL * mysql);
    char const * (* read_statistics)(MYSQL * mysql);
    my_bool (* next_result)(MYSQL * mysql);
    int (* read_change_user_result)(MYSQL * mysql, char * buff, char const * passwd);
    int (* read_rows_from_cursor)(MYSQL_STMT * stmt);
  };

API and ABI removal: identifier net_safe_read is gone
 was defined at /usr/include/mysql/mysql.h:839:
extern unsigned long int net_safe_read(MYSQL * mysql);

ABI is not compatible
API is not compatible

Christian how did you run icheck?

I have pushed an "icheck patch" to some of our build machines and it's detecting differences between machines. :(

Need some improvement, we should not include the machine dependent code in the icheck analysis.

I run manually after applying *only* the http://lists.mysql.com/commits/14100
patch from Chad Miller:
 icheck --canonical mysql-5.0.24a/include/mysql.h > icheck-5.0.24a
 icheck --canonical mysql-5.0.26/include/mysql.h > icheck-5.0.26
 icheck --compare icheck-5.0.24a icheck-5.0.26

After unpacking a source you can either put a mysql_version.h manually in there or compile the package.

I used amd64 as architecture - it can be that for different architectures the header file looks a bit different, I have no experience with icheck and cannot
tell how a decent automatic Makefile check could be done.
(Maybe by deleting all #include statments via sed so that only the mysql 
structures are left?)

If reproducing with icheck does not work for you, just verify manually if the structures in mysql.h that icheck complained about, are indeed different or not, that should be fairly easy and we would know if there is an open ABI change 
problem, or not.

bye,

-christian-

Removal of 'net_safe_read' should be safe. Anyone using that is doing weird things.

The addition of MYSQL_STMT* to the "advanced_command" function pointer in struct st_mysql_method is only used by embedded MySQL Server and should thus also be safe.

The plan is to reduce these publicly visible structs into "typedef void * MYSQL"  so the implementation of "struct mysql" is hidden in future versions of MySQL.

Thanks for your help!

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14449

ChangeSet@1.2346, 2006-10-26 22:20:43+02:00, msvensson@neptunus.(none) +2 -0
  Bug#23427  	incompatible ABI change in 5.0.26?
   - Improve icheck rules in Makefile.am

Noted in 5.0.27 (CS), 5.0.28 (ES), 5.1.13 changelogs.

MySQL 5.0.26 introduced an ABI incompatibility, which this release
reverts. Programs compiled against 5.0.26 are not compatible with any
other version and must be recompiled.

Suphap

Use ABI-compliance-checker instead of icheck because it provides more convenient and appropriate compatibility report
http://ispras.linux-foundation.org/index.php/ABI_compliance_checker