Bug #44295 SSL connection fails on bigendian

• Submitted: 15 Apr 2009 13:59
• Modified: 14 May 2009 13:36
• Reporter: Magnus Blåudd
• Status: Closed
• Category: MySQL Cluster: Cluster (NDB) storage engine
• Severity: S2 (Serious)
• Version: 7.0.4
• OS: Any
• Assigned to: Magnus Blåudd
• CPU Architecture: Any

[15 Apr 2009 13:59] Magnus Blåudd

Description:
When SSL connection is created the fd of the incoming connection is passed to the SSL library using 'SSL_set_fd'. The function's second arg is defined as "my_socket" to the user and "int" internally, thus causing an implicit cast from "struct my_socket" to "int" - which fails on big endian machines.

How to repeat:
PB2 for example

Suggested fix:
Change second arg for 'SSL_get_fd' to use "int" on unix and SOCKET on Windows.

[15 Apr 2009 15:10] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72180

[15 Apr 2009 15:11] Bugs System

Pushed into 5.1.32-ndb-7.0.6 (revid:[e-mail]-20090415151000-zj716oud02udxxkv) (version source revid:[e-mail]-20090415151000-zj716oud02udxxkv) (merge vers: 5.1.32-ndb-7.0.6) (pib:6)

[15 Apr 2009 15:29] Magnus Blåudd

Reviewed on IRC

[15 Apr 2009 17:43] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72210

2898 MySQL Build Team	2009-04-15
      Bug#44295 SSL connection fails on bigendian
      
      Cherry-pick Magnus' fix from the ongoing development branch (at 7.0.6)
      into the clone used to build 7.0.5
      
      For further comments, see the original changeset
        2868 Magnus Blåudd     2009-04-15
      modified:
        extra/yassl/include/openssl/ssl.h
        include/my_socket_posix.h
        include/my_socket_win32.h
        include/violite.h
        vio/viossl.c

[16 Apr 2009 9:14] Bugs System

Pushed into 5.1.32-ndb-7.0.6 (revid:[e-mail]-20090416090122-4rp6pgjuc47uzxg5) (version source revid:[e-mail]-20090416090122-4rp6pgjuc47uzxg5) (merge vers: 5.1.32-ndb-7.0.6) (pib:6)

[16 Apr 2009 11:10] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72264

[22 Apr 2009 13:16] Bugs System

Pushed into 5.1.32-ndb-7.0.6 (revid:[e-mail]-20090422131237-vo0kbk7kzwz17d95) (version source revid:[e-mail]-20090422131237-vo0kbk7kzwz17d95) (merge vers: 5.1.32-ndb-7.0.6) (pib:6)

[7 May 2009 15:44] Jon Stephens

Hi Magnus,

Isn't this more of a general Server bug than a bug in Cluster? Will this be backported to 5.1-main?

Thanks!

jon.

[8 May 2009 10:22] Magnus Blåudd

We have introduced an encapsulation of sockets in our version of MySQL Server that we call "my_socket patch" and that's why we got this problem.

In theory it should be fixed in main tree for MySQL Server but since it "magically" works anyway, there "sizeof(int) == sizeof(my_socket)", but not in our three.

[14 May 2009 13:36] Jon Stephens

Documented bugfix in the NDB-7.0.6 changelog as follows:

        SSL connections to SQL nodes failed on big-endian platforms.