Bug #44295 SSL connection fails on bigendian

Description:
When SSL connection is created the fd of the incoming connection is passed to the SSL library using 'SSL_set_fd'. The function's second arg is defined as "my_socket" to the user and "int" internally, thus causing an implicit cast from "struct my_socket" to "int" - which fails on big endian machines.

How to repeat:
PB2 for example

Suggested fix:
Change second arg for 'SSL_get_fd' to use "int" on unix and SOCKET on Windows.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72180

Pushed into 5.1.32-ndb-7.0.6 (revid:magnus.blaudd@sun.com-20090415151000-zj716oud02udxxkv) (version source revid:magnus.blaudd@sun.com-20090415151000-zj716oud02udxxkv) (merge vers: 5.1.32-ndb-7.0.6) (pib:6)

Reviewed on IRC

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72210

2898 MySQL Build Team	2009-04-15
      Bug#44295 SSL connection fails on bigendian
      
      Cherry-pick Magnus' fix from the ongoing development branch (at 7.0.6)
      into the clone used to build 7.0.5
      
      For further comments, see the original changeset
        2868 Magnus Blåudd     2009-04-15
      modified:
        extra/yassl/include/openssl/ssl.h
        include/my_socket_posix.h
        include/my_socket_win32.h
        include/violite.h
        vio/viossl.c

Pushed into 5.1.32-ndb-7.0.6 (revid:magnus.blaudd@sun.com-20090416090122-4rp6pgjuc47uzxg5) (version source revid:magnus.blaudd@sun.com-20090416090122-4rp6pgjuc47uzxg5) (merge vers: 5.1.32-ndb-7.0.6) (pib:6)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72264

Pushed into 5.1.32-ndb-7.0.6 (revid:build@mysql.com-20090422131237-vo0kbk7kzwz17d95) (version source revid:build@mysql.com-20090422131237-vo0kbk7kzwz17d95) (merge vers: 5.1.32-ndb-7.0.6) (pib:6)

Hi Magnus,

Isn't this more of a general Server bug than a bug in Cluster? Will this be backported to 5.1-main?

Thanks!

jon.

We have introduced an encapsulation of sockets in our version of MySQL Server that we call "my_socket patch" and that's why we got this problem.

In theory it should be fixed in main tree for MySQL Server but since it "magically" works anyway, there "sizeof(int) == sizeof(my_socket)", but not in our three.

Documented bugfix in the NDB-7.0.6 changelog as follows:

        SSL connections to SQL nodes failed on big-endian platforms.