Bug #9737 BDB fails on s390x because of datatype size mismatch

• Submitted: 7 Apr 2005 23:28
• Modified: 21 Aug 2006 22:19
• Reporter: [ name withheld ]
• Status: Unsupported
• Category: MySQL Server: BDB
• Severity: S3 (Non-critical)
• Version: 4.1.11
• OS: Linux (Fedora)
• CPU Architecture: Any

[7 Apr 2005 23:28] [ name withheld ]

Description:
MySQL fails the BDB regression test when built on S390x under Fedora Core 3 (or any other recent Red Hat release).  This has been known for a long time but I finally found the cause today: ha_berkeley::external_lock uses the thread_safe_add macro to increment share->rows.  But share->rows is ha_rows which is an 8-byte quantity if BIG_TABLES is enabled.  And thread_safe_add can be implemented as atomic_add, which assumes its argument is atomic_t, which is only 4 bytes.  The bug exists on all arches with this combination, but it's only readily obvious on a big-endian platform. On a little-endian platform you would only see that the addition didn't increment the high-order word, so it wouldn't manifest until beyond 2^32 rows in a table.

How to repeat:
Build on big-endian architecture where HAVE_ATOMIC_ADD is true, and try to run "make test".

Suggested fix:
For the moment I'm using the following patch.  This isn't really portable since it assumes we are using pthreads.  Possibly you should make a variant of thread_safe_add that doesn't assume its target is atomic_t.

diff -Naur mysql-4.1.11.orig/libmysqld/ha_berkeley.cc mysql-4.1.11/libmysqld/ha_berkeley.cc
--- mysql-4.1.11.orig/libmysqld/ha_berkeley.cc	2005-04-01 06:36:49.000000000 -0500
+++ mysql-4.1.11/libmysqld/ha_berkeley.cc	2005-04-07 18:54:26.885974477 -0400
@@ -1834,7 +1834,9 @@
   else
   {
     lock.type=TL_UNLOCK;			// Unlocked
-    thread_safe_add(share->rows, changed_rows, &share->mutex);
+    pthread_mutex_lock((&share->mutex));
+    share->rows += changed_rows;
+    pthread_mutex_unlock((&share->mutex));
     changed_rows=0;
     if (!--thd->transaction.bdb_lock_count)
     {
diff -Naur mysql-4.1.11.orig/sql/ha_berkeley.cc mysql-4.1.11/sql/ha_berkeley.cc
--- mysql-4.1.11.orig/sql/ha_berkeley.cc	2005-04-01 06:36:49.000000000 -0500
+++ mysql-4.1.11/sql/ha_berkeley.cc	2005-04-07 18:54:20.000000000 -0400
@@ -1834,7 +1834,9 @@
   else
   {
     lock.type=TL_UNLOCK;			// Unlocked
-    thread_safe_add(share->rows, changed_rows, &share->mutex);
+    pthread_mutex_lock((&share->mutex));
+    share->rows += changed_rows;
+    pthread_mutex_unlock((&share->mutex));
     changed_rows=0;
     if (!--thd->transaction.bdb_lock_count)
     {