Bug #31799 Scrambled number output due to integer overflow
Submitted: 23 Oct 2007 21:36 Modified: 20 Dec 2007 0:27
Reporter: Kristian Nielsen Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Data Types Severity:S3 (Non-critical)
Version:5.0-bk OS:Linux (x86_64)
Assigned to: Kristian Nielsen
Triage: D4 (Minor)

[23 Oct 2007 21:36] Kristian Nielsen
Description:
Using gcc 4.2.1 with -O3 produces wrong numeric output:

mysql> create table t1 (a bigint) as select '-9223372036854775807.5' as a;
mysql> select * from t1;
+----------------------+
| a                    |
+----------------------+
| -'..--).0-*(+,))+(0( | 
+----------------------+

(This is also seen in the test suite, test case 'round').

How to repeat:
Compile with BUILD/compile-amd64-max on x86_64 using GCC 4.2.1.
Run this: mysql-test-run.pl round

Suggested fix:
The problem is (I think) this code, in strings/ctype-simple.c:

int my_longlong10_to_str_8bit(CHARSET_INFO *cs __attribute__((unused)),
		      char *dst, uint len, int radix, longlong val)
{
...
 if (radix < 0)
  {
    if (val < 0)
    {
      val = -val;
...

If val == INT64_MIN, this negation will overflow, causing undefined behavior for ANSI C. This seems to be what causes gcc to generate code that does not work.

Since negation overflow is undefined in C, I guess the mysqld code has the bug and gcc is doing the "correct" thing here.

Maybe just add a special case for val = INT64_MIN in the code. I think there are several similar functions which must be fixed.
[24 Oct 2007 6:30] Kristian Nielsen
The expression (-x) can overflow for signed integer types, if x is equal to the smallest integer of the type. The result is undefined.

The well-defined way of getting the absolute value of x into an unsigned ux is:

   ux = (x >= 0 ? x : (unsigned)0 - (unsigned)x);
[24 Oct 2007 7:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/36230

ChangeSet@1.2488, 2007-10-24 09:26:25+02:00, knielsen@ymer.(none) +4 -0
  BUG#31799: Scrambled number output due to integer overflow
  
  An integer overflow in number->string conversion caused completely
  wrong output of the number LONGLONG_MIN with gcc 4.2.1.
  
  Fixed by eliminating the overflow, using only operations that are
  well-defined in ANSI C.
[24 Oct 2007 11:12] Alexander Barkov
The patch looks ok to push.
[24 Oct 2007 19:11] Kristian Nielsen
Stand-alone test program demonstarting the problem, shows gcc only generates non-working code when there is integer overflow

Attachment: test1.c (text/x-csrc), 996 bytes.

[24 Oct 2007 19:12] Kristian Nielsen
Assembler output generated from gcc 4.2.1 (-O3 -fno-inline)

Attachment: test1.s (application/octet-stream, text), 2.18 KiB.

[31 Oct 2007 13:29] Kristian Nielsen
Pushed to mysql-5.0-maint, mysql-5.1-new-maint, and mysql-5.2-maint.
[7 Dec 2007 23:08] Bugs System
Pushed into 6.0.5-alpha
[7 Dec 2007 23:09] Bugs System
Pushed into 5.1.23-rc
[7 Dec 2007 23:10] Bugs System
Pushed into 5.0.54
[20 Dec 2007 0:27] Paul Dubois
Noted in 5.0.54, 5.1.23, 6.0.5 changelogs.

A gcc problem caused incorrect numeric output due to integer overflow.