Bug #71853 eval_num_suffix return overflowed/underflowed value

• Submitted: 27 Feb 2014 6:37
• Modified: 5 Mar 2014 12:00
• Reporter: Yasushi Miyazaki
• Status: Verified
• Category: MySQL Server: Options
• Severity: S3 (Non-critical)
• Version: 5.6.16, 5.6.17
• OS: Linux
• CPU Architecture: Any

[27 Feb 2014 6:37] Yasushi Miyazaki

Description:
Function 'eval_num_suffix' in mysys_ssl/my_getopt.cc is possibility returing value overflow/underflow at 'K', 'M' or 'G' suffix.

How to repeat:
1. Set the following in my.cnf:
[mysqld]
optimizer_trace_offset = 9223372036854775807K
2. mysqld start
3. mysql -e 'select @@global.optimizer_trace_offset \G'
*************************** 1. row ***************************
@@global.optimizer_trace_offset: -1024

Suggested fix:
overflow/underflow value caps.
at function 'eval_num_suffix' in mysys_ssl/my_getopt.cc
for example, before:

  if (*endchar == 'k' || *endchar == 'K')
    num*= 1024L;
  else if (*endchar == 'm' || *endchar == 'M')
    num*= 1024L * 1024L;
  else if (*endchar == 'g' || *endchar == 'G')
    num*= 1024L * 1024L * 1024L;

after (using strtoll on eval_num_suffix):
  if (*endchar == 'k' || *endchar == 'K')
  {
    if (num > (LLONG_MAX / 1024L))
    {
      num = LLONG_MAX;
    }
    else if (num < (LLONG_MIN / 1024L))
    {
      num = LLONG_MIN;
    }
    else
    {
      num*= 1024L;
    }
  }
  else if (*endchar == 'm' || *endchar == 'M')
  {
    if (num > (LLONG_MAX / (1024L * 1024L)))
    {
      num = LLONG_MAX;
    }
    else if (num < (LLONG_MIN / (1024L * 1024L)))
    {
      num = LLONG_MIN;
    }
    else
    {
      num*= 1024L * 1024L;
    }
  }
  else if (*endchar == 'g' || *endchar == 'G')
  {
    if (num > (LLONG_MAX / (1024L * 1024L * 1024L)))
    {
      num = LLONG_MAX;
    }
    else if (num < (LLONG_MIN / (1024L * 1024L * 1024L)))
    {
      num = LLONG_MIN;
    }
    else
    {
      num*= 1024L * 1024L * 1024L;
    }
  }

[5 Mar 2014 12:00] MySQL Verification Team

Hello!,

Thank you for the bug report.
Verified as described.

Thanks,
Umesh