Bug #47167 "set global innodb_file_format_check" cannot set value by User-Defined Variable

• Submitted: 7 Sep 2009 10:07
• Modified: 19 Jun 2010 0:31
• Reporter: Yasufumi Kinoshita
• Status: Closed
• Category: MySQL Server: InnoDB Plugin storage engine
• Severity: S2 (Serious)
• Version: 5.1.37 + 1.0.4
• OS: Any
• Assigned to: Satya B
• CPU Architecture: Any

[7 Sep 2009 10:07] Yasufumi Kinoshita

Description:
"set global innodb_file_format_check" command seems to not able to set value using User-Defined Variable.

How to repeat:
mysql> set @old_innodb_file_format_check=@@innodb_file_format_check;
Query OK, 0 rows affected (0.00 sec)

mysql> select @old_innodb_file_format_check;
+-------------------------------+
| @old_innodb_file_format_check |
+-------------------------------+
| Antelope                      |
+-------------------------------+
1 row in set (0.00 sec)

mysql> set global innodb_file_format_check = Barracuda;
Query OK, 0 rows affected (0.00 sec)

mysql> select @@innodb_file_format_check;
+----------------------------+
| @@innodb_file_format_check |
+----------------------------+
| Barracuda                  |
+----------------------------+
1 row in set (0.00 sec)

mysql> set global innodb_file_format_check = @old_innodb_file_format_check;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> show warnings;
+---------+------+--------------------------------------+
| Level   | Code | Message                              |
+---------+------+--------------------------------------+
| Warning | 1210 | Ignoring SET innodb_file_format=XG |
+---------+------+--------------------------------------+
1 row in set (0.00 sec)

mysql> select @@innodb_file_format_check;
+----------------------------+
| @@innodb_file_format_check |
+----------------------------+
| Barracuda                  |
+----------------------------+
1 row in set (0.00 sec)

[7 Sep 2009 12:23] Sveta Smirnova

Thank you for the report.

Verified as described.

Seems to be some memory corruption:

set @old_innodb_file_format_check=@@innodb_file_format_check;
select @old_innodb_file_format_check;
@old_innodb_file_format_check
Antelope
set global innodb_file_format_check = Barracuda;
select @@innodb_file_format_check;
@@innodb_file_format_check
Barracuda
set global innodb_file_format_check = @old_innodb_file_format_check;
Warnings:
Warning 1210    Ignoring SET innodb_file_format=@?(H
select @@innodb_file_format_check;
@@innodb_file_format_check
Barracuda

[15 Sep 2009 1:56] Jimmy Yang

In summary, this bug is caused by using a local string buffer (buff[] in innodb_file_format_check_validate) across functions by saving its pointer in var->save_result in sys_var_pluginvar::check() and re-using it in  sys_var_pluginvar::update(). And this results in wrong return value innodb_file_format_check_update() due to the stale string buffer and wrong information in the buffer, causing the update fail.

In this case, we are setting a variable value "Antelope" from  '@old_innodb_file_format_check' to variable 'innodb_file_format_check':

mysql> set global innodb_file_format_check = @old_innodb_file_format_check;

In innodb_file_format_check_validate(), the variable name is put in a local name buffer. And it is saved for future use (which is wrong):

innodb_file_format_check_validate()
{
        const char*     file_format_input;
        char            buff[STRING_BUFFER_USUAL_SIZE]; <===

        file_format_input = value->val_str(value, buff, &len);
        ....
        *static_cast<const char**>(save) = file_format_input; <== Saved

        .....
}

(gdb) print file_format_input
$2 = 0x48d08300 "Antelope"
(gdb) print &buff
$3 = (char (*)[80]) 0x48d08300
(gdb) print (char *)buff
$4 = 0x48d08300 "Antelope"

(gdb) print save
$5 = (void *) 0x1af14e48

This "save" is the "&var->save_result" in the caller function sys_var_pluginvar::check():

(gdb)  print (char *)var->save_result->locale_value
$14 = 0x48d08300 "Antelope"

function stack:
innodb_file_format_check_validate()
sys_var_pluginvar::check()
set_var::check()
sql_set_variables()
mysql_execute_command()

Subsequently, this "save" value (var->save_result) is used for the update call:
 plugin_var->update(thd, plugin_var, tgt, &var->save_result);

function stack:
innodb_file_format_check_update()
sys_var_pluginvar::update()
set_var::update()
sql_set_variables()
mysql_execute_command()

The update call innodb_file_format_check_update(), and the value to be updated is passed from update() call with that saved stale name buffer (var->save_result->locale_value):

innodb_file_format_check_update()
{
   const char*     format_name_in;
   ...
   format_name_in = *static_cast<const char*const*>(save);
   format_id = innobase_file_format_name_lookup(format_name_in);
   ...
}

(gdb)  print format_name_in
$17 = 0x48d08300 "Antelope" <=== Notice this is a pointer to local variable buff[] defined innodb_file_format_check_validate(), which no longer allocated

In the innobase_file_format_name_lookup(), this name buffer conntent becomes stale after strtoul() (it can be messed up any time, but it happens here in this case).

(gdb) print format_name
$19 = 0x48d08300 ""

And as a result innobase_file_format_name_lookup() can never locate the name's corresponding id and returns an id > DICT_TF_FORMAT_MAX and triggers the error:

innodb_file_format_check_update()
{
        ...
        format_id = innobase_file_format_name_lookup(format_name_in);
        if (format_id > DICT_TF_FORMAT_MAX) {
                /* DEFAULT is "on", which is invalid at runtime. */
                push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_WARN, <===
                                    ER_WRONG_ARGUMENTS,
                                    "Ignoring SET innodb_file_format=%s",
                                    format_name_in);
                return;
        }

}

This warning is shown when "show warnings" called:

mysql> set global innodb_file_format_check = @old_innodb_file_format_check;
Query OK, 0 rows affected, 1 warning (33 min 43.91 sec)

mysql> show warnings;
+---------+------+--------------------------------------+
| Level   | Code | Message                              |
+---------+------+--------------------------------------+
| Warning | 1210 | Ignoring SET innodb_file_format=P??H |
+---------+------+--------------------------------------+
1 row in set (0.00 sec)

To fix this, we might need to pre-allocated the value buffer in functions higher in stack such as sql_set_variables(), or find another alternative to fetch the correct value information in update().

thanks
Jimmy

[19 Sep 2009 0:03] Jimmy Yang

Although we know the cause for this problem, the fix could be a bit complicated. As it could involve code in the mysql layer. 

The observation is that the assignment of constant value does not have any problem. This is due to the different behavior how the memory is allocated in the val_str() calls.

In fact, the value->val_str() call behaves differently with constant value assignment and variable value assignment as functions map to functions in their own class:

Item_string::val_str   <== constant value assignment
Item_func_get_user_var::val_str  item_func.cc:4557 <== Variable value assignment

In the constant value assignment, the value->val_str() did not use the buffer passed in, instead, it allocates a memory to hold it in item_val_str() call:

static const char *item_val_str()
{

  String str(buffer, *length, system_charset_info), *res;
  if (!(res= ((st_item_value_holder*)value)->item->val_str(&str)))
    return NULL;
  *length= res->length();
  if (res->c_ptr_quick() == buffer)
    return buffer;

  /*
    Lets be nice and create a temporary string since the
    buffer was too small
  */
  return current_thd->strmake(res->c_ptr_quick(), res->length()); <===
}

we could also mimic the behavior by setting the len to 0 when calling  value->val_str() in innodb_file_format_name_validate(), this will also allow the memory to be allocated. This will fix the stale problem. However, we need to consult with mysql team to see whether this change will lead of some form of memory leak during the string operation by allocating new memory everytime.

Thanks
Jimmy

[30 Nov 2009 11:56] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/92065

3223 Satya B	2009-11-30
      Applying InnoDB Plugin 1.0.6 snapshot, part 3. Fixes BUG#47167
      
      applied revisions: r6157
      
      Detailed revision comments:
      
      r6157 | jyang | 2009-11-11 14:27:09 +0200 (Wed, 11 Nov 2009) | 10 lines
      branches/zip: Fix an issue that a local variable defined
      in innodb_file_format_check_validate() is being referenced
      across function in innodb_file_format_check_update().
      In addition, fix "set global innodb_file_format_check =
      DEFAULT" call.
      Bug #47167: "set global innodb_file_format_check" cannot
      set value by User-Defined Variable."
      rb://169 approved by Sunny Bains and Marko.

[1 Dec 2009 10:10] Satya B

patch queued to mysql-5.1-bugteam storage/innodb_plugin only.
WIll be merge mysql-trunk.

[2 Dec 2009 8:02] Bugs System

Pushed into 5.1.42 (revid:[e-mail]-20091202080033-mndu4sxwx19lz2zs) (version source revid:[e-mail]-20091130115621-ulbhn5xq15uunanu) (merge vers: 5.1.42) (pib:13)

[3 Dec 2009 3:03] Paul DuBois

Noted in 5.1.42 changelog.

The innodb_file_format_check system variable could not be set at
runtime to DEFAULT or to the value of a user-defined variable.

Setting report to NDI pending push to 5.6.x+.

[16 Dec 2009 8:39] Bugs System

Pushed into 6.0.14-alpha (revid:[e-mail]-20091216083311-xorsasf5kopjxshf) (version source revid:[e-mail]-20091214191830-wznm8245ku8xo702) (merge vers: 6.0.14-alpha) (pib:14)

[16 Dec 2009 8:46] Bugs System

Pushed into 5.5.0-beta (revid:[e-mail]-20091216082430-s0gtzibcgkv4pqul) (version source revid:[e-mail]-20091202140050-nh3ebk6s3bziv8cb) (merge vers: 5.5.0-beta) (pib:14)

[16 Dec 2009 8:52] Bugs System

Pushed into mysql-next-mr (revid:[e-mail]-20091216083231-rp8ecpnvkkbhtb27) (version source revid:[e-mail]-20091212203859-fx4rx5uab47wwuzd) (merge vers: 5.6.0-beta) (pib:14)

[16 Dec 2009 16:45] Paul DuBois

Noted in 5.5.1 changelog.

Does not apply to 6.0.x.

[16 Dec 2009 16:45] Paul DuBois

Noted in 5.5.1 changelog.

Does not apply to 6.0.x.

[12 Mar 2010 14:13] Bugs System

Pushed into 5.1.44-ndb-7.0.14 (revid:[e-mail]-20100312135944-t0z8s1da2orvl66x) (version source revid:[e-mail]-20100312115609-woou0te4a6s4ae9y) (merge vers: 5.1.44-ndb-7.0.14) (pib:16)

[12 Mar 2010 14:29] Bugs System

Pushed into 5.1.44-ndb-6.2.19 (revid:[e-mail]-20100312134846-tuqhd9w3tv4xgl3d) (version source revid:[e-mail]-20100312060623-mx6407w2vx76h3by) (merge vers: 5.1.44-ndb-6.2.19) (pib:16)

[12 Mar 2010 14:45] Bugs System

Pushed into 5.1.44-ndb-6.3.33 (revid:[e-mail]-20100312135724-xcw8vw2lu3mijrhn) (version source revid:[e-mail]-20100312103652-snkltsd197l7q2yg) (merge vers: 5.1.44-ndb-6.3.33) (pib:16)

[12 Mar 2010 17:31] Paul DuBois

Fixed in earlier 5.1.x, 5.5.x.

[5 May 2010 15:14] Bugs System

Pushed into 5.1.47 (revid:[e-mail]-20100505145753-ivlt4hclbrjy8eye) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

[6 May 2010 17:39] Paul DuBois

Push resulted from incorporation of InnoDB tree. No changes pertinent to this bug.
Re-closing.

[28 May 2010 5:49] Bugs System

Pushed into mysql-next-mr (revid:[e-mail]-20100524190136-egaq7e8zgkwb9aqi) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (pib:16)

[28 May 2010 6:18] Bugs System

Pushed into 6.0.14-alpha (revid:[e-mail]-20100524190941-nuudpx60if25wsvx) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

[28 May 2010 6:46] Bugs System

Pushed into 5.5.5-m3 (revid:[e-mail]-20100524185725-c8k5q7v60i5nix3t) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

[30 May 2010 0:12] Paul DuBois

Push resulted from incorporation of InnoDB tree. No changes pertinent to this bug.
Re-closing.

[15 Jun 2010 8:14] Bugs System

Pushed into 5.5.5-m3 (revid:[e-mail]-20100615080459-smuswd9ooeywcxuc) (version source revid:[e-mail]-20100415070122-1nxji8ym4mao13ao) (merge vers: 5.1.47) (pib:16)

[15 Jun 2010 8:31] Bugs System

Pushed into mysql-next-mr (revid:[e-mail]-20100615080558-cw01bzdqr1bdmmec) (version source revid:[e-mail]-20100415070122-1nxji8ym4mao13ao) (pib:16)

[17 Jun 2010 11:49] Bugs System

Pushed into 5.1.47-ndb-7.0.16 (revid:[e-mail]-20100617114014-bva0dy24yyd67697) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

[17 Jun 2010 12:27] Bugs System

Pushed into 5.1.47-ndb-6.2.19 (revid:[e-mail]-20100617115448-idrbic6gbki37h1c) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

[17 Jun 2010 13:14] Bugs System

Pushed into 5.1.47-ndb-6.3.35 (revid:[e-mail]-20100617114611-61aqbb52j752y116) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)