Bug #32034 check_func_enum() does not check correct values but set it to impossible int val

Description:
Due to incorrect types casting check_func_enum() do not detect error code from ind_type().

How to repeat:
Write engine with MYSQL_SYSVAR_ENUM() and assign incorrect value.

Suggested fix:
sql/sql_plugin.cc 1.75 vs edited =====
--- 1.75/sql/sql_plugin.cc      2007-10-08 22:00:21 +03:00
+++ edited/sql/sql_plugin.cc    2007-11-01 11:02:48 +02:00
@@ -1945,7 +1945,7 @@
     length= sizeof(buff);
     if (!(str= value->val_str(value, buff, &length)))
       goto err;
-    if ((result= find_type(typelib, str, length, 1)-1) < 0)
+    if ((result= ((long)find_type(typelib, str, length, 1)) - 1) < 0)
     {
       strvalue= str;
       goto err;

32bit Linux is not affected.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/37705

ChangeSet@1.2648, 2007-11-14 01:29:17+04:00, gshchepa@gleb.loc +4 -0
  Fixed bug #32034: On 64bit platforms assigning values of
  storage engine system variables was not validated and
  unexpected value was assigned.
  
  The check_func_enum function used subtraction from the uint
  value with the probably negative result. That result of
  type uint was compared with 0 after casting to signed long
  type. On architectures where long type is longer than int
  type the result of comparison was unexpected.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/37731

ChangeSet@1.2648, 2007-11-14 13:48:21+04:00, gshchepa@gleb.loc +4 -0
  Fixed bug #32034: On 64bit platforms assigning values of
  storage engine system variables was not validated and
  unexpected value was assigned.
  
  The check_func_enum function used subtraction from the uint
  value with the probably negative result. That result of
  type uint was compared with 0 after casting to signed long
  type. On architectures where long type is longer than int
  type the result of comparison was unexpected.

Pushed into 5.1.23-rc

Pushed into 6.0.4-alpha

Noted in 5.1.23, 6.0.4 changelogs.

On 64-bit platforms, assignments of values to storage engine-specific
system variables were not validated and could result in unexpected
values.

storage engine-specific system variables of ENUM (one value from the set of strings) type

Updated the changelog entries.