Bug #15108 mysqld crashes when innodb_log_file_size is set > 4G

Description:
A recent change done in mysqld.cc regarding the variable innodb_log_file_size which should not be larger than 4G and was set to ~0L in previous versions, changed to LONGLONG_MAX which is 2^64-1, leads to server crash.

Run mysqld with the variable set to 6G for example :
./mysqld --innodb-log-file-size=6G

How to repeat:
andrey@lmy004:/work/mysql-5.0-clean/sql> gdb mysqld
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i586-suse-linux"...Using host libthread_db library "/lib/tls/libthread_db.so.1".

(gdb) run --innodb-log-file-size=6G
Starting program: /work/mysql-5.0-clean/sql/mysqld --innodb-log-file-size=6G
[Thread debugging using libthread_db enabled]
[New Thread 1075742944 (LWP 10443)]
051121 20:55:12 [ERROR] innobase_log_file_size can't be over 4GB on 32-bit systems

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1075742944 (LWP 10443)]
0x0838c042 in mutex_enter_func (mutex=0x0, file_name=0x84ecad4 "trx0trx.c", line=1960) at sync0sync.ic:254
254       mutex->count_using++;
Current language:  auto; currently c
(gdb) bt
#0  0x0838c042 in mutex_enter_func (mutex=0x0, file_name=0x84ecad4 "trx0trx.c", line=1960) at sync0sync.ic:254
#1  0x08351d12 in trx_recover_for_mysql (xid_list=0x86469a0, len=128) at trx0trx.c:1960
#2  0x08288617 in innobase_xa_recover (xid_list=0x86469a0, len=128) at ha_innodb.cc:7382
#3  0x0827278b in ha_recover (commit_list=0x0) at handler.cc:990
#4  0x08199e60 in init_server_components () at mysqld.cc:3065
#5  0x0819a29f in main (argc=2, argv=0xbffff0c4) at mysqld.cc:3353
(gdb) p mutex
$1 = (ib_mutex_t *) 0x0

Osku,

please find out why mysqld is crashing.

Also note the following:
srv0start.c:
"
        if (srv_n_log_files * srv_log_file_size >= 262144) {
                fprintf(stderr,
                "InnoDB: Error: combined size of log files must be < 4 GB\n");

                return(DB_ERROR);
        }
"

The COMBINED log size must be less than 4 GB. 

Please correct the check and the error text in ha_innodb.cc.

To allow the combined log file size to grow over 4 GB, we should at least fix the following function to cope with differences >= 4G. Probably also ut_dulint_add(), and also some other changes to get the arithmetics in log0* to work right. Then VERY careful testing.

/***********************************************************
Subtracts a dulint from another. NOTE that the difference must be positive
and smaller that 4G. */
UNIV_INLINE
ulint
ut_dulint_minus(
/*============*/
                        /* out: a - b */
        dulint  a,      /* in: dulint; NOTE a must be >= b and at most
                        2 to power 32 - 1 greater */
        dulint  b)      /* in: dulint */
{
        ulint   diff;

        if (a.high == b.high) {
                ut_ad(a.low >= b.low);

                return(a.low - b.low);
        }

        ut_ad(a.high == b.high + 1);

        diff = (ulint)(0xFFFFFFFFUL - b.low);
        diff += 1 + a.low;

        ut_ad(diff > a.low);

        return(diff);
}

Regards,

Heikki

I suggest we leave the >= 4 GB support to a later date. Recovery time with a >= 4 GB log is probably too long for it to be used right now.

Regards,

Heikki

Andrey, as your GDB session shows, we already print an error message about the invalid log file size. The code responsible for this is in innobase_init:

+	/* Check that values don't overflow on 32-bit systems. */
+	if (sizeof(ulint) == 4) {
+		if (innobase_buffer_pool_size > UINT_MAX32) {
+			sql_print_error(
+				"innobase_buffer_pool_size can't be over 4GB"
+				" on 32-bit systems");
+
+			DBUG_RETURN(0);
+		}
+
+		if (innobase_log_file_size > UINT_MAX32) {
+			sql_print_error(
+				"innobase_log_file_size can't be over 4GB"
+				" on 32-bit systems");
+
+			DBUG_RETURN(0);
+		}
+	}

When I originally made this patch months ago, returning 0 from the function was handled correctly on the MySQL side and it aborted everything and closed down and didn't crash. I know you changed things regarding handler initialization, but I have no idea how, so either tell me how innobase_init should signal failure or if the bug is on your side fix it.

Changing the category, since this probably is not an InnoDB bug.

It's on MySQL side, not on InnoDB side, therefore I rated it initially not as InnoDB bug. Miguel changed it to InnoDB.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/32646

Patch looks OK, assuming that this is the way we want to solve the problem.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/439

fixed in 5.0.19

Added entry to 5.0.19 changelog:

      <listitem>
        <para>
          Setting <literal>innodb_log_file_size</literal> to a value
          greater than 4G crashed the server. (Bug #15108)
        </para>
      </listitem>

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/1136

Pushed into 5.1.47 (revid:joro@sun.com-20100505145753-ivlt4hclbrjy8eye) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Push resulted from incorporation of InnoDB tree. No changes pertinent to this bug. Re-closing.

Pushed into mysql-next-mr (revid:alik@sun.com-20100524190136-egaq7e8zgkwb9aqi) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (pib:16)

Pushed into 6.0.14-alpha (revid:alik@sun.com-20100524190941-nuudpx60if25wsvx) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Pushed into 5.5.5-m3 (revid:alik@sun.com-20100524185725-c8k5q7v60i5nix3t) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Push resulted from incorporation of InnoDB tree. No changes pertinent to this bug.
Re-closing.

Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)