Bug #88223 Replication with no tmpdir space and InnoDB as tmp_storage_engine can break
Submitted: 25 Oct 2017 11:48 Modified: 7 Nov 2017 5:13
Reporter: Sveta Smirnova (OCA) Email Updates:
Status: Analyzing Impact on me:
Category:MySQL Server: Row Based Replication ( RBR ) Severity:S3 (Non-critical)
Version:5.7.19, 8.0.3 OS:Any
Assigned to: Umesh Shastry CPU Architecture:Any

[25 Oct 2017 11:48] Sveta Smirnova
This is same as bug #72457, but is repeatable only if internal_tmp_disk_storage_engine and default_tmp_storage_engine set to InnoDB. If set to MyISAM server crashes or not depending on binlog_error_action option.

Stack trace is also different:

2017-10-25T11:29:26.519497Z 0 [Note] /home/sveta/build/mysql-8.0/bin/mysqld: ready for connections. Version: '8.0.3-rc-debug-log'  socket: '/home/sveta/build/mysql-8.0/mysql-test/var/tmp/mysqld.1.sock'  port: 13000  Source distribution
mysqld: /home/sveta/src/mysql-server/sql/binlog.cc:7913: bool MYSQL_BIN_LOG::do_write_cache(IO_CACHE*, Binlog_event_writer*): Assertion `((cache)->pos_in_file + (size_t) (*(cache)->current_pos - (cache)->request_pos)) == expected_total_len' failed.
11:43:12 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 68195 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7fedb0000be0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fee18e2dd38 thread_stack 0x46000
/home/sveta/build/mysql-8.0/bin/mysqld(my_print_stacktrace+0x55) [0x36f0e56]
/home/sveta/build/mysql-8.0/bin/mysqld(handle_fatal_signal+0x3f2) [0x27c325b]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7fee21da7390]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7fee20940428]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7fee2094202a]
/lib/x86_64-linux-gnu/libc.so.6(+0x2dbd7) [0x7fee20938bd7]
/lib/x86_64-linux-gnu/libc.so.6(+0x2dc82) [0x7fee20938c82]
/home/sveta/build/mysql-8.0/bin/mysqld(MYSQL_BIN_LOG::do_write_cache(st_io_cache*, Binlog_event_writer*)+0x200) [0x362cedc]
/home/sveta/build/mysql-8.0/bin/mysqld(MYSQL_BIN_LOG::write_cache(THD*, binlog_cache_data*, Binlog_event_writer*)+0x239) [0x362da71]
/home/sveta/build/mysql-8.0/bin/mysqld(binlog_cache_data::flush(THD*, unsigned long long*, bool*)+0x27b) [0x361b619]
/home/sveta/build/mysql-8.0/bin/mysqld(binlog_cache_mngr::flush(THD*, unsigned long long*, bool*)+0x81) [0x363b6dd]
/home/sveta/build/mysql-8.0/bin/mysqld(MYSQL_BIN_LOG::flush_thread_caches(THD*)+0x54) [0x36307e4]
/home/sveta/build/mysql-8.0/bin/mysqld(MYSQL_BIN_LOG::process_flush_stage_queue(unsigned long long*, bool*, THD**)+0x14e) [0x3630a1a]
/home/sveta/build/mysql-8.0/bin/mysqld(MYSQL_BIN_LOG::ordered_commit(THD*, bool, bool)+0x4f1) [0x363217d]
/home/sveta/build/mysql-8.0/bin/mysqld(MYSQL_BIN_LOG::rollback(THD*, bool)+0xaaf) [0x361d48f]
/home/sveta/build/mysql-8.0/bin/mysqld(ha_rollback_trans(THD*, bool)+0x148) [0x292e37e]
/home/sveta/build/mysql-8.0/bin/mysqld(trans_rollback_stmt(THD*)+0xdb) [0x251b97b]
/home/sveta/build/mysql-8.0/bin/mysqld(mysql_execute_command(THD*, bool)+0x64fa) [0x23da4f6]
/home/sveta/build/mysql-8.0/bin/mysqld(mysql_parse(THD*, Parser_state*)+0x60c) [0x23dc154]
/home/sveta/build/mysql-8.0/bin/mysqld(dispatch_command(THD*, COM_DATA const*, enum_server_command)+0x11f2) [0x23d1ad1]
/home/sveta/build/mysql-8.0/bin/mysqld(do_command(THD*)+0x484) [0x23d053f]
/home/sveta/build/mysql-8.0/bin/mysqld() [0x27b1ff0]
/home/sveta/build/mysql-8.0/bin/mysqld() [0x3c77bde]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7fee21d9d6ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fee20a123dd]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fedb0009390): insert into t1 select * from t1
Connection ID (thread ID): 7

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file
safe_process[28930]: Child process: 28931, killed by signal: 6

How to repeat:
See bug #72457:

1. Create small fs:

dd if=/dev/zero of=`pwd`/bug64849 bs=1k count=50000
mkdir ./bug64849_data
sudo /sbin/mkfs.ext2 -F `pwd`/bug64849
sudo mount -o loop `pwd`/bug64849 `pwd`/bug64849_data
sudo chown sveta:sveta bug64849_data/

2. Start MTR:

./mtr --start --suite=rpl --mysqld=--tmpdir=`pwd`/bug64849_data rpl_alter &

3. Connect to master and run following queries:

mysql> set binlog_format='row';
Query OK, 0 rows affected (0.00 sec)

mysql> create table t1(f1 text) engine=myisam;
Query OK, 0 rows affected (0.11 sec)

mysql> insert into t1 values(md5(1));
Query OK, 1 row affected (0.00 sec)

mysql> insert into t1 select * from t1;
Query OK, 1 row affected (0.00 sec)
Records: 1  Duplicates: 0  Warnings: 0
mysql> insert into t1 select * from t1;
Query OK, 524288 rows affected (6.10 sec)
Records: 524288  Duplicates: 0  Warnings: 0

mysql> insert into t1 select * from t1;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Suggested fix:
Don't crash if binlog_error_action=IGNORE_ERROR  or crash with nice message, not assertion if binlog_error_action=ABORT_SERVER