Bug #74260 Assertion `sort_field->length >= length' failed in sql/filesort.cc:1029
Submitted: 8 Oct 2014 6:13 Modified: 14 Aug 2017 7:03
Reporter: Ramesh Sivaraman Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: DML Severity:S3 (Non-critical)
Version:5.6.20, 5.6.22, 5.7.19, 5.6.37 OS:Linux (CentOS 7)
Assigned to: CPU Architecture:Any

[8 Oct 2014 6:13] Ramesh Sivaraman
Description:
Assertion `sort_field->length >= length' failed in sql/filesort.cc:1029 | abort (sig=6) in make_sortkey

***************** Error Info

mysqld: /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.21-69.0/sql/filesort.cc:1029: void make_sortkey(Sort_
param*, uchar*, uchar*): Assertion `sort_field->length >= length' failed.
12:44:11 UTC - mysqld got signal 6 ;
--
--
Thread pointer: 0x7fd426eaf000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fd4d6212e18 thread_stack 0x40000
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(my_print_stacktrace+0x35)[0xab6ee8]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(handle_fatal_signal+0x41a)[0x72f6a6]
/lib64/libpthread.so.0(+0xf130)[0x7fd4d5c24130]
/lib64/libc.so.6(gsignal+0x39)[0x7fd4d48255c9]
/lib64/libc.so.6(abort+0x148)[0x7fd4d4826cd8]
/lib64/libc.so.6(+0x2e536)[0x7fd4d481e536]
/lib64/libc.so.6(+0x2e5e2)[0x7fd4d481e5e2]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z12make_sortkeyP10Sort_paramPhS1_+0x2ca)[0x938f83]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld[0x9385d8]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z8filesortP3THDP5TABLEP8FilesortbPyS5_+0x8ca)[0x936d91]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld[0x7b80f7]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_ZN13st_join_table10sort_tableEv+0x121)[0x7b5ba9]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z21join_init_read_recordP13st_join_table+0x61)[0x7b5679]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x1b5)[0x7b2f23]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld[0x7b290d]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_ZN4JOIN4execEv+0x6a5)[0x7b0869]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld[0x813ebb]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_P10SQL_I_ListI8st_orderESB_S7_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x24c)[0x8141b7]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z13handle_selectP3THDP13select_resultm+0x1c4)[0x812246]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld[0x7ea732]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z21mysql_execute_commandP3THD+0xd9c)[0x7e2c89]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x5a6)[0x7ed033]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xc88)[0x7df49c]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z10do_commandP3THD+0x3ac)[0x7de3c8]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x1c4)[0x7a6385]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(handle_one_connection+0x33)[0x7a5e8d]
/ssd/ramesh/Percona-Server-5.6.21-rel69.0-671.Linux.x86_64-debug/bin/mysqld(pfs_spawn_thread+0x159)[0xdc5590]
/lib64/libpthread.so.0(+0x7df3)[0x7fd4d5c1cdf3]
/lib64/libc.so.6(clone+0x6d)[0x7fd4d48e601d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fd3dbc1f010): SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a)
Connection ID (thread ID): 1
Status: NOT_KILLED

How to repeat:
create table t1(a set('a','a')not null) engine=csv;
insert into t1(a)values(1);
SET @@session.max_sort_length=4;
SET NAMES latin1 COLLATE latin1_german2_ci;
SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a);
[8 Oct 2014 6:14] Ramesh Sivaraman
Error log

Attachment: error.log (text/x-log), 7.71 KiB.

[8 Oct 2014 6:27] Ramesh Sivaraman
Please use debug build to reproduce the issue.

Server version :
[ramesh@qaserver-01 bin]$ ./mysqld --version
./mysqld  Ver 5.6.20-debug for Linux on x86_64 (MySQL Community Server (GPL))
[ramesh@qaserver-01 bin]$
[8 Oct 2014 7:50] MySQL Verification Team
see my internal Bug 18740249 - ASSERTION FAILED: SORT_FIELD->LENGTH >= LENGTH
[8 Oct 2014 7:57] MySQL Verification Team
Hello Ramesh Sivaraman,

Thank you for the bug report.
As Shane pointed out, duplicate of internal bug and only debug builds asserts.

Thanks,
Umesh
[8 Oct 2014 7:57] MySQL Verification Team
// 5.6.22
mysql> create table t1(a set('a','b')not null) engine=csv;
Query OK, 0 rows affected (0.02 sec)

mysql> insert into t1(a)values(1);
Query OK, 1 row affected (0.00 sec)

mysql> SET @@session.max_sort_length=4;
Query OK, 0 rows affected (0.00 sec)

mysql> SET NAMES latin1 COLLATE latin1_german2_ci;
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a);
ERROR 2013 (HY000): Lost connection to MySQL server during query

// snippet from error log

2014-10-10 03:42:03 10123 [Note] /data/ushastry/server/mysql-advanced-5.6.22/bin/mysqld-debug: ready for connections.
Version: '5.6.22-enterprise-commercial-advanced-debug'  socket: '/tmp/74260/socket.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition Debug (Commercial)
mysqld-debug: /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/filesort.cc:1026: void make_sortkey(Sort_param*, uchar*, uchar*): Assertion `sort_field->length >= length
' failed.
22:12:20 UTC - mysqld got signal 6 ;

(gdb) bt
#0  0x0000003bf260c8ac in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000000000afe982 in my_write_core (sig=6) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/mysys/stacktrace.c:422
#2  0x000000000077e6cc in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/signal_handler.cc:230
#3  <signal handler called>
#4  0x0000003bf2232635 in raise () from /lib64/libc.so.6
#5  0x0000003bf2233e15 in abort () from /lib64/libc.so.6
#6  0x0000003bf222b75e in __assert_fail_base () from /lib64/libc.so.6
#7  0x0000003bf222b820 in __assert_fail () from /lib64/libc.so.6
#8  0x000000000097e2ef in make_sortkey (param=0x7fe3299f77a0, to=0x7fe2fc04e899 "2001-01-01", ref_pos=0x7fe2fc00e6e0 "")
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/filesort.cc:1026
#9  0x000000000097d8d4 in find_all_keys (param=0x7fe3299f77a0, select=0x7fe2fc006d28, fs_info=0x7fe3299f7820, buffpek_pointers=0x7fe3299f7560, tempfile=0x7fe3299f7680, pq=0x0,
    found_rows=0x7fe3299f7ae0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/filesort.cc:814
#10 0x000000000097c032 in filesort (thd=0x92396b0, table=0x7fe2fc014940, filesort=0x7fe2fc006e98, sort_positions=false, examined_rows=0x7fe3299f7ae8, found_rows=0x7fe3299f7ae0)
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/filesort.cc:337
#11 0x0000000000802b44 in create_sort_index (thd=0x92396b0, join=0x7fe2fc005ff0, tab=0x7fe2fc01ad88)
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_executor.cc:3339
#12 0x00000000008005f6 in st_join_table::sort_table (this=0x7fe2fc01ad88) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_executor.cc:2508
#13 0x00000000008000ca in join_init_read_record (tab=0x7fe2fc01ad88) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_executor.cc:2382
#14 0x00000000007fd859 in sub_select (join=0x7fe2fc005ff0, join_tab=0x7fe2fc01ad88, end_of_records=false)
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_executor.cc:1256
#15 0x00000000007fd252 in do_select (join=0x7fe2fc005ff0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_executor.cc:933
#16 0x00000000007fb103 in JOIN::exec (this=0x7fe2fc005ff0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_executor.cc:194
#17 0x000000000085ccec in mysql_execute_select (thd=0x92396b0, select_lex=0x923bf28, free_join=true)
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_select.cc:1100
#18 0x000000000085d003 in mysql_select (thd=0x92396b0, tables=0x7fe2fc0051b0, wild_num=0, fields=..., conds=0x0, order=0x923c0f0, group=0x923c028, having=0x0, select_options=2147748608,
    result=0x7fe2fc005fc8, unit=0x923b8e0, select_lex=0x923bf28) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_select.cc:1221
#19 0x000000000085b00f in handle_select (thd=0x92396b0, result=0x7fe2fc005fc8, setup_tables_done_option=0)
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_select.cc:110
#20 0x0000000000834dfb in execute_sqlcom_select (thd=0x92396b0, all_tables=0x7fe2fc0051b0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_parse.cc:5103
#21 0x000000000082d553 in mysql_execute_command (thd=0x92396b0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_parse.cc:2649
#22 0x00000000008375d1 in mysql_parse (thd=0x92396b0, rawbuf=0x7fe2fc004ff0 "SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a)", length=43, parser_state=0x7fe3299f9680)
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_parse.cc:6245
#23 0x000000000082a3a2 in dispatch_command (command=COM_QUERY, thd=0x92396b0, packet=0x9369851 "", packet_length=43)
    at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_parse.cc:1332
#24 0x0000000000829407 in do_command (thd=0x92396b0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_parse.cc:1034
#25 0x00000000007f0887 in do_handle_one_connection (thd_arg=0x92396b0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_connect.cc:982
#26 0x00000000007f0358 in handle_one_connection (arg=0x92396b0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/sql/sql_connect.cc:898
#27 0x0000000000e5c19d in pfs_spawn_thread (arg=0x934b6a0) at /export/home/pb2/build/sb_0-13146752-1409933888.09/mysqlcom-pro-5.6.22/storage/perfschema/pfs.cc:1860
#28 0x0000003bf26079d1 in start_thread () from /lib64/libpthread.so.0
#29 0x0000003bf22e886d in clone () from /lib64/libc.so.6
[13 Jan 2015 14:30] Stefan Hinz
Closed by mistake, sorry. Reopened. -Stefan
[6 Apr 2015 6:14] Ramesh Sivaraman
Found similar type of assertion failure.

** Testcase

DROP DATABASE test;CREATE DATABASE test;USE test;
CREATE TABLE t1(cik int,pk int,ci int,key(cik),primary key (pk))ENGINE=InnoDB;
insert INTO t1 values(1,0,0);
insert INTO t1 values(17,4,4);
SELECT * FROM t1 ORDER BY HEX(UNCOMPRESSED_LENGTH(pk));

** error info

mysqld: /sda/build/mysql-5.6.23/sql/filesort.cc:1011: void make_sortkey(Sort_param*, uchar*, uchar*): Assertion `0' failed.
06:05:29 UTC - mysqld got signal 6 ;

** GDB info

#1  0x0000000000a9804e in my_write_core (sig=6) at /sda/build/mysql-5.6.23/mysys/stacktrace.c:422
#2  0x00000000007277f0 in handle_fatal_signal (sig=6) at /sda/build/mysql-5.6.23/sql/signal_handler.cc:230
#3  <signal handler called>
#4  0x00007fa3b878b5d7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#5  0x00007fa3b878ccc8 in __GI_abort () at abort.c:90
#6  0x00007fa3b8784546 in __assert_fail_base (fmt=0x7fa3b88d4128 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0xf8bc6f "0", file=file@entry=0xf8b900 "/sda/build/mysql-5.6.23/sql/filesort.cc", line=line@entry=1011, function=function@entry=0xf8c460 <make_sortkey(Sort_param*, unsigned char*, unsigned char*)::__PRETTY_FUNCTION__> "void make_sortkey(Sort_param*, uchar*, uchar*)") at assert.c:92
#7  0x00007fa3b87845f2 in __GI___assert_fail (assertion=0xf8bc6f "0", file=0xf8b900 "/sda/build/mysql-5.6.23/sql/filesort.cc", line=1011, function=0xf8c460 <make_sortkey(Sort_param*, unsigned char*, unsigned char*)::__PRETTY_FUNCTION__> "void make_sortkey(Sort_param*, uchar*, uchar*)") at assert.c:101
#8  0x000000000091c754 in make_sortkey (param=0x7fa3b9f35440, to=0x7fa39b17f108 "", ref_pos=0x7fa39b0983a8 "") at /sda/build/mysql-5.6.23/sql/filesort.cc:1011
#9  0x000000000091be10 in find_all_keys (param=0x7fa3b9f35440, select=0x7fa39b020d70, fs_info=0x7fa3b9f353d0, buffpek_pointers=0x7fa3b9f355e0, tempfile=0x7fa3b9f354c0, pq=0x0, found_rows=0x7fa3b9f35860) at /sda/build/mysql-5.6.23/sql/filesort.cc:814
#10 0x000000000091a5e5 in filesort (thd=0x7fa3a93ee000, table=0x7fa39b04f800, filesort=0x7fa39b020ee0, sort_positions=false, examined_rows=0x7fa3b9f35868, found_rows=0x7fa3b9f35860) at /sda/build/mysql-5.6.23/sql/filesort.cc:337
#11 0x00000000007a97eb in create_sort_index (thd=0x7fa3a93ee000, join=0x7fa39b01fc68, tab=0x7fa39b07d018) at /sda/build/mysql-5.6.23/sql/sql_executor.cc:3339
#12 0x00000000007a7321 in st_join_table::sort_table (this=0x7fa39b07d018) at /sda/build/mysql-5.6.23/sql/sql_executor.cc:2508
#13 0x00000000007a6df0 in join_init_read_record (tab=0x7fa39b07d018) at /sda/build/mysql-5.6.23/sql/sql_executor.cc:2382
#14 0x00000000007a46f7 in sub_select (join=0x7fa39b01fc68, join_tab=0x7fa39b07d018, end_of_records=false) at /sda/build/mysql-5.6.23/sql/sql_executor.cc:1256
#15 0x00000000007a40e1 in do_select (join=0x7fa39b01fc68) at /sda/build/mysql-5.6.23/sql/sql_executor.cc:933
#16 0x00000000007a203d in JOIN::exec (this=0x7fa39b01fc68) at /sda/build/mysql-5.6.23/sql/sql_executor.cc:194
#17 0x00000000008027a7 in mysql_execute_select (thd=0x7fa3a93ee000, select_lex=0x7fa3a93f0878, free_join=true) at /sda/build/mysql-5.6.23/sql/sql_select.cc:1100
#18 0x0000000000802aa3 in mysql_select (thd=0x7fa3a93ee000, tables=0x7fa39b01f248, wild_num=1, fields=..., conds=0x0, order=0x7fa3a93f0a40, group=0x7fa3a93f0978, having=0x0, select_options=2147748608, result=0x7fa39b01fc40, unit=0x7fa3a93f0230, select_lex=0x7fa3a93f0878) at /sda/build/mysql-5.6.23/sql/sql_select.cc:1221
#19 0x0000000000800b2b in handle_select (thd=0x7fa3a93ee000, result=0x7fa39b01fc40, setup_tables_done_option=0) at /sda/build/mysql-5.6.23/sql/sql_select.cc:110
#20 0x00000000007da980 in execute_sqlcom_select (thd=0x7fa3a93ee000, all_tables=0x7fa39b01f248) at /sda/build/mysql-5.6.23/sql/sql_parse.cc:5103
#21 0x00000000007d3730 in mysql_execute_command (thd=0x7fa3a93ee000) at /sda/build/mysql-5.6.23/sql/sql_parse.cc:2649
#22 0x00000000007dd54a in mysql_parse (thd=0x7fa3a93ee000, rawbuf=0x7fa39b01f010 "SELECT * FROM t1 ORDER BY HEX(UNCOMPRESSED_LENGTH(pk))", length=54, parser_state=0x7fa3b9f36e70) at /sda/build/mysql-5.6.23/sql/sql_parse.cc:6357
#23 0x00000000007d06bc in dispatch_command (command=COM_QUERY, thd=0x7fa3a93ee000, packet=0x7fa3a27e1001 "", packet_length=54) at /sda/build/mysql-5.6.23/sql/sql_parse.cc:1332
#24 0x00000000007cf7ab in do_command (thd=0x7fa3a93ee000) at /sda/build/mysql-5.6.23/sql/sql_parse.cc:1034
#25 0x0000000000797b3d in do_handle_one_connection (thd_arg=0x7fa3a93ee000) at /sda/build/mysql-5.6.23/sql/sql_connect.cc:982
#26 0x0000000000797626 in handle_one_connection (arg=0x7fa3a93ee000) at /sda/build/mysql-5.6.23/sql/sql_connect.cc:898
#27 0x0000000000b5d798 in pfs_spawn_thread (arg=0x7fa3b43fe600) at /sda/build/mysql-5.6.23/storage/perfschema/pfs.cc:1860
#28 0x00007fa3b997adf5 in start_thread (arg=0x7fa3b9f38700) at pthread_create.c:308
#29 0x00007fa3b884c1ad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
[24 May 2017 9:47] Erlend Dahl
Not reproducible on recent 5.7/trunk.
[13 Aug 2017 23:06] Roel Van de Paar
The first testcase was likely fixed elsewhere. The first line in that case now gives;

mysql> create table t1(a set('a','a')not null) engine=csv;
ERROR 1291 (HY000): Column 'a' has duplicated value 'a' in SET

If the testcase is altered to 'a','b' like so;

drop table t1;
create table t1(a set('a','b')not null) engine=csv;
insert into t1(a)values(1);
SET @@session.max_sort_length=4;
SET NAMES latin1 COLLATE latin1_german2_ci;
SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a);

the server still crashes, even on 5.7.17

mysql> SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a);
ERROR 2013 (HY000): Lost connection to MySQL server during query

Core was generated by `/sda/MS010217-mysql-5.7.17-linux-x86_64-debug/bin/mysqld --no-defaults --log-bi'.
Program terminated with signal 6, Aborted.
#0  0x00007fa97cf8b741 in __pthread_kill (threadid=<optimized out>, signo=6) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:61
61	  val = INTERNAL_SYSCALL (tgkill, err, 3, THREAD_GETMEM (THREAD_SELF, pid),
(gdb) bt
#0  0x00007fa97cf8b741 in __pthread_kill (threadid=<optimized out>, signo=6) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:61
#1  0x0000000001826d71 in my_write_core (sig=6) at /git/mysql-server_dbg/mysys/stacktrace.c:249
#2  0x0000000000e75574 in handle_fatal_signal (sig=6) at /git/mysql-server_dbg/sql/signal_handler.cc:220
#3  <signal handler called>
#4  0x00007fa97b31f1d7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#5  0x00007fa97b3208c8 in __GI_abort () at abort.c:90
#6  0x00007fa97b318146 in __assert_fail_base (fmt=0x7fa97b4693a8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x1d75ddf "sort_field->length >= length", file=file@entry=0x1d756a0 "/git/mysql-server_dbg/sql/filesort.cc", line=line@entry=1362, 
    function=function@entry=0x1d77040 <Sort_param::make_sortkey(unsigned char*, unsigned char const*)::__PRETTY_FUNCTION__> "uint Sort_param::make_sortkey(uchar*, const uchar*)") at assert.c:92
#7  0x00007fa97b3181f2 in __GI___assert_fail (assertion=0x1d75ddf "sort_field->length >= length", file=0x1d756a0 "/git/mysql-server_dbg/sql/filesort.cc", line=1362, 
    function=0x1d77040 <Sort_param::make_sortkey(unsigned char*, unsigned char const*)::__PRETTY_FUNCTION__> "uint Sort_param::make_sortkey(uchar*, const uchar*)") at assert.c:101
#8  0x0000000000edee14 in Sort_param::make_sortkey (this=0x7fa97d57aaf0, to=0x7fa93fae4021 "2001-01-01", ref_pos=0x7fa93f8aa938 "") at /git/mysql-server_dbg/sql/filesort.cc:1362
#9  0x0000000000ede061 in find_all_keys (param=0x7fa97d57aaf0, qep_tab=0x7fa93f8a56e0, fs_info=0x7fa97d57aa70, chunk_file=0x7fa97d57ace0, tempfile=0x7fa97d57ab80, pq=0x0, found_rows=0x7fa97d57afd8) at /git/mysql-server_dbg/sql/filesort.cc:1026
#10 0x0000000000edc48c in filesort (thd=0x7fa93f819000, filesort=0x7fa93f82cf98, sort_positions=false, examined_rows=0x7fa97d57afe0, found_rows=0x7fa97d57afd8, returned_rows=0x7fa97d57afd0) at /git/mysql-server_dbg/sql/filesort.cc:423
#11 0x00000000014b2ed6 in create_sort_index (thd=0x7fa93f819000, join=0x7fa93f8a5030, tab=0x7fa93f8a56e0) at /git/mysql-server_dbg/sql/sql_executor.cc:3665
#12 0x00000000014b0164 in QEP_TAB::sort_table (this=0x7fa93f8a56e0) at /git/mysql-server_dbg/sql/sql_executor.cc:2599
#13 0x00000000014afb60 in join_init_read_record (tab=0x7fa93f8a56e0) at /git/mysql-server_dbg/sql/sql_executor.cc:2465
#14 0x00000000014ace7f in sub_select (join=0x7fa93f8a5030, qep_tab=0x7fa93f8a56e0, end_of_records=false) at /git/mysql-server_dbg/sql/sql_executor.cc:1271
#15 0x00000000014ac812 in do_select (join=0x7fa93f8a5030) at /git/mysql-server_dbg/sql/sql_executor.cc:944
#16 0x00000000014aa793 in JOIN::exec (this=0x7fa93f8a5030) at /git/mysql-server_dbg/sql/sql_executor.cc:199
#17 0x0000000001544750 in handle_query (thd=0x7fa93f819000, lex=0x7fa93f81b2f8, result=0x7fa93f82cc98, added_options=0, removed_options=0) at /git/mysql-server_dbg/sql/sql_select.cc:184
#18 0x00000000014fa6e6 in execute_sqlcom_select (thd=0x7fa93f819000, all_tables=0x7fa93f82c328) at /git/mysql-server_dbg/sql/sql_parse.cc:5195
#19 0x00000000014f3df9 in mysql_execute_command (thd=0x7fa93f819000, first_level=true) at /git/mysql-server_dbg/sql/sql_parse.cc:2794
#20 0x00000000014fb5f5 in mysql_parse (thd=0x7fa93f819000, parser_state=0x7fa97d57c550) at /git/mysql-server_dbg/sql/sql_parse.cc:5611
#21 0x00000000014f0c1d in dispatch_command (thd=0x7fa93f819000, com_data=0x7fa97d57ccb0, command=COM_QUERY) at /git/mysql-server_dbg/sql/sql_parse.cc:1461
#22 0x00000000014efa44 in do_command (thd=0x7fa93f819000) at /git/mysql-server_dbg/sql/sql_parse.cc:999
#23 0x000000000161f9d0 in handle_connection (arg=0x7fa952630960) at /git/mysql-server_dbg/sql/conn_handler/connection_handler_per_thread.cc:300
#24 0x00000000018534f9 in pfs_spawn_thread (arg=0x7fa960e76220) at /git/mysql-server_dbg/storage/perfschema/pfs.cc:2188
#25 0x00007fa97cf86dc5 in start_thread (arg=0x7fa97d57d700) at pthread_create.c:308
#26 0x00007fa97b3e173d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
[13 Aug 2017 23:06] Roel Van de Paar
The second testcase also still gives problems;

mysql> SELECT * FROM t1 ORDER BY HEX(UNCOMPRESSED_LENGTH(pk));
+------+----+------+
| cik  | pk | ci   |
+------+----+------+
|    1 |  0 |    0 |
|   17 |  4 |    4 |
+------+----+------+
2 rows in set, 2 warnings (0.00 sec)

mysql> show warnings;
+---------+------+----------------------------+
| Level   | Code | Message                    |
+---------+------+----------------------------+
| Warning | 1259 | ZLIB: Input data corrupted |
| Warning | 1259 | ZLIB: Input data corrupted |
+---------+------+----------------------------+

This looks like a serious bug and should be re-opened.
[14 Aug 2017 7:03] MySQL Verification Team
With altered test case as posted by Roel, 5.7.19 debug build is affected.

rm -rf 74260
bin/mysqld  --initialize-insecure --basedir=$PWD --datadir=$PWD/74260 -v
bin/mysqld-debug  --basedir=$PWD --datadir=$PWD/74260 --core-file --socket=/tmp/mysql_ushastry.sock  --port=5719 --log-error=$PWD/74260/log.err 2>&1 &

(gdb) bt
#0  0x00007f329eed7771 in pthread_kill () from /lib64/libpthread.so.0
#1  0x00000000018d40b5 in my_write_core (sig=6) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/mysys/stacktrace.c:249
#2  0x0000000000ed66b6 in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/signal_handler.cc:220
#3  <signal handler called>
#4  0x00007f329d8d15d7 in raise () from /lib64/libc.so.6
#5  0x00007f329d8d2cc8 in abort () from /lib64/libc.so.6
#6  0x00007f329d8ca546 in __assert_fail_base () from /lib64/libc.so.6
#7  0x00007f329d8ca5f2 in __assert_fail () from /lib64/libc.so.6
#8  0x0000000000f3e7b0 in Sort_param::make_sortkey (this=0x7f3294077670, to=0x7f3238932d81 "2001-01-01", ref_pos=0x7f32380118c8 "")
    at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/filesort.cc:1362
#9  0x0000000000f3d968 in find_all_keys(Sort_param *, QEP_TAB *, Filesort_info *, IO_CACHE *, IO_CACHE *, Bounded_queue<uchar*, uchar*, Sort_param, <unnamed>::Mem_compare> *, ha_rows *) (
    param=0x7f3294077670, qep_tab=0x7f32389312f0, fs_info=0x7f3294077700, chunk_file=0x7f3294077430, tempfile=0x7f3294077550, pq=0x0, found_rows=0x7f3294077a48)
    at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/filesort.cc:1026
#10 0x0000000000f3bc98 in filesort (thd=0x7f3238000b70, filesort=0x7f3238007538, sort_positions=false, examined_rows=0x7f3294077a50, found_rows=0x7f3294077a48, returned_rows=0x7f3294077a40)
    at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/filesort.cc:423
#11 0x0000000001548585 in create_sort_index (thd=0x7f3238000b70, join=0x7f3238930c40, tab=0x7f32389312f0)
    at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_executor.cc:3665
#12 0x000000000154575d in QEP_TAB::sort_table (this=0x7f32389312f0) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_executor.cc:2599
#13 0x0000000001545144 in join_init_read_record (tab=0x7f32389312f0) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_executor.cc:2465
#14 0x0000000001542373 in sub_select (join=0x7f3238930c40, qep_tab=0x7f32389312f0, end_of_records=false)
    at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_executor.cc:1271
#15 0x0000000001541cf8 in do_select (join=0x7f3238930c40) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_executor.cc:944
#16 0x000000000153fbce in JOIN::exec (this=0x7f3238930c40) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_executor.cc:199
#17 0x00000000015dcb5e in handle_query (thd=0x7f3238000b70, lex=0x7f3238002ce0, result=0x7f3238007238, added_options=0, removed_options=0)
    at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_select.cc:184
#18 0x0000000001591d2e in execute_sqlcom_select (thd=0x7f3238000b70, all_tables=0x7f32380068c8) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_parse.cc:5158
#19 0x000000000158a922 in mysql_execute_command (thd=0x7f3238000b70, first_level=true) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_parse.cc:2795
#20 0x0000000001592ba7 in mysql_parse (thd=0x7f3238000b70, parser_state=0x7f3294079690) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_parse.cc:5574
#21 0x000000000158756e in dispatch_command (thd=0x7f3238000b70, com_data=0x7f3294079e00, command=COM_QUERY)
    at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_parse.cc:1461
#22 0x0000000001586371 in do_command (thd=0x7f3238000b70) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/sql_parse.cc:999
#23 0x00000000016bc734 in handle_connection (arg=0x4168110) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/sql/conn_handler/connection_handler_per_thread.cc:300
#24 0x000000000198c9f8 in pfs_spawn_thread (arg=0x411a160) at /export/home/pb2/build/sb_0-23948522-1498142411.21/mysql-5.7.19/storage/perfschema/pfs.cc:2188
#25 0x00007f329eed2df5 in start_thread () from /lib64/libpthread.so.0
#26 0x00007f329d99260d in clone () from /lib64/libc.so.6
(gdb)
[14 Aug 2017 7:03] MySQL Verification Team
-- 5.7.19 release build not affected

root@localhost [test]> drop table t1;
Query OK, 0 rows affected (0.00 sec)

root@localhost [test]> create table t1(a set('a','b')not null) engine=csv;
Query OK, 0 rows affected (0.00 sec)

root@localhost [test]> insert into t1(a)values(1);
Query OK, 1 row affected (0.00 sec)

root@localhost [test]> SET @@session.max_sort_length=4;
Query OK, 0 rows affected (0.00 sec)

root@localhost [test]> SET NAMES latin1 COLLATE latin1_german2_ci;
Query OK, 0 rows affected (0.00 sec)

root@localhost [test]> SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a);
+---+
| 1 |
+---+
| 1 |
+---+
1 row in set (0.00 sec)
[14 Aug 2017 7:07] MySQL Verification Team
-- 5.6.37 - only debug build affected, release build not crashing
rm -rf 74260
scripts/mysql_install_db --basedir=$PWD --datadir=$PWD/74260
bin/mysqld-debug --basedir=$PWD --datadir=$PWD/74260 --core-file --socket=/tmp/mysql.sock  --port=3306 --log-error=$PWD/74260/log.err 2>&1 &

(gdb) bt
#0  0x00007f0cb3902771 in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000000000acab63 in my_write_core (sig=6) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/mysys/stacktrace.c:424
#2  0x0000000000742b40 in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/signal_handler.cc:230
#3  <signal handler called>
#4  0x00007f0cb22fc5d7 in raise () from /lib64/libc.so.6
#5  0x00007f0cb22fdcc8 in abort () from /lib64/libc.so.6
#6  0x00007f0cb22f5546 in __assert_fail_base () from /lib64/libc.so.6
#7  0x00007f0cb22f55f2 in __assert_fail () from /lib64/libc.so.6
#8  0x0000000000945d7d in make_sortkey (param=0x7f0c88fe77e0, to=0x7f0c50039889 "2001-01-01", ref_pos=0x7f0c50011560 "")
    at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/filesort.cc:1048
#9  0x0000000000945362 in find_all_keys (param=0x7f0c88fe77e0, select=0x7f0c50006d08, fs_info=0x7f0c88fe7860, buffpek_pointers=0x7f0c88fe75a0, tempfile=0x7f0c88fe76c0, pq=0x0,
    found_rows=0x7f0c88fe7b20) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/filesort.cc:836
#10 0x0000000000943a0f in filesort (thd=0x1c9f4e0, table=0x7f0c5000fb80, filesort=0x7f0c50006e78, sort_positions=false, examined_rows=0x7f0c88fe7b28, found_rows=0x7f0c88fe7b20)
    at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/filesort.cc:347
#11 0x00000000007c8366 in create_sort_index (thd=0x1c9f4e0, join=0x7f0c50005fd0, tab=0x7f0c5001b198) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_executor.cc:3346
#12 0x00000000007c5e18 in st_join_table::sort_table (this=0x7f0c5001b198) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_executor.cc:2515
#13 0x00000000007c58ec in join_init_read_record (tab=0x7f0c5001b198) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_executor.cc:2389
#14 0x00000000007c3063 in sub_select (join=0x7f0c50005fd0, join_tab=0x7f0c5001b198, end_of_records=false)
    at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_executor.cc:1259
#15 0x00000000007c2a5c in do_select (join=0x7f0c50005fd0) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_executor.cc:936
#16 0x00000000007c0903 in JOIN::exec (this=0x7f0c50005fd0) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_executor.cc:194
#17 0x0000000000822e04 in mysql_execute_select (thd=0x1c9f4e0, select_lex=0x1ca19b0, free_join=true) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_select.cc:1101
#18 0x000000000082311b in mysql_select (thd=0x1c9f4e0, tables=0x7f0c50005190, wild_num=0, fields=..., conds=0x0, order=0x1ca1b78, group=0x1ca1ab0, having=0x0, select_options=2147748608,
    result=0x7f0c50005fa8, unit=0x1ca1368, select_lex=0x1ca19b0) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_select.cc:1222
#19 0x0000000000821137 in handle_select (thd=0x1c9f4e0, result=0x7f0c50005fa8, setup_tables_done_option=0)
    at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_select.cc:110
#20 0x00000000007fa901 in execute_sqlcom_select (thd=0x1c9f4e0, all_tables=0x7f0c50005190) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_parse.cc:5186
#21 0x00000000007f306a in mysql_execute_command (thd=0x1c9f4e0) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_parse.cc:2694
#22 0x00000000007fd47e in mysql_parse (thd=0x1c9f4e0, rawbuf=0x7f0c50004fd0 "SELECT 1 FROM t1 ORDER BY @x:=makedate(a,a)", length=43, parser_state=0x7f0c88fe96f0)
    at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_parse.cc:6438
#23 0x00000000007eff41 in dispatch_command (command=COM_QUERY, thd=0x1c9f4e0, packet=0x2479471 "", packet_length=43)
    at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_parse.cc:1376
#24 0x00000000007eeef1 in do_command (thd=0x1c9f4e0) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_parse.cc:1039
#25 0x00000000007b5294 in do_handle_one_connection (thd_arg=0x1c9f4e0) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_connect.cc:982
#26 0x00000000007b5029 in handle_one_connection (arg=0x1c9f4e0) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/sql/sql_connect.cc:899
#27 0x0000000000e1bde0 in pfs_spawn_thread (arg=0x243cd80) at /export/home/pb2/build/sb_0-23718186-1496438722.64/mysql-5.6.37/storage/perfschema/pfs.cc:1860
#28 0x00007f0cb38fddf5 in start_thread () from /lib64/libpthread.so.0
#29 0x00007f0cb23bd60d in clone () from /lib64/libc.so.6
(gdb)