MySQL Bugs: #13619: SIG11: encrypt(): salt beginning with "

Bug #13619	SIG11: encrypt(): salt beginning with "_" followed by any char out ascii64
Submitted:	29 Sep 2005 19:02	Modified:	20 Oct 2005 1:10
Reporter:	Raphael Becker	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S1 (Critical)
Version:	5.0.12beta (any before!), 4.1.14	OS:	FreeBSD (FreeBSD, OpenBSD)
Assigned to:	Jim Winstead	CPU Architecture:	Any

Description:

On FreeBSD and OpenBSD (virtually any version in the last years) with mysql of any generation 3.23.x, 4.0.x, 4.1.x and 5.x the following statement triggers a signal 11

Tested with:
    FreeBSD 4.2, mysql 3.23.$veryold
    FreeBSD 4.10, mysql 3.23.x and 4.0
    FreeBSD 5.[34], mysql 4.[01].x 
    FreeBSD 6.0BETA4, mysql 5.0.12beta (tested here!)
    OpenBSD 3.x, mysql 3.23.x and 4.0

mysql> select encrypt('1234','_!');
ERROR 2013 (HY000): Lost connection to MySQL server during query

Going into sources:

/usr/local/libexec/mysqld:
        libz.so.3 => /lib/libz.so.3 (0x28421000)
        libwrap.so.4 => /usr/lib/libwrap.so.4 (0x28431000)
        libssl.so.4 => /usr/lib/libssl.so.4 (0x28438000)
        libcrypto.so.4 => /lib/libcrypto.so.4 (0x28466000)
        libcrypt.so.3 => /lib/libcrypt.so.3 (0x28558000)        
        libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x28570000)
        libm.so.4 => /lib/libm.so.4 (0x2863a000)
        libpthread.so.2 => /usr/lib/libpthread.so.2 (0x28650000)
        libc.so.6 => /lib/libc.so.6 (0x28675000)

---> looking for /lib/libcrypt.so.3
---> looking into sources (FreeBSD) http://www.freebsd.org/cgi/cvsweb.cgi/src/secure/lib/libcrypt/crypt-des.c?rev=1.14&content...

static u_char   ascii64[] =
         "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

Chars get transformed using the array above in the function 
     static INLINE int ascii_to_bin(char ch)
which is used by
     static void setup_salt(u_int32_t salt)
and
     char * crypt_des(const char *key, const char *setting)
     (looking into the "old style" salt)

Using any salt beginning with "_" followed by a character "outside" ascii64[] plus "." seems to result in a Sig11 of mysqld. 

Lookin on some code:

static INLINE int
ascii_to_bin(char ch)
{
	if (ch > 'z')
		return(0);
	if (ch >= 'a')
		return(ch - 'a' + 38);
	if (ch > 'Z')
		return(0);
	if (ch >= 'A')
		return(ch - 'A' + 12);
	if (ch > '9')
		return(0);
	if (ch >= '.')
		return(ch - '.');
	return(0);
}

==> any time, this function return(0) for the second char of the salt (this includes the '.', which gets return(0) too), mysqld dies (empiric!)

In fact, any mysql-user with just plain select-privileges on ANY table is able to Sig11 the mysqld:

(error-log)
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

I havn't tracked down the mysqld-sources, but i guess, it's not proper checking what is returned from crypt().

How to repeat:
log into mysql with any user and just do one of

select encrypt('1234','_!');
select encrypt('1234','__');
select encrypt('1234','_.'); <-- inside ascii64[] but fails too ( see: return(0)! )
[...]

BTW: The same happens to perl, but is handled more gracefully (no sig11):

some working:
$ perl -e 'print crypt("1234","_2")."\n";'
_22XYrNsoS6hE
$ perl -e 'print crypt("1234","_1")."\n";'
_1.1uHOEDNOlI
$ perl -e 'print crypt("1234","_0")."\n";'
_0sqCNeFRMDl.

some failing:
$ perl -e 'print crypt("1234","_.")."\n";'
$ perl -e 'print crypt("1234","__")."\n" ;'
$ perl -e 'print crypt("1234","_!")."\n" ;'

Suggested fix:
check salt before calling crypt(), handle gracefully
check the return-value of crypt()

not a problem on linux: (SuSE 9.0, mysql 5.0bk)

select encrypt('1234','_!');
+----------------------+
| encrypt('1234','_!') |
+----------------------+
| *0                   |
+----------------------+
1 row in set (0.36 sec)

I was able to repeat this behaviour on our buildb FreeBSD 5.4 machine with 4.1.14:

-bash-2.05b$ bin/mysql -uroot -S/users/vkravchuk/mysql4.sock
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.14

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select encrypt('1234','_!');
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=1
max_connections=100
threads_connected=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Server crashed. On other platforms everything work much better.

Windows (4.1.14 and 5.0.13-rc):

mysql> select encrypt('1234','_!');
+----------------------+
| encrypt('1234','_!') |
+----------------------+
| NULL                 |
+----------------------+
1 row in set (0.01 sec)

mysql> select encrypt('1234','_');
+---------------------+
| encrypt('1234','_') |
+---------------------+
| NULL                |
+---------------------+
1 row in set (0.00 sec)

mysql> select encrypt('1234','__');
+----------------------+
| encrypt('1234','__') |
+----------------------+
| NULL                 |
+----------------------+
1 row in set (0.00 sec)

mysql> select encrypt('1234','_.');
+----------------------+
| encrypt('1234','_.') |
+----------------------+
| NULL                 |
+----------------------+
1 row in set (0.00 sec)

Linux (4.1.15-BK):

mysql> select encrypt('1234','_!');
+----------------------+
| encrypt('1234','_!') |
+----------------------+
| _!jz9NkNiFXAs        |
+----------------------+
1 row in set (0,00 sec)

mysql> select encrypt('1234','__');
+----------------------+
| encrypt('1234','__') |
+----------------------+
| __f0/W3dRvHoA        |
+----------------------+
1 row in set (0,01 sec)

mysql> select encrypt('1234','_.');
+----------------------+
| encrypt('1234','_.') |
+----------------------+
| _.VVq0RdNc8.w        |
+----------------------+
1 row in set (0,00 sec)

mysql> select version();
+--------------+
| version()    |
+--------------+
| 4.1.15-debug |
+--------------+
1 row in set (0,00 sec)

So, it is yet another BSD-specific bug.

Posix define formely the salt to be in the 'a-zA-Z0-9./' range :
http://www.opengroup.org/onlinepubs/009695399/functions/crypt.html

So returning '_!jz9NkNiFXAs' for a "select encrypt('1234','_!');"
on linux isn't very smart either (at least, an undefined behaviour
from a posix viewpoint).

The specific *BSD behaviours on '_*' and '$*' salts is described on:
http://www.openbsd.org/cgi-bin/man.cgi?query=crypt
http://www.freebsd.org/cgi/man.cgi?query=crypt&sektion=3
And the specifics of '$*' on Linux/glibc2, here for instance:
http://linux.about.com/od/commands/l/blcmdl3_crypt.htm

Things like "select encrypt('1234', '$1$abc$');" (for instance) will
probably return differents results on windows and linux.

Appart from the suggested fixes on the parent bug report, the doc
may need to be corrected: the salt format (and restrictions) isn't
described, and the doc don't tell how system dependent the result
may be depending on the salt: 
http://dev.mysql.com/doc/mysql/en/encryption-functions.html

Thanks for the exhaustive analysys!

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/30787

Fixed in 4.1.16 and 5.0.15.

Noted in 4.1.16, 5.0.15 changelogs.