Bug #14085 AES_ENCRYPT and AES_DECRYPT do not conform to standard
Submitted: 17 Oct 2005 20:55 Modified: 17 Oct 2005 21:45
Reporter: Brandon Bates Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:4.1.7 OS:Windows (Windows 2000)
Assigned to: CPU Architecture:Any

[17 Oct 2005 20:55] Brandon Bates
Description:
AES_ENCRYPT (incorrectly?) pads the end of encoded data with the number of bytes it is padding:
For example IF I encoded:
'hello world'  It would pad the end as such:
'hello world55555' (where the 5's are binary values, not ascii)
Everyone else (as far as I can find, for example mcrypt in PHP and a few other places) pads it as such:
'hello world00000' (again 0's are binary not ascii)

How to repeat:
SELECT HEX(AES_ENCRYPT('hello world','a'))
52B737BC51F201CF9DFCD274F47D108A

SELECT HEX(AES_ENCRYPT('hello world     ','a')) 
A1E41652BDD49562A17F26A2290238640EE48611167765B43EABB54FA548BFE1

PHP Mcrypt result:
"hello world" = 45e5b24a046c3cc5a2fb6a6fb753d01f 
"hello world     " = a1e41652bdd49562a17f26a229023864 (note nothing on the end of this here (but on MySQL there is), apparently the MySQL source code says if it is an even block, go ahead and pad it with a full block anyway (why?))

Suggested fix:
In line 153 of mysys/my_aes.c
CHANGE:
bfill(block + AES_BLOCK_SIZE - pad_len, pad_len, pad_len)
TO:
bfill(block + AES_BLOCK_SIZE - pad_len, pad_len, 0)

and the corresponding code in decrypt.  Also I don't know where it's best to do it (I'm not really a C programmer), but the extra padding block should probably not be there either.
[17 Oct 2005 21:45] Jim Winstead
Here's a document that explains the different padding methods, and why the one we use is generally recommended: http://www.di-mgt.com.au/cryptopad.html

It is also the method specified by RFC 2630.