Bug #27265 strange sanity check in void taocrypt HexDecoder::Decode()
Submitted: 19 Mar 2007 17:45 Modified: 7 Sep 2007 12:54
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: General Severity:S3 (Non-critical)
Version:5.0BK OS:Any (*)
Assigned to: Magnus Blåudd CPU Architecture:Any
Tags: taocrypt

[19 Mar 2007 17:45] Shane Bester
Description:
when looking at the hex decode function  HexDecoder::Decode() in extra\yassl\taocrypt\src\coding.cpp we have this:

const byte bad = 0xFF;  // invalid encoding

<cut>

const byte hexDecode[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
                           bad, bad, bad, bad, bad, bad, bad,
                           10, 11, 12, 13, 14, 15 
                         };  // A starts at 0x41 not 0x3A

<cut>

// Hex Decode
void HexDecoder::Decode()
{
    <cut>

    while (bytes) {
        byte b  = coded_.next() - 0x30;  // 0 starts at 0x30
        byte b2 = coded_.next() - 0x30;

        // sanity checks
        assert( b  < sizeof(hexDecode)/sizeof(hexDecode[0]) );
        assert( b2 < sizeof(hexDecode)/sizeof(hexDecode[0]) );
        assert( b != bad && b2 != bad );

        b  = hexDecode[b];
        b2 = hexDecode[b2];
        
<cut>
    }
<cut>
}

the line "assert( b != bad && b2 != bad );" looks wrong.

How to repeat:
read the code

Suggested fix:
looks like they meant to do this:

// sanity checks
assert( b  < sizeof(hexDecode)/sizeof(hexDecode[0]) );
assert( b2 < sizeof(hexDecode)/sizeof(hexDecode[0]) );

b  = hexDecode[b];
b2 = hexDecode[b2];
assert( b != bad && b2 != bad );
[21 Mar 2007 9:08] Valeriy Kravchuk
Thank you for a bug report. Verified just as described on latest 5.0.40-BK by code review.
[4 May 2007 8:52] Magnus Blåudd
Update of /cvsroot/yassl/yassl/taocrypt/src
In directory sc8-pr-cvs10.sourceforge.net:/tmp/cvs-serv11257

Modified Files:
	coding.cpp 
Log Message:
fix bad assertion

Index: coding.cpp
===================================================================
RCS file: /cvsroot/yassl/yassl/taocrypt/src/coding.cpp,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** coding.cpp	15 Mar 2007 02:15:16 -0000	1.5
--- coding.cpp	3 May 2007 17:31:49 -0000	1.6
***************
*** 116,123 ****
          assert( b  < sizeof(hexDecode)/sizeof(hexDecode[0]) );
          assert( b2 < sizeof(hexDecode)/sizeof(hexDecode[0]) );
-         assert( b != bad && b2 != bad );
  
          b  = hexDecode[b];
          b2 = hexDecode[b2];
          
          decoded_[i++] = (b << 4) | b2;
--- 116,124 ----
          assert( b  < sizeof(hexDecode)/sizeof(hexDecode[0]) );
          assert( b2 < sizeof(hexDecode)/sizeof(hexDecode[0]) );
  
          b  = hexDecode[b];
          b2 = hexDecode[b2];
+ 
+         assert( b != bad && b2 != bad );
          
          decoded_[i++] = (b << 4) | b2;
[28 Aug 2007 8:36] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/33198

ChangeSet@1.2503, 2007-08-28 10:35:55+02:00, msvensson@pilot.(none) +15 -0
  Import yaSSL version 1.7.2
  - Fix bug#27265
  - Support for fixing bug#18441
[4 Sep 2007 17:11] Bugs System
Pushed into 5.1.23-beta
[4 Sep 2007 17:13] Bugs System
Pushed into 5.0.50
[7 Sep 2007 12:54] Paul DuBois
No user-visible effects here. No changelog entry needed.