MySQL Bugs: #35413: NDBAPI : NdbInterpretedCode 2*buffer extension may not be enough

Bug #35413	NDBAPI : NdbInterpretedCode 2*buffer extension may not be enough
Submitted:	18 Mar 2008 17:39	Modified:	9 Jul 2008 8:57
Reporter:	Frazer Clement	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: NDB API	Severity:	S2 (Serious)
Version:	5.1-6.3	OS:	Any
Assigned to:	Frazer Clement	CPU Architecture:	Any

Description:
Recent changes to NdbInterpretedCode implemented an automatically extending internal buffer which doubles in size when the current buffer length is insufficient for adding the next instruction.

Column comparison instructions have length proportional to the maximum length of the column, and can require more extra space than double the current buffer size.  Currently the code writes past the end of the allocated buffer, which can result in a SEGV.

How to repeat:
1) Create a table with a non-indexed, large VARCHAR.
   e.g. CREATE TABLE t1 (a int primary key, b VARCHAR(5000)) engine ndb;
2) Perform a SELECT which will result in condition pushdown on this VARCHAR
   e..g SELECT * FROM t1 WHERE b="Some string";
3) Max length of b should be sufficient to require buffer extension, but buffer extension will be insufficient, and it is likely that the server will get a SEGV.

Suggested fix:
Fix automatic extension code to keep doubling new buffer size until it is sufficient for the next instruction, or exceeds the maximum length.
Add assertions.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/44387

ChangeSet@1.2536, 2008-03-25 13:26:15+00:00, frazer@forth.ndb.mysql.com +6 -0
  Bug#35413
  Fix to NdbInterpretedCode dynamic buffer extension

Documented in the 5.1.23-ndb-6.3.11 changelog as follows:

        A SELECT on a table with a non-indexed, large VARCHAR column which
        resulted in condition pushdown on this column could cause mysqld to
        crash.

Left status as Patch Pending waiting for additional merges.

telco-6.3 fix already documented, appears to affect no other versions; closed.

Pushed into 6.0.6-alpha  (revid:sp1r-frazer@forth.ndb.mysql.com-20080325132615-55038) (version source revid:sp1r-tomas@poseidon.ndb.mysql.com-20080516085603-30848) (pib:5)