MySQL Bugs: #60481: mysql misprocesses lines longer than MAX_BATCH_BUFFER

Bug #60481	mysql misprocesses lines longer than MAX_BATCH_BUFFER_SIZE
Submitted:	15 Mar 2011 21:56	Modified:	25 Mar 2011 18:54
Reporter:	[ name withheld ]	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: Command-line Clients	Severity:	S3 (Non-critical)
Version:	5.0.80, 5.1.57, 5.6.3	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	Contribution

Description:
The patch committed for bug #41486 has two serious errors, which will manifest when trying to read lines close to or exceeding MAX_BATCH_BUFFER_SIZE. First, batch_readline() supposes that it can null-terminate the returned line by scribbling on the buffer. This was okay in the previous coding, where it could only have been overwriting the buffer's existing null terminator byte or the no-longer-interesting newline terminating the current line. It is not okay when we have truncated the line, because the first byte of the to-be-scanned remainder of the line is lost. Second, the code in intern_read_line() imagines that it can break the line wherever it darn well pleases, but breaking immediately following a backslash is no good because then the code in mysql.cc will fail to process the backslash escape correctly.

I observed this problem in 5.0.92, but so far as I can see the 5.1 and 5.5 branches contain the same bug. It's been band-aided by someone who evidently couldn't work out the real problem but instead increased the value of MAX_BATCH_BUFFER_SIZE so that typical cases wouldn't encounter it. That is a poor solution however, because it opens the command-line clients to failing on long input lines, if they are being run under restrictive ulimit.

How to repeat:
Reduce MAX_BATCH_BUFFER_SIZE to some realistic value, and then try to process scripts containing lines longer than that. Observe syntax errors and/or lost data bytes due to missing characters in the lines sent to the server.

Suggested fix:
Will attach proposed patch. In addition to this patch I'd suggest reverting MAX_BATCH_BUFFER_SIZE down to 1MB or so.

proposed patch

Attachment: mysql-readline.patch (application/octet-stream, text), 2.30 KiB.

Thank you for the report.

I can not repeat error if reduce size of MAX_BATCH_BUFFER_SIZE. Please provide how-to instructions to cause wrong behavior.

You can reproduce a failure in the regular regression tests, if you set MAX_BATCH_BUFFER_SIZE small enough.  For example, setting it to 100 like this:

diff -aur mysql-5.5.10.orig/client/mysql.cc mysql-5.5.10/client/mysql.cc
--- mysql-5.5.10.orig/client/mysql.cc	2011-03-09 10:11:40.000000000 -0500
+++ mysql-5.5.10/client/mysql.cc	2011-03-25 11:05:15.318726104 -0400
@@ -110,7 +110,7 @@
 #define PROMPT_CHAR '\\'
 #define DEFAULT_DELIMITER ";"
 
-#define MAX_BATCH_BUFFER_SIZE (1024L * 1024L * 1024L)
+#define MAX_BATCH_BUFFER_SIZE (100L)
 
 typedef struct st_status
 {

gets me the following failures in mysql 5.5.10:

Failing test(s): main.mysql-bug45236 binlog.binlog_old_versions perfschema.pfs_upgrade main.mysqldump binlog.binlog_tmp_table

Obviously nobody would have it that small in practice; this is just an easy way to demonstrate that the code fails when the input line is longer than the buffer size.

Thank you for the feedback.

Verified as described.