MySQL Bugs: #53190: Huge sript loading

Bug #53190	Huge sript loading
Submitted:	27 Apr 2010 8:54	Modified:	21 May 2010 12:28
Reporter:	Alexey Puzankov	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Workbench	Severity:	S3 (Non-critical)
Version:	5.2.18, 5.2.19	OS:	Windows (XP)
Assigned to:	Sergei Tkachenko	CPU Architecture:	Any

Description:
I have tries load a script which contain more than 56K lines of inserts

How to repeat:
open db model
add script
laod from file

Thank you for the report.

I asusme you are using MySQL Workbench. What happened when you load this script?

out of memory exception

Attachment: mysql_workbanch_error.jpg (image/jpeg, text), 40.45 KiB.

first type of errors is in attached screenshot.
second one is the fillowing:
can't allocate 67108864 bytes to read file: "<file name>".
At the same time, real filesize is 33MB

Thank you for the feedback.

Verified as described.

33M is pretty big, but shouldn't be enough to eat up all system memory. 
For this bug I think its enough to catch the OutOfMem exception and cancel the script loading, showing a warning. Long term the mem usage should be investigated.

by the way, huge script has been loaded well (I seen whole text). I can run it after closing error box :-)

I just loaded a 95K lines script without any problem, but that was only 6MB in size. If the script contains very long lines, however, then the editor might have trouble to handle that (say, longer than 2000 chars). But then it is a problem of the editor control, which we cannot fix.

Sveta, how did you verify this report? Do you have a file that behaves like reported?

Could you please attach a script that produced the error described?

Inefficient EOL conversion was the cause of memory exhaustion. SQL Editor was enhanced to support custom EOL formats and now preserves original line endings used in the file. EOL conversion isn't used any more and loading of big files can be done much faster.

fix confirmed in repository

An entry has been added to the 5.2.22 changelog:

When a large script was loaded into the SQL Editor, an out of memory exception occurred.

Inefficient end of line conversion in MySQL Workbench was the cause of memory exhaustion. The SQL Editor has been enhanced to support custom end of line formats and now preserves original line endings used in the file. End of line conversion is not used any more and loading of big files can be done faster and more efficiently.

Bug #53127 was marked as duplicate of this one.