Bug #86579 Table data export wizard is VERY SLOW exporting "large" datasets

Submitted: 4 Jun 2017 13:42 Modified: 4 Jun 2017 14:44
Reporter: Lee Butler Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Workbench Severity:S3 (Non-critical)
Version:6.3.9 OS:Windows (10)
Assigned to: CPU Architecture:Any
Tags: export wizard, workbench

[4 Jun 2017 13:42] Lee Butler
Description:
Workbench is very slow exporting large datasets through the CSV export wizard. Disproportionately slow comapred to a smaller set. However, this is something I've come across before with .NET.

How to repeat:
Get a table with 15k or so records or more, and export through the wizard. Note how long it takes and then export a subset of that data and see how the time taken does not correlate linearly with the amount of rows. 

Suggested fix:
Something I've noticed when building CSV export applications is that the MS .NET framework can't deal with huge strings very well, and tends to perform badly as a result.

I found a solution though. When building up the huge string to the dump into the file when you've done generating it, instead of building 1 huge string and writing it to file all at once when the export is done, I get much better performance by only doing a few hundred rows of CSV generated at a time, write it to the file and flush the buffer you have been writing the generated data to. 

I'd recommend writing to a temp file, then rename/move it to the user's specified one when done. The Write to temp and then move/rename is the way Photoshop and some other applications save their data. And the writing x rows and flushing I've found when developing myself is much faster than trying to get .NET to manage a 20MB string.
[4 Jun 2017 14:44] MySQL Verification Team
Thank you for the bug report. Duplicate/related to https://bugs.mysql.com/bug.php?id=86430.

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.048 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.