Bug #95700 SQL worbench is unable to import UTF8 csv file
Submitted: 9 Jun 17:08 Modified: 10 Jun 6:11
Reporter: Jirka Stejskal Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Workbench: Administration Severity:S2 (Serious)
Version:8.0.16 OS:Mac OS X (macOS 10.14.x Mojave x86_64)
Assigned to: CPU Architecture:Any
Tags: WBBugReporter

[9 Jun 17:08] Jirka Stejskal
Description:
inmporting UTF8 csv file will show unhandled exception:

UnicodeEncodeError: 'ascii' codec can't encode character u'\ufeff' in position 0: ordinal not in range(128)
18:53:10 [ERR][       pymforms]: Unhandled exception in Python code: 
Traceback (most recent call last):
  File "/Applications/MySQLWorkbench.app/Contents/Resources/libraries/workbench/wizard_page_widget.py", line 97, in go_next
    self.main.go_next_page()
  File "/Applications/MySQLWorkbench.app/Contents/Resources/libraries/workbench/wizard_form.py", line 76, in go_next_page
    self.pages[index].page_activated(True)
  File "/Applications/MySQLWorkbench.app/Contents/Resources/plugins/sqlide_power_import_wizard.py", line 186, in page_activated
    self.call_create_preview_table()
  File "/Applications/MySQLWorkbench.app/Contents/Resources/plugins/sqlide_power_import_wizard.py", line 344, in call_create_preview_table
    self.create_preview_table(self.call_analyze())
  File "/Applications/MySQLWorkbench.app/Contents/Resources/plugins/sqlide_power_import_wizard.py", line 353, in call_analyze
    if not self.active_module.analyze_file():
  File "/Applications/MySQLWorkbench.app/Contents/Resources/plugins/sqlide_power_import_export_be.py", line 537, in analyze_file
    self.has_header = csv.Sniffer().has_header(csvsample)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 399, in has_header
    header = rdr.next() # assume first row is header
Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

from the text it is obvious, that Workbench is not handling UTF8 signature bytes at the beggining of the CSV file and crashes.

How to repeat:
create excel with any international characters (I can give you some, but I don't see a way to attach it here)
save the excel as UTF8 encoded csv file (latest Excel)
try to import the content.
the procedure will show the dialog with text: "Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?"

Suggested fix:
simple - properly handle the file with UTF8 signature bytes (u'\ufeff') at the beggining of the file.
[9 Jun 17:12] Jirka Stejskal
file to recreate the problem

Attachment: test_csv.csv (text/csv), 610 bytes.

[10 Jun 6:11] Umesh Shastry
Hello Jirka Stejskal,

Thank you for the report.

regards,
Umesh