MySQL Bugs: #73958: Cant run unittest with Python3

Bug #73958	Cant run unittest with Python3
Submitted:	17 Sep 2014 21:32	Modified:	6 May 2022 15:03
Reporter:	Sandro Tosi	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	Connector / Python	Severity:	S3 (Non-critical)
Version:	1.2.3, 2.0.4	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
Hello,
when running unittest.py with py3k we got this error:

Traceback (most recent call last):
  File "unittests.py", line 807, in <module>
    main()
  File "unittests.py", line 699, in main
    init_mysql_server(port=(options.port), options=options)
  File "unittests.py", line 634, in init_mysql_server
    mysql_server.bootstrap()
  File "/tmp/buildd/mysql-connector-python-1.2.3/tests/mysqld.py", line 438, in bootstrap
    sql.extend([line.strip() for line in fp.readlines()])
  File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcf in position 880: ordinal not in range(128)

How to repeat:
.

Hi Sandro,

Which MySQL version is this exactly? I'm running with MySQL 5.6 and it works OK.
The SQL files it is reading contains something odd. Can you run unittests.py with --debug. It should say which file has problems.

(Note, Python 3.4 is not officially supported for Connector/Python v1.2, though I don't expect much trouble there..)

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

Hello,
sorry for the delay.

I'm using MySQL 5.5.39 (as packaged by Debian); with --debug I got

```
2014-11-06 11:22:22,688 [myconnpy_tests:DEBUG] Reading SQL from '/usr/share/mysql/fill_help_tables.sql'
Traceback (most recent call last):
  File "unittests.py", line 807, in <module>
    main()
  File "unittests.py", line 699, in main
    init_mysql_server(port=(options.port), options=options)
  File "unittests.py", line 634, in init_mysql_server
    mysql_server.bootstrap()
  File "/tmp/buildd/mysql-connector-python-1.2.3/tests/mysqld.py", line 438, in bootstrap
    sql.extend([line.strip() for line in fp.readlines()])
  File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcf in position 880: ordinal not in range(128)
```

I can repeat error message by putting non-ascii characters into some place of fill_help_tables.sql , e.g. Ï (0xC3) and save file in  utf-8 encoding.

Probably Debian puts some non-ascii into fill_help_tables.sql or that is environment-specific.

So workaround should be following:  make sure that these files are in ascii encoding:

mysql_system_tables.sql     
mysql_system_tables_data.sql
fill_help_tables.sql

I think such behavior is a minor bug and it would be nice if C/Py would be able to handle those scripts in UTF-8

indeed, /usr/share/mysql/fill_help_tables.sql is a unicode file:

$ file /usr/share/mysql/fill_help_tables.sql 
/usr/share/mysql/fill_help_tables.sql: UTF-8 Unicode text, with very long lines

this command will show the chars not printable in ascii:

grep --color='auto' -P -n '[^\x00-\x7F]'  /usr/share/mysql/fill_help_tables.sql

(there are several minus signs and a \pi sign)

i agree it would be nice c/py is able to handle it, as it currently is not running any unittests (at least on debian) for the py3k version

Testing with Debian 10 and 11, and using the latest supported MySQL 5.7 and 8.0 versions, this issue is not reproducible.
Maybe this is something specific to your environment.
If you feel this is reproducible in a general way, feel free to reopen the bug and we can follow up.