Bug #75730 TypeError: unhashable type: 'bytearray'
Submitted: 2 Feb 2015 18:46 Modified: 8 May 2015 9:43
Reporter: Paul Kreker Email Updates:
Status: Verified Impact on me:
None 
Category:Connector / Python Severity:S2 (Serious)
Version:2.0.3, 2.0.4 OS:Any
Assigned to: CPU Architecture:Any
Tags: Django

[2 Feb 2015 18:46] Paul Kreker
Description:
I just created a new project and followed the tutorial, when i got this:

(env) paul@Kreker-Server:~/public_html/p_kreker$ python manage.py migrate
Operations to perform:
  Apply all migrations: auth, sessions, contenttypes, admin
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying sessions.0001_initial... OK
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/core/management/commands/migrate.py", line 165, in handle
    emit_post_migrate_signal(created_models, self.verbosity, self.interactive, connection.alias)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/core/management/sql.py", line 268, in emit_post_migrate_signal
    using=db)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/dispatch/dispatcher.py", line 198, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/contrib/auth/management/__init__.py", line 64, in create_permissions
    if not is_latest_migration_applied('auth'):
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/db/migrations/loader.py", line 292, in is_latest_migration_applied
    loader = MigrationLoader(connection)
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/db/migrations/loader.py", line 49, in __init__
    self.build_graph()
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/db/migrations/loader.py", line 184, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/home/paul/public_html/p_kreker/env/lib/python3.4/site-packages/django/db/migrations/recorder.py", line 60, in applied_migrations
    return set(tuple(x) for x in self.migration_qs.values_list("app", "name"))
TypeError: unhashable type: 'bytearray'

How to repeat:
Use Python3.4 to create django1.74 project
create a virtual environment with "pyvenv /path/to/new/virtual/environment"
source venv
run ./manage.py migrate
[4 Feb 2015 23:26] Paul Kreker
Version 2.0.3 still available
[4 Feb 2015 23:29] Paul Kreker
changed OS from Ubuntu to Any
[5 Feb 2015 12:25] Peeyush Gupta
I'm not able to reproduce this, can you share your settings.py and other scripts you used.
[12 Feb 2015 20:12] Piotr Jurkiewicz
I have just run into exactly the same bug.

My web application was working fine. mysql.connector was translating utf8 varchar columns to Python's unicode() objects as it should.

Then I changed column collation type from utf8_general_ci to the utf8_bin. After this change, mysql.connector started to return this same column as bytearray() instead of unicode(), what broke my application.

I suspect that the same bug is described here: http://stackoverflow.com/questions/27566078/how-return-str-from-mysql-using-mysql-connecto...

In my opinion mysql.connector should return unicode() objects for all utf8 columns, no matter what collation is set.
[29 May 2015 16:03] Ed D
This bug occurs because the cursor returns bytearray for any column that that has a binary collation regardless of the connection settings.  The column descriptions currently take precedence to connection settings in the conversion back to python strings.  

Here's a simple example with the mysql.User table:

from mysql.connector import connect
from mysql.connector.cursor import MySQLCursorNamedTuple

connection = connect(user = 'user', passwd = 'pass')
cursor = connection.cursor(cursor_class=MySQLCursorNamedTuple)
cursor.execute("select * from mysql.user limit 1")
result = cursor.fetchone()
cursor.close()

print type(result.User)
print type(result.Select_priv)
foo = {result.User: 'foo'}

Table definition for my version of mysql 5.5:
CREATE TABLE `user` (
  `Host` char(60) COLLATE utf8_bin NOT NULL DEFAULT '',
  `User` char(16) COLLATE utf8_bin NOT NULL DEFAULT '',
  `Password` char(41) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL DEFAULT '',
  `Select_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Insert_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Update_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Delete_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Drop_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Reload_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Shutdown_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Process_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `File_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Grant_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `References_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Index_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Alter_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Show_db_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Super_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_tmp_table_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Lock_tables_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Execute_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Repl_slave_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Repl_client_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Show_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Alter_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_user_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Event_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Trigger_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `Create_tablespace_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
  `ssl_type` enum('','ANY','X509','SPECIFIED') CHARACTER SET utf8 NOT NULL DEFAULT '',
  `ssl_cipher` blob NOT NULL,
  `x509_issuer` blob NOT NULL,
  `x509_subject` blob NOT NULL,
  `max_questions` int(11) unsigned NOT NULL DEFAULT '0',
  `max_updates` int(11) unsigned NOT NULL DEFAULT '0',
  `max_connections` int(11) unsigned NOT NULL DEFAULT '0',
  `max_user_connections` int(11) unsigned NOT NULL DEFAULT '0',
  `plugin` char(64) COLLATE utf8_bin DEFAULT '',
  `authentication_string` text COLLATE utf8_bin,
  PRIMARY KEY (`Host`,`User`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Users and global privileges'

dsc passed to MysqlConversion::_STRING_to_python:
(u'User', 254, None, None, None, None, 0, 16515)
(u'Select_priv', 254, None, None, None, None, 0, 257)

The only workaround right now is to cast known problematic columns to unicode after fetching results
[29 Jul 2015 15:29] Stefan Brozinski
Another workaround is patching the driver. Please be aware that this patch has not been extensively tested and I am not sure what side effects it may have. Use at your own risk.

--- conversion.py       2015-07-29 17:23:11.409097600 +0200
+++ /cygdrive/c/Temp/site-packages/mysql/connector/conversion.py    2015-07-29 17:24:03.485695500 +0200
@@ -545,8 +545,6 @@
             # Check if we deal with a SET
             if dsc[7] & FieldFlag.SET:
                 return self._SET_to_python(value, dsc)
-            if dsc[7] & FieldFlag.BINARY:
-                return value

         if self.charset == 'binary':
             return value
[7 Apr 2017 18:06] monty solomon
This still fails in version 2.1.5.

What are the plans to fix it?