Bug #107259 Character Set 'utf8mb3' is not supported by .Net Framework
Submitted: 11 May 9:26 Modified: 19 Sep 16:53
Reporter: Matthew Khew Email Updates:
Status: No Feedback Impact on me:
None 
Category:Connector / NET Severity:S1 (Critical)
Version:8.0.29 OS:Windows
Assigned to: CPU Architecture:Any
Tags: .Net application crash, mysql upgrade issue to 8.0.29

[11 May 9:26] Matthew Khew
Description:
My .NET web application is working fine with MySQL version 8.0.28 database.

After upgrading to MySQL version 8.0.29, I encountered a MySQLException error when trying to connect and read data from a table. 

The error message is "Character Set 'utf8mb3' is not supported by .Net Framework".

Nothing changes at the application side, only upgraded the MySQL version.

How to repeat:
1) Create .Net application using C# to connect to MySQL server and read data from a table.

2) .Net Framework version 4.5.1

3) MySql.Data.MySqlClient MySql.Data.Entity.EF6, Version=6.9.9.0

4) MySQL version 8.0.29
[18 May 12:42] MySQL Verification Team
Hello Matthew Khew,

Thank you for the bug report.
I tried to reproduce your issue on windows 10 with MySQL 8.0.29 but I am not seeing any issues at my end.
Could you please provide repeatable test case(please make it as private if you prefer) to reproduce this issue at our end? 

Regards,
Ashwini Patil
[19 May 3:51] Chris Smith
I can confirm this issue as part of 8.0.29

Using MySQL:8.0 (which is presently the 8.0.29 version) docker image, I was able to spin up a base mysql instance and then when attempting to run .NET code (RoundhousE, a .NET database deployment tool) on the database to apply schema data changes, I received the following error:

could not complete schema update
System.NotSupportedException: Character set 'utf8mb3' is not supported by .Net Framework.
   at MySql.Data.MySqlClient.CharSetMap.GetCharacterSet(DBVersion version, String charSetName)
   at MySql.Data.MySqlClient.MySqlField.SetFieldEncoding()
   at MySql.Data.MySqlClient.MySqlField.set_CharacterSetIndex(Int32 value)
   at MySql.Data.MySqlClient.NativeDriver.GetColumnData(MySqlField field)
   at MySql.Data.MySqlClient.NativeDriver.GetColumnsData(MySqlField[] columns)
   at MySql.Data.MySqlClient.Driver.GetColumns(Int32 count)
   at MySql.Data.MySqlClient.ResultSet.LoadColumns(Int32 numCols)

Upon using the MySQL:8.0.28 instance of the docker container, the above .NET code ran to completion successfully.

I attempted to run this:
1. In a Windows command prompt
2. A Powershell 7.2.4 shell
3. Ubuntu 20 LTS WSL2 Bash

dotnet SDK version is 6.0.105
 

I can provide additional details / files, as requested.
[23 May 7:06] Patryk Krysik
I use Net Framework 4.5.2, MySQL Connector Net 8.0.29, MySQL for Visual Studio 1.2.8 (I try to load tables to Visual Studio 2013). The error I get is "Character set 'utf8mb3' is not supported by .Net Framework". I did convert every table to utf8mb4, the issue still subsists. It doesn't help to manually request for utf8mb4 in connection details. Every version before that worked perfectly. For now I switched for ODBC Connector but it's not perfect.
[17 Jun 18:57] Bill Karwin
The default character_set_system is still utf8mb3. This is hard-coded; it is not configurable.

mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.29    |
+-----------+

mysql> select @@character_set_system;
+------------------------+
| @@character_set_system |
+------------------------+
| utf8mb3                |
+------------------------+

The system character set is used in certain fixed places, like the default character set reported for INFORMATION_SCHEMA:

mysql> show create schema information_schema\G
*************************** 1. row ***************************
       Database: information_schema
Create Database: CREATE DATABASE `information_schema` /*!40100 DEFAULT CHARACTER SET utf8mb3 */ /*!80016 DEFAULT ENCRYPTION='N' */

If a client (like .NET apparently) does some metadata checks when it connects, it could get confused by the new name for utf8. 

Ideally, the next step of migrating MySQL Server 8.0 to utf8mb4 should be to change the system character set, and any other variables or schemas that use the old character set.
[19 Jun 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[19 Jun 20:10] Chris Smith
Is there any information on this bug?  It seems the software you use has automatically suspended the issue, but no one has responded in any way as to if it is a.) noted on your side and b.) if it is in the queue to be fixed.

Please advise

Cheers
[19 Jun 20:12] Chris Smith
I will also point out that while your software tells us that it is suspending this issue from lack of activity, and tells us we can keep the issue going by setting its status back to 'Open', there is no means by which we can set it to 'Open' (or at least there is no such means on my end.  I'm not sure if the original person who reported this can do so)
[20 Jun 8:28] Patryk Krysik
@Chris Smith, as @Bill Karwin mentioned, it looks like the problem is @@character_set_system in information_schema. I do not have required permissions to change that since I'm using hosting service so I'm waiting for admin to reply. I'll add a comment if that change will help (or if they will refuse to change it). For now I just use ODBC driver and I suggest for you to do the same. It's not perfect but it's better than nothing.
[20 Jun 13:35] Patryk Krysik
It looks like neither @@character_set_system and information_schema can be altered. Now we can only wait for mysql team to fix it.
[20 Jun 21:49] Chris Smith
@Patryk Krysik, thanks for the fast reply.  Yeah, I noted the same myself.

As previous versions did not have this issue, for now I've rolled back to an earlier incarnation and will hold fire there until we see how this changes / gets resolved.

Cheers
[3 Jul 16:56] Adam Bogart
Same experience as the others.  I created a new 8.0.29 version of mySQL (on AWS/RDS) and cannot connect due to the issue of the .NET framework not recognizing utf8mb3.  I now have to delete the instance and revert to a previous version. I'll remain on the older version as long as possible because .NET connectivity is business critical. 

This is highly concerning because this bugs breaks the tools I use regularly to access the database. If this doesn't get resolved, it will become catastrophic as we move towards the newer versions over time.
[4 Jul 11:08] MySQL Verification Team
Hello Matthew,

Thank you for the report and feedback.
Verified as described.

Regards,
Ashwini Patil
[4 Jul 11:09] MySQL Verification Team
8.0.29 test results

Attachment: 107259_test_results.PNG (image/png, text), 54.78 KiB.

[5 Jul 20:37] Michael Caditz
Same problem here. I installed MySQL Server 8.0.29 and my .net framework 4.8 application is broken.

Amy idea when a fix can be expected?
[5 Jul 20:37] Michael Caditz
Same problem here. I installed MySQL Server 8.0.29 and my .net framework 4.8 application is broken.

Amy idea when a fix can be expected?
[12 Jul 4:38] Matthew Khew
Dear Ashwini Patil (MySQL Verification Team),

Great to know that you able to simulate the error in your response on 4 Jul 2022.
Please advice, the estimated release date of next patching to address this issue.

Thank you very much.

Regards,
Matthew
[27 Jul 20:23] emad Mahayni
Hi,
I tested today MySQL 8.0.30 and the problem still exists.
"Character set 'utf8mb3' is not supported by .Net Framework."

I wonder when this will be fixed?

Best Regards
Emad Mahayni
[5 Aug 20:56] Jordan Smith
Hey, 

We've been following this, as it's been affecting one of our applications with the same error. Are there any updates on this, please? 

Thanks,
Jord
[19 Aug 14:53] Daniel Valdez
Hi,

Can you please share more information regarding your issue, e.g. MySQL Server
version, Connector/NET version, database schema?
It would help a lot to have a reproducible scenario.

Thanks,
Daniel
[26 Aug 9:11] Patryk Krysik
Didn't you just upload screen of the error message we all had? What details do you need? Again: Net Framework 4.5.2, MySQL Connector Net 8.0.29, MySQL for Visual Studio 1.2.8, it happens even on newly created database with conversion to utf8mb4.
[26 Aug 14:43] Daniel Valdez
Hi Patryk Krysik,

Have you tried using Connector/NET 8.0.30? I'm asking you this because there was a fix in that release that was related to the character mappings.
And also it would very helpful to know the version of the MySQL Server.

Thanks,
Daniel
[1 Sep 10:35] Keso BIBO
8.0.30 also has this problem
[1 Sep 17:00] Daniel Valdez
Hi @Keso BIBO

Do you have more details, like if you are using EF, EFCore, a simple .NET app, MySQL Server version, etc? I'm trying to figure it out cause it can be possible that this issue is somehow related to the server. There was an important change regarding Character Support (https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-30.html#mysqld-8-0-30-charset:~:t...).

All the cases provided above were using past releases, so if you could provide us a test case, that would really helpful for us in order to fix any possible issue.

Thanks!
[1 Sep 18:24] Patryk Krysik
I confirm the problem still occurs at 8.0.30 version.
[1 Sep 18:25] Patryk Krysik
I will try to provide a test case in the next week.
[1 Sep 18:53] Daniel Valdez
Thanks, Patryk Krysik. That would be really appreciated.
[2 Sep 5:35] Dani Kaplan
I've upgraded to 8.0.30 and I've got the same issue as well. This is pretty serious.
[20 Sep 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".