| Bug #45058 | init_available_charsets uses double checked locking | ||
|---|---|---|---|
| Submitted: | 25 May 2009 3:52 | Modified: | 19 Jul 2010 20:14 |
| Reporter: | Mark Callaghan | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server | Severity: | S3 (Non-critical) |
| Version: | 5.0, 5.1 | OS: | Any |
| Assigned to: | Staale Smedseng | CPU Architecture: | Any |
| Tags: | CHECKED, double, init_available_charsets, locking | ||
[25 May 2009 3:52]
Mark Callaghan
[25 May 2009 6:05]
Sveta Smirnova
Thank you for the report.
The paper shows the problem if one uses object which method going to return as flag:
Singleton* Singleton::instance() {
if (pInstance == 0) { // 1st test
Lock lock;
if (pInstance == 0) { // 2nd test
pInstance = new Singleton;
}
}
return pInstance;
}
while MySQL uses separate flag charset_initialized which sets to 1 after all work is done. Why do you think MySQL code affected by the same problem?
[25 May 2009 6:33]
Jonas Oreland
the code needs a memory barrier before the flag is set,
error= my_read_charset_file(fname,myflags);
>> mb();
charset_initialized=1;
this is a problem, as a CPU might(will) reorder writes to memory,
which can be prevented by a memory barrier
(and since we probably can't enforce a read-protocol on the reader,
it likely will have to be a full barrier)
an portable(but expensive) way of making a memory barrier to to lock/unlock
a mutex
[25 Jun 2009 23:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[25 Jun 2009 23:11]
Mark Callaghan
Move the charset_initialized assignment after pthread_mutex_unlock
[29 Jun 2009 10:11]
Sveta Smirnova
Mark, Jonas, thank you for the feedback. Verified as described.
[27 Jul 2009 21:12]
Davi Arnaut
The proposed solution just goes from one gray scenario to another. AFAIR, pthread makes no guarantees with respect to variables modified outside of the of the lock. Also, whether a memory barrier is issued, it is implementation specific. As others have mentioned, this whole optimization seems unwarranted. It would perhaps be better to just initialize it once at server start.
[11 Sep 2009 17:24]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/83079 3120 Staale Smedseng 2009-09-11 Bug #45058 init_available_charsets uses double checked locking As documented in the bug report, the double checked locking pattern has inherent issues, and cannot guarantee correct initialization. This patch modifies the charset initialization code to do its work during server/client initialization, and not through calls to get_charset_*(). This is achieved by splitting init_available_charsets() into two functions: * load_compiled_charsets(): loads the compiled character sets, called from my_init(), * load_available_charsets(): loads charset index file after any options regarding charset directory have been processed; called explicitly in mysqld.cc, mysqldump.cc a.o., and implicitly in clients through mysql_library_init(). The split is necessary to support functions relying on charsets that are used early on, while giving the possibility of changing the source charsets_dir (e.g., during options parsing) before loading charset definitions. @ client/mysqldump.c Adding explicit loading of charset info from file. @ extra/charset2html.c Adding explicit loading of charset info from file. @ extra/comp_err.c Adding explicit loading of charset info from file. @ include/my_sys.h init_available_charsets() is split into two functions. @ libmysql/libmysql.c Adding explicit loading of charset info from file. @ mysys/charset.c init_available_charsets() is split into two functions. Calls to init_available_charsets() are dropped from the get_charset*() functions. @ mysys/my_init.c Adding explicit loading of compiled charsets in my_init(). @ netware/libmysqlmain.c Charset loading now happens in my_init() and mysql_server_init(). @ sql/mysqld.cc Adding explicit loading of charset info from file.
[16 Oct 2009 14:54]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/87152 3153 Staale Smedseng 2009-10-16 Bug #45058 init_available_charsets uses double checked locking As documented in the bug report, the double checked locking pattern has inherent issues, and cannot guarantee correct initialization. This is a new patch with an alternative solution, intended to avoid any possible API incompatibilities introduced in the initial patch. This patch replaces the logic in init_available_charsets() with the use of pthread_once(3). A wrapper function, my_pthread_once(), is introduced and is used in lieu of direct calls to init_available_charsets(). For the Windows platform, the implementation in lp:sysbench is ported. For single-thread use, a simple define calls the function and sets the pthread_once control variable. @ include/my_no_pthread.h Dummy my_pthread_once() for single thread use. @ include/my_pthread.h Declaration for new function my_pthread_once(). @ mysys/charset.c Logic in init_available_charsets() is simplified. Using my_pthread_once() for all calls to this func.
[19 Dec 2009 8:29]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091219082307-f3i4fn0tm8trb3c0) (version source revid:alik@sun.com-20091216180721-eoa754i79j4ssd3m) (merge vers: 6.0.14-alpha) (pib:15)
[19 Dec 2009 8:33]
Bugs System
Pushed into 5.5.1-m2 (revid:alik@sun.com-20091219082021-f34nq4jytwamozz0) (version source revid:alexey.kopytov@sun.com-20091216134707-o96eqw0u2ynvo9gm) (merge vers: 5.5.0-beta) (pib:15)
[19 Dec 2009 8:37]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20091219082213-nhjjgmphote4ntxj) (version source revid:alik@sun.com-20091216180221-a5ps59gajad3pip9) (pib:15)
[8 Jan 2010 21:24]
Paul DuBois
Noted in 5.5.1, 6.0.14 changelogs. It was possible for init_available_charsets() not to initialize correctly. Setting report to NDI pending push to Celosia.
[15 Jan 2010 8:58]
Bugs System
Pushed into 5.1.43 (revid:joro@sun.com-20100115085139-qkh0i0fpohd9u9p5) (version source revid:staale.smedseng@sun.com-20091212181125-w4deewgjviz548i0) (merge vers: 5.1.42) (pib:16)
[15 Jan 2010 18:21]
Paul DuBois
Noted in 5.1.43 changelog. Setting report to NDI pending push to Celosia.
[20 Feb 2010 5:15]
[ name withheld ]
According to the discussion at https://bugzilla.redhat.com/show_bug.cgi?id=566547 as well as several reports linked from there, the version of this patch pushed into 5.1.43 does not work. It breaks akonadi and probably other applications. I have not tried to trace the logic changes in detail but it looks to me like there is a code path wherein the charset info doesn't get initialized.
[20 Feb 2010 15:52]
Bartosz Fabianowski
Using MySQL 5.5.1, KDE's akonadi fails in the same way as with MySQL 5.1.43. So this change must have introduced problems in the 5.5 branch as well. Here is a backtrace: #0 0x0000000802f77344 in my_strcasecmp_8bit () from /usr/local/lib/mysql/libmysqlclient_r.so.16 #1 0x0000000802f6bd8b in get_collation_number () from /usr/local/lib/mysql/libmysqlclient_r.so.16 #2 0x0000000802f6bdd0 in get_charset_by_name () from /usr/local/lib/mysql/libmysqlclient_r.so.16 #3 0x0000000802f7f4f2 in mysql_init_character_set () from /usr/local/lib/mysql/libmysqlclient_r.so.16 #4 0x0000000802f81650 in mysql_real_connect () from /usr/local/lib/mysql/libmysqlclient_r.so.16 #5 0x0000000802e0ef66 in QMYSQLDriver::open () from /usr/local/lib/qt4/plugins/sqldrivers/libqsqlmysql.so #6 0x0000000800f0ba61 in QSqlDatabase::open () from /usr/local/lib/qt4/libQtSql.so.4 #7 0x000000080076d55c in Akonadi::DataStore::open (this=0x802c7df80) at /usr/ports/databases/akonadi/work/akonadi-1.3.1/server/src/storage/datastore.cpp:87 #8 0x000000080076da9d in DataStore (this=0x802c7df80) at /usr/ports/databases/akonadi/work/akonadi-1.3.1/server/src/storage/datastore.cpp:64 #9 0x000000080076db7c in Akonadi::DataStore::self () at /usr/ports/databases/akonadi/work/akonadi-1.3.1/server/src/storage/datastore.cpp:140 #10 0x0000000800790cbb in Akonadi::CacheCleaner::run (this=0x8028aa340) at /usr/ports/databases/akonadi/work/akonadi-1.3.1/server/src/cachecleaner.cpp:43 #11 0x00000008009ad793 in QThreadPrivate::start () from /usr/local/lib/qt4/libQtCore.so.4 #12 0x0000000801cc32e1 in pthread_getprio () from /lib/libthr.so.3 #13 0x0000000000000000 in ?? () Cannot access memory at address 0x7fffff9fe000
[20 Feb 2010 19:18]
Rex Dieter
This fix seems to have caused problems initializing akonadi(mysql) databases used in kde. Errors reported include: [akonadiserver] Character set 'latin1' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file [akonadiserver] Database error: Cannot open database. See also downstream reports, http://bugs.kde.org/show_bug.cgi?id=226960 https://bugzilla.redhat.com/show_bug.cgi?id=566547 In particular, Tom Lane has come up with the minimal set of changes, that if reverted, makes the problem go away wrt akonadi initialization, https://bugzilla.redhat.com/show_bug.cgi?id=566547#c11 Please advise.
[22 Feb 2010 23:16]
Robin Johnson
Another ping for reopening this please. The patch on the RedHat bug is a working fix, verified by others on FreeBSD. The patch is now applied/available to users in Fedora (FC12/13) and Gentoo.
[22 Feb 2010 23:37]
Davi Arnaut
The patch is a reversal, it does not explain the problem. Anyone knows what the problem is?
[22 Feb 2010 23:55]
Robin Johnson
@Davi: See the RedHat and KDE reports, that note when akonadi tries to start mysqld, mysqld crashes with: Character set 'latin1' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file Despite the latin1 being defined in there, and the files for it existing.
[23 Feb 2010 0:13]
Davi Arnaut
I saw that. I was hoping for a explanation on why it breaks.
[23 Feb 2010 0:24]
Robin Johnson
As to what akonadi is doing to cause a break, I wish I knew. Not a KDE user here, just the MySQL maintainer @ Gentoo. @Rex Dieter: Do you have the parameters that akonadi has to start the mysqld instance?
[26 Feb 2010 13:37]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/101654 3358 Staale Smedseng 2010-02-26 Bug #45058 init_available_charsets uses double checked locking A client doing multiple mysql_library_init() and mysql_library_end() calls over the lifetime of the process may experience lost character set data, potentially even a SIGSEGV. This patch reinstates the reloading of character set data when a mysql_library_init() is done after a mysql_library_end().
[26 Feb 2010 14:30]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/101668 3362 Staale Smedseng 2010-02-26 Bug #45058 init_available_charsets uses double checked locking A client doing multiple mysql_library_init() and mysql_library_end() calls over the lifetime of the process may experience lost character set data, potentially even a SIGSEGV. This patch reinstates the reloading of character set data when a mysql_library_init() is done after a mysql_library_end().
[1 Mar 2010 20:25]
Rex Dieter
Based on some preliminary testing, I can confirm that http://lists.mysql.com/commits/101654 appears to fix the akonadi/charset issue reported here. Thank you.
[2 Mar 2010 14:34]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100302142746-u1gxdf5yk2bjrq3e) (version source revid:alik@sun.com-20100301095421-4cz64ibem1h2quve) (merge vers: 6.0.14-alpha) (pib:16)
[2 Mar 2010 20:55]
Paul DuBois
Setting report to Need Merge pending push to Celosia.
[5 Mar 2010 10:28]
Alex Dupre
I think the committed patch is wrong. You are trying to fake the pthread_once behavior, but actually you cannot reset the charsets_initialized variable. It probably works on Linux, because PTHREAD_ONCE_INIT is defined as '0', but on other systems it could be different (for example on FreeBSD) and fails miserably.
[5 Mar 2010 10:38]
Davi Arnaut
We know and tried to workaround it: http://lists.mysql.com/commits/101737
[12 Mar 2010 14:10]
Bugs System
Pushed into 5.1.44-ndb-7.0.14 (revid:jonas@mysql.com-20100312135944-t0z8s1da2orvl66x) (version source revid:jonas@mysql.com-20100312115609-woou0te4a6s4ae9y) (merge vers: 5.1.44-ndb-7.0.14) (pib:16)
[12 Mar 2010 14:25]
Bugs System
Pushed into 5.1.44-ndb-6.2.19 (revid:jonas@mysql.com-20100312134846-tuqhd9w3tv4xgl3d) (version source revid:jonas@mysql.com-20100312060623-mx6407w2vx76h3by) (merge vers: 5.1.44-ndb-6.2.19) (pib:16)
[12 Mar 2010 14:39]
Bugs System
Pushed into 5.1.44-ndb-6.3.33 (revid:jonas@mysql.com-20100312135724-xcw8vw2lu3mijrhn) (version source revid:jonas@mysql.com-20100312103652-snkltsd197l7q2yg) (merge vers: 5.1.44-ndb-6.3.33) (pib:16)
[12 Mar 2010 17:42]
Paul DuBois
Fixed in earlier 5.1.x, 5.5.x.
[26 Mar 2010 8:21]
Bugs System
Pushed into 5.5.4-m3 (revid:alik@sun.com-20100326080914-2pz8ns984e0spu03) (version source revid:alexey.kopytov@sun.com-20100307164059-cri8typa32cypq0l) (merge vers: 5.5.3-m2) (pib:16)
[26 Mar 2010 8:26]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100326081116-m3v4l34yhr43mtsv) (version source revid:alik@sun.com-20100325072612-4sds00ix8ajo1e84) (pib:16)
[6 Apr 2010 7:57]
Bugs System
Pushed into 5.1.46 (revid:sergey.glukhov@sun.com-20100405111026-7kz1p8qlzglqgfmu) (version source revid:joro@sun.com-20100301084434-ytctk3ceebjvqo7a) (merge vers: 5.1.45) (pib:16)
[17 Jun 2010 11:54]
Bugs System
Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[17 Jun 2010 12:32]
Bugs System
Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100609211156-tsac5qhw951miwtt) (merge vers: 5.1.46-ndb-6.2.19) (pib:16)
[17 Jun 2010 13:19]
Bugs System
Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[20 Jul 2010 15:34]
Domas Mituzas
This is still broken - a multithreaded mysql app breaks just fine with latest 5.1.49-bzr: $ while true; do env LD_LIBRARY_PATH=/usr/local/mysql-5.1/lib/mysql/ ./pmysql "SET @a=1" < servers; done Character set 'latin1' is not a compiled character set and is not specified in the '/usr/local/mysql-5.1/share/mysql/charsets/Index.xml' file Character set 'latin1' is not a compiled character set and is not specified in the '/usr/local/mysql-5.1/share/mysql/charsets/Index.xml' file ** (process:6969): WARNING **: Could not connect to 10..........: Can't initialize character set latin1 (path: /usr/local/mysql-5.1/share/mysql/charsets/) ** (process:6969): WARNING **: Could not connect to 10..........: : Can't initialize character set latin1 (path: /usr/local/mysql-5.1/share/mysql/charsets/) Segmentation fault Segmentation fault *** glibc detected *** ./pmysql: munmap_chunk(): invalid pointer: 0x00002aaaac024720 *** Segmentation fault Character set 'latin1' is not a compiled character set and is not specified in the '/usr/local/mysql-5.1/share/mysql/charsets/Index.xml' file Character set 'latin1' is not a compiled character set and is not specified in the '/usr/local/mysql-5.1/share/mysql/charsets/Index.xml' file Character set 'latin1' is not a compiled character set and is not specified in the '/usr/local/mysql-5.1/share/mysql/charsets/Index.xml' file ** (process:8630): WARNING **: Could not connect to 10..........: : Can't initialize character set latin1 (path: /usr/local/mysql-5.1/share/mysql/charsets/) ** (process:8630): WARNING **: Could not connect to 10..........: : Can't initialize character set latin1 (path: /usr/local/mysql-5.1/share/mysql/charsets/) ** (process:8630): WARNING **: Could not connect to 10...........: : Can't initialize character set latin1 (path: /usr/local/mysql-5.1/share/mysql/charsets/)
[20 Jul 2010 16:01]
Domas Mituzas
Disregard my previous comment. libmysqlclient was used instead of libmysqlclient_r... As I was calling mysql_thread_init(), assertion/warning/etc when using not-thread-safe client would have been especially helpful! I'll open a feature-request for API about that.
[29 Aug 2016 8:56]
Chandrakanth Nayak N
hi, In my code I am using mysql_library_init() function before calling any mysql library functions. And similarly closing by using mysql_library_end(). But some time we see below error Character set 'latin1' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file Can't initialize character set latin1 (path: /usr/share/mysql/charsets/) I am not sure what causing this problem. Is this fix merged in below mysql version. This one causing lot of problem in our environment. OS : centos 6.2 MySQL version: mysql> SHOW VARIABLES LIKE "%version%"; +-------------------------+---------------------+ | Variable_name | Value | +-------------------------+---------------------+ | protocol_version | 10 | | version | 5.1.52-log | | version_comment | Source distribution | | version_compile_machine | x86_64 | | version_compile_os | unknown-linux-gnu | +-------------------------+---------------------+ Even after upgrading to centos 6.8 we are seeing charset latin1 error. OS : centos 6.8 MySQL version : 5.1.73-log.
