| Bug #32991 | mysqlimport --use-threads test fails on solaris | ||
|---|---|---|---|
| Submitted: | 5 Dec 2007 9:32 | Modified: | 6 Aug 2009 23:54 |
| Reporter: | Alexander Nozdrin | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Command-line Clients | Severity: | S2 (Serious) |
| Version: | 5.1, 6.0 | OS: | Any |
| Assigned to: | Jim Winstead | CPU Architecture: | Any |
| Tags: | disabled, mysqldump.test, mysqlimport, pushbuild error, race condition | ||
[13 Feb 2008 14:07]
Magnus Blåudd
Experimented a little more with this and it is easily reproduced on sol10-sparc-a.
The problem seems to be that mysqlimport fsils to connect when run with --n um-threads>1, is it really built with the thread safe library?
To reproduce, build mysql-5.1 on sol10-sparc-a. Start the server with
./mtr mysqldump --start-and-exit
Create and fill tables:
mysql> create table t1 (a text , b text);
mysql> create table t2 (a text , b text);
mysql> insert t1 values ("Duck, Duck", "goose");
mysql> insert t1 values ("Duck, Duck", "pidgeon");
mysql> insert t2 values ("We the people", "in order to perform");
mysql> insert t2 values ("a more perfect", "union");
mysql> select * from t1;
mysql> select * from t2;
Run mysqldump to create t1.txt and t2. txt in var/tmp
../client/mysqldump -S var/tm p/master.socl k--tab=var/tmp/ test
Run mysqlimport continuously in a while loop.
$> while mysqlimport.sh ; do date; done
It will fail and report that mysqlimport couldn't connect to mysqld through socket
mysqlimport.sh would look something like:
../client/mysqlimport -uroot --debug-check --port=10170 --socket=/tmp/MfLgwlfAyM/master.sock --password= --silent --use-threads=2 test /export/home/mysqldev/users/magnus/mysql-5.1.24-rc-pb472/mysql-test/var/tmp/t1.txt /export/home/mysqldev/users/magnus/mysql-5.1.24-rc-pb472/mysql-test/var/tmp/t2.txt /export/home/mysqldev/users/magnus/mysql-5.1.24-rc-pb472/mysql-test/var/std_data_ln/words.dat /export/home/mysqldev/users/magnus/mysql-5.1.24-rc-pb472/mysql-test/var/std_data_ln/words2.dat $1
Posible problems:
1. Not using thread safe library. All clients in client/ is compiled with DUNDEF_THREADS_HACK, remove that defined and let each client include what they need to get either thread safe or non thread safe code.
2. The mysqlimport need a small retry loop when connecting - not so good, the clients should connect every time.
[13 Feb 2008 14:09]
Magnus Blåudd
"All clients in client/ is compiled with DUNDEF_THREADS_HACK," and then we go though hoops in order to get thread safety in the one that needs it.
[13 Feb 2008 16:37]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/42228 ChangeSet@1.2548, 2008-02-13 17:39:23+01:00, sven@riska.(none) +2 -0 BUG#32991: Races in mysqldump.test (or mysqldump.test fails sporadically) This is *not* a fix to the bug. I'm only disabling the failing part of mysqldump.test until the bug is fixed. Whoever fixes it, please re-enable the test.
[13 Feb 2008 17:25]
Sven Sandberg
The submitted patch does *not* fix the bug. It only disables the test case until the bug is fixed, so that not everyone gets errors in pushbuild. Whoever fixes this bug, please re-enable the test. The error is in mysqlimport, and happens when the --use-threads flag is used. I looked superficially at it (checked global variables used from worker_thread() in client/mysqlimport.c), and could not see anything strange. (Except that the global 'exitcode' variable is used without a mutex, but it should not cause this sort of error). The error log on sol10-sparc-a says "errno=23", which on this machine means "illegal seek". I'm just guessing, but could printf() be non-thread-safe on this platform?
[25 Mar 2008 11:23]
Bugs System
Pushed into 5.1.24-rc
[26 Mar 2008 19:01]
Bugs System
Pushed into 6.0.5-alpha
[2 May 2008 18:54]
Jim Winstead
Considering that there's a comment about the -DUNDEF_THREADS_HACK saying it a fix for MIT-pthreads, it can probably just be removed. Otherwise the same CFLAGS used for mysqlslap should be used for mysqlimport.
[2 May 2008 22:22]
Jim Winstead
Unset UNDEF_THREADS_HACK for mysqlimport, because it uses threads
Attachment: bug32991.patch (text/plain), 426 bytes.
[2 May 2008 22:22]
Jim Winstead
I can't reproduce the problem on sol10-sparc-a, but the attached patch will compile mysqlimport the same way as mysqlslap, to avoid the UNDEF_THREADS_HACK hack.
[3 Jul 2008 21:30]
Sveta Smirnova
Probably bug #37789 is related.
[7 Jan 2009 0:46]
Timothy Smith
Looks ok; I agree it would be nice to get rid of UNDEF_THREADS_HACK entirely, but maybe needs more research. This patch is simple.
[12 May 2009 17:46]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/73853 2894 Jim Winstead 2009-05-12 mysqlimport was not always compiled correctly to allow thread support, required for the --use-threads option. (Bug #32991) modified: client/Makefile.am mysql-test/r/mysqldump.result mysql-test/t/mysqldump.test
[13 Jul 2009 19:41]
Jim Winstead
Pushed to 5.1-bugteam and mysql-pe.
[4 Aug 2009 19:51]
Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090804194615-h40sa098mx4z49qg) (version source revid:jimw@mysql.com-20090713193847-xj6gbgm2a1zwwanv) (merge vers: 5.4.4-alpha) (pib:11)
[4 Aug 2009 20:45]
Bugs System
Pushed into 5.1.38 (revid:davi.arnaut@sun.com-20090804204317-ggodqkik7de6nfpz) (version source revid:davi.arnaut@sun.com-20090804204317-ggodqkik7de6nfpz) (merge vers: 5.1.38) (pib:11)
[6 Aug 2009 23:54]
Paul DuBois
Noted in 5.1.38, 5.4.4 changelogs. mysqlimport was not always compiled correctly to enable thread support, which is required for the --use-threads option.
[12 Aug 2009 22:59]
Paul DuBois
Noted in 5.4.2 changelog because next 5.4 version will be 5.4.2 and not 5.4.4.
[15 Aug 2009 2:16]
Paul DuBois
Ignore previous comment about 5.4.2.
[1 Oct 2009 5:59]
Bugs System
Pushed into 5.1.39-ndb-6.3.28 (revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (version source revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (merge vers: 5.1.39-ndb-6.3.28) (pib:11)
[1 Oct 2009 7:25]
Bugs System
Pushed into 5.1.39-ndb-7.0.9 (revid:jonas@mysql.com-20091001072547-kv17uu06hfjhgjay) (version source revid:jonas@mysql.com-20091001071652-irejtnumzbpsbgk2) (merge vers: 5.1.39-ndb-7.0.9) (pib:11)
[1 Oct 2009 13:25]
Bugs System
Pushed into 5.1.39-ndb-7.1.0 (revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (version source revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (merge vers: 5.1.39-ndb-7.1.0) (pib:11)
[5 Oct 2009 10:50]
Bugs System
Pushed into 5.1.39-ndb-6.2.19 (revid:jonas@mysql.com-20091005103850-dwij2dojwpvf5hi6) (version source revid:jonas@mysql.com-20090930185117-bhud4ek1y0hsj1nv) (merge vers: 5.1.39-ndb-6.2.19) (pib:11)
[9 Oct 2009 1:28]
Paul DuBois
The 5.4 fix has been pushed to 5.4.2.
Description: The following code in mysqldump.test sometimes causes failures: ------------------------------------------------------------ # # Added for use-thread option # create table t1 (a text , b text); create table t2 (a text , b text); insert t1 values ("Duck, Duck", "goose"); insert t1 values ("Duck, Duck", "pidgeon"); insert t2 values ("We the people", "in order to perform"); insert t2 values ("a more perfect", "union"); select * from t1; select * from t2; --exec $MYSQL_DUMP --tab=$MYSQLTEST_VARDIR/tmp/ test --exec $MYSQL test < $MYSQLTEST_VARDIR/tmp/t1.sql --exec $MYSQL test < $MYSQLTEST_VARDIR/tmp/t2.sql # The first load tests the pausing code --exec $MYSQL_IMPORT --use-threads=1 test $MYSQLTEST_VARDIR/tmp/t1.txt $MYSQLTEST_VARDIR/tmp/t2.txt # Now we test with multiple threads! --exec $MYSQL_IMPORT --silent --use-threads=5 test $MYSQLTEST_VARDIR/tmp/t1.txt $MYSQLTEST_VARDIR/tmp/t2.txt select * from t1; select * from t2; # Now we test with multiple threads, but less threads than files. create table words(a varchar(255)); create table words2(b varchar(255)); --exec $MYSQL_IMPORT --silent --use-threads=2 test $MYSQLTEST_VARDIR/tmp/t1.txt $MYSQLTEST_VARDIR/tmp/t2.txt $MYSQLTEST_VARDIR/std_data_ln/words.dat $MYSQLTEST_VARDIR/std_data_ln/words2.dat select * from t1; select * from t2; select * from words; select * from words2; ------------------------------------------------------------ The failures look like: ------------------------------------------------------------ main.mysqldump [ fail ] mysqldump: Couldn't find table: "non_existing" mysqldump: Got error: 1356: View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them when using LOCK TABLES mysqldump: Couldn't execute 'SHOW FIELDS FROM `v1`': View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them (1356) mysqldump: user2 has insufficent privileges to SHOW CREATE PROCEDURE `sp1`! --- /export/home/pushbuild/pb/mysql-5.2-maint/37/mysql-6.0.4-alpha-pb37/mysql-test/r/mysqldump.result Sat Dec 1 16:04:04 2007 +++ /export/home/pushbuild/pb/mysql-5.2-maint/37/mysql-6.0.4-alpha-pb37/mysql-test/r/mysqldump.reject Sun Dec 2 15:01:23 2007 @@ -4019,7 +4019,7 @@ aberrant aberration drop table words; -mysql-import: Error: 1146, Table 'test.words' doesn't exist, when using table: words +mysql-import: Error: 2002 Can't connect to local MySQL server through socket '/tmp/pbtmp-n_mix-100/master.sock' (146) drop table t1; drop table t2; drop table words2; mysqltest: Result content mismatch Stopping All Servers Restoring snapshot of databases Resuming Tests ------------------------------------------------------------ or ------------------------------------------------------------ main.mysqldump [ fail ] mysqldump: Couldn't find table: "non_existing" mysqldump: Got error: 1356: View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them when using LOCK TABLES mysqldump: Couldn't execute 'SHOW FIELDS FROM `v1`': View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them (1356) mysqldump: user2 has insufficent privileges to SHOW CREATE PROCEDURE `sp1`! /export/home/pushbuild/pb/mysql-5.1-runtime/586/mysql-5.1.23-rc-pb586/client/mysqlimport: Error: 2002 Can't connect to local MySQL server through socket '/tmp/pbtmp-n_row-5/master.sock' (146) mysqltest: At line 1636: command "$MYSQL_IMPORT --silent --use-threads=2 test $MYSQLTEST_VARDIR/tmp/t1.txt $MYSQLTEST_VARDIR/tmp/t2.txt $MYSQLTEST_VARDIR/std_data_ln/words.dat $MYSQLTEST_VARDIR/std_data_ln/words2.dat" failed The result from queries just before the failure was: < snip > a b We the people in order to perform a more perfect union test.t1: Records: 2 Deleted: 0 Skipped: 0 Warnings: 0 test.t2: Records: 2 Deleted: 0 Skipped: 0 Warnings: 0 select * from t1; a b Duck, Duck goose Duck, Duck pidgeon Duck, Duck goose Duck, Duck pidgeon select * from t2; a b We the people in order to perform a more perfect union We the people in order to perform a more perfect union create table words(a varchar(255)); create table words2(b varchar(255)); exec of '/export/home/pushbuild/pb/mysql-5.1-runtime/586/mysql-5.1.23-rc-pb586/client/mysqlimport -uroot --debug-check --port=10050 --socket=/tmp/pbtmp-n_row-5/master.sock --password= --silent --use-threads=2 test /tmp/var-n_row-5/tmp/t1.txt /tmp/var-n_row-5/tmp/t2.txt /tmp/var-n_row-5/std_data_ln/words.dat /tmp/var-n_row-5/std_data_ln/words2.dat' failed, error: 256, status: 1, errno: 29 More results from queries before failure can be found in /tmp/var-n_row-5/log/mysqldump.log Stopping All Servers Restoring snapshot of databases Resuming Tests ------------------------------------------------------------ How to repeat: https://intranet.mysql.com/secure/pushbuild/xref.pl?testname=main.mysqldump https://intranet.mysql.com/secure/pushbuild/xref.pl?plat=not+%25win2003%25&testname=main.m...