MySQL Bugs: #7677: Make test (test suite) wierdness in 4.1.8a

Bug #7677	Make test (test suite) wierdness in 4.1.8a
Submitted:	5 Jan 2005 5:01	Modified:	29 Sep 2008 20:59
Reporter:	Brian Wright	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	MySQL Server	Severity:	S3 (Non-critical)
Version:	4.1.8a	OS:	Linux (Fedora Core 3)
Assigned to:		CPU Architecture:	Any

Description:
Pardon if this bug has been reported before, I searched but didn't find anything similar.  

So, it compiles ok.  Then, I run 'make test' and it runs through 262 tests successfully:

> make test
cd mysql-test; ./mysql-test-run && ./mysql-test-run --ps-protocol
Installing Test Databases
Removing Stale Files
Installing Master Databases
running  ../sql/mysqld --no-defaults --bootstrap --skip-grant-tables     --basedir=. --datadir=./var/master-data --skip-innodb --skip-ndbcluster --skip-bdb --language=../sql/share/english/ --character-sets-dir=../sql/share/charsets/
Installing Slave Databases
running  ../sql/mysqld --no-defaults --bootstrap --skip-grant-tables     --basedir=. --datadir=./var/slave-data --skip-innodb --skip-ndbcluster --skip-bdb     --language=../sql/share/english/ --character-sets-dir=../sql/share/charsets/
Manager disabled, skipping manager start.
Loading Standard Test Databases
Starting Tests
...
...
Ending Tests
Shutting-down MySQL daemon

Master shutdown finished
Slave shutdown finished
All 262 tests were successful.
...

And then it runs the test suite again (immediately following the successful test), this time failing miserably:

...
Installing Test Databases
Removing Stale Files
Installing Master Databases
running  ../sql/mysqld --no-defaults --bootstrap --skip-grant-tables     --basedir=. --datadir=./var/master-data --skip-innodb --skip-ndbcluster --skip-bdb --language=../sql/share/english/ --character-sets-dir=../sql/share/charsets/
Installing Slave Databases
running  ../sql/mysqld --no-defaults --bootstrap --skip-grant-tables     --basedir=. --datadir=./var/slave-data --skip-innodb --skip-ndbcluster --skip-bdb     --language=../sql/share/english/ --character-sets-dir=../sql/share/charsets/
Manager disabled, skipping manager start.
Loading Standard Test Databases
Starting Tests

TEST                            RESULT
-------------------------------------------------------
alias                          [ fail ]

Errors are (from /home/bwright/mysql-4.1.8a/mysql-test/var/log/mysqltest-time) :/home/bwright/mysql-4.1.8a/client/.libs/mysqltest: At line 88: Result length mismatch
(the last lines may be the most important ones)
Below are the diffs between actual and expected results:
-------------------------------------------------------
*** r/alias.result      2004-12-14 17:43:29.000000000 +0300
--- r/alias.reject      2005-01-05 07:24:49.932875827 +0300
***************
*** 48,53 ****
--- 48,54 ----
  KEY kundentyp(kundentyp),
  KEY p_nr(p_nr,suffix)
  ) ENGINE=MyISAM;
+ Warnings:
  INSERT INTO t1 VALUES (3359356,405,3359356,'Mustermann Musterfrau',52500,'2000-05-20','workflow','Auftrag erledigt','Originalvertrag eingegangen und gepr<ffff>ft','','privat',1485525,2122316,'+','','N',1909160,'MobilComSuper92000D2',NULL,NULL,'MS9ND2',3,24,'MobilCom Shop Koeln',52500,NULL,'auto',20010202105916,'Mobilfunk','PP','','','');
  INSERT INTO t1 VALUES (3359357,468,3359357,'Mustermann Musterfrau',7001,'2000-05-20','workflow','Auftrag erledigt','Originalvertrag eingegangen und gepr<ffff>ft','','privat',1503580,2139699,'+','','P',1909171,'MobilComSuper9D1T10SFreisprech(Akquise)',NULL,NULL,'MS9NS1',327,24,'MobilCom Intern',7003,NULL,'auto',20010202105916,'Mobilfunk','PP','','','');
  INSERT INTO t1 VALUES (3359358,407,3359358,'Mustermann Musterfrau',7001,'2000-05-20','workflow','Auftrag erledigt','Originalvertrag eingegangen und gepr<ffff>ft','','privat',1501358,2137473,'N','','N',1909159,'MobilComSuper92000D2',NULL,NULL,'MS9ND2',325,24,'MobilCom Intern',7003,NULL,'auto',20010202105916,'Mobilfunk','PP','','','');
-------------------------------------------------------
Please follow the instructions outlined at
http://www.mysql.com/doc/en/Reporting_mysqltest_bugs.html 
to find the reason to this problem and how to report this.

Aborting: alias failed. To continue, re-run with '--force'.

Ending Tests
Shutting-down MySQL daemon

Master shutdown finished
Slave shutdown finished
make: *** [test] Error 1
....

So, what should I make of this?  Was it supposed to run the test suite twice?  I've never seen this before.  Apparently, it compiled and at least tested correctly for the first run of the test suite.  Any ideas?

Thanks.

How to repeat:
1) Uncompress the mysql-4.1.8a.tar.gz
2) ./configure --prefix=/usr/local/mysql4
3) make
4) make test
5) watch it fail?

Hi,

Thank you for the report, but I can't repeat it. Ran the test suite twice and both were successful.

So the question is, then, should I be concerned about this failure?  Is this a problem with the test suite or of the database?  Should it be ok to run the database even though the test suite failed in this way.

Also, it now a normal part of the test suite to run it twice?  I hadn't seen this before?  if so, is there any specific difference between the first run of the test suite and the second?

From the behavior of the failure, I have a sneaking suspicion that it's a racing condition failure (i.e., the previous database cleanup and shutdown hadn't completed when the next test suite started).  So, it couldn't bind to the port and/or socket it wanted.  If it had failed two or three tests into the suite, I wouldn't say this.  But, considering it was the very first test in the second run indicates that it likely had problems starting and/or initializing the database for the tests.

Thanks.

Definitely not a racing condition.  I cd to mysql-test and type './mysql-test-run --ps-protocol' and I get the same exact failure every time.  But, without the --ps-protocol argument, the tests pass.

Thanks.

More specifics about the OS I'm running:
# uname -a
Linux dbhost 2.6.9-1.667 #1 Tue Nov 2 14:41:25 EST 2004 i686 i686 i386 GNU/Linux
# more /etc/redhat-release
Fedora Core release 3 (Heidelberg)

I haven't added any external packages to the OS other than those that come from up2date (bug fixes, kernel and os updates).  Although, it may not be 100% up2date.

Brian, this is no OS-related bug.

You may be aware that in addition to the standard protocol between client and server, MySQL is now implementing a new protocol called 'prepared statements'. This is still on-going work and not yet operational for all kinds of statements.
If the client running the tests is started with the option '--ps-protocol', it will use this new protocol. Now we need to test this, and therefore the Makefile target 'test' has been modified to run both tests. You are seeing the effect of this change and the on-going work, and that should not have happened.

If you just want to get rid of this effect, you can change the top level 'Makefile':
For the target 'test', remove the new part ' && ./mysql-test-run --ps-protocol'
This will return the old behaviour: Just one run, using the traditional protocol.

Please report whether that is an acceptable solution for you.

It now a normal part of the test suite to run it twice - it uses different API's in these runs. You did not see it before because --ps-protocol switch was implemented only recently.

The test failure that you see probably means that in one very specific case mysqld reports non-zero amount of warnings while really there are no warnings (and SHOW WARNINGS shows nothing), or that mysqltest forgets to clear warning flag from the previous query. I cannot see how it can happen and would be happy to have a repeatable test case.

Anyway for all practical purposes this is absolutely harmless, so you don't have to be concerned.

For the moment, we will not be using prepared statements.  We will, however, evaluate the use of these statements to determine if they will work in our environment for implementation at a later date.  But, as of right now, using the standard protocol will work fine.  We're in the process of upgrading from 3.x to 4.1.x.  Our 3.x environment has been nearly rock stable since we installed it.  So, I'm just trying to do everything to ensure that our upgrade goes as smoothly as possible (i.e., by testing this version of mysql thoroughly).

I am quite satisifed that the standard protocol version will function correctly for our needs based on the tests (at least, phase 1 of upgrading).   I still have phase 2, which is putting it into a test environment and running it through its paces.  If we decide to start using the prepared statements version later, we will have to revisit this issue.  But, that won't be for sometime yet.

Thanks.

Given the customer reaction, I set it to 'to be fixed later' as we know that the PS protocol is ongoing work. Also, 'showstopper' is cleared.

Setting a correct status.
A fix for this bug is "unalias make"