Bug #9982 addition of ORDER BY to a query corrupts the output from the query
Submitted: 18 Apr 2005 21:12 Modified: 27 Jan 2006 16:43
Reporter: Trevor Marshall Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:4.0.24 OS:Microsoft Windows (Windows 2000)
Assigned to: CPU Architecture:Any

[18 Apr 2005 21:12] Trevor Marshall
Description:
When running a query, if ORDER BY is part of the query, the result set of the query can become corrupted - either duplicating previous results, or containing garbage results (possibly the results of other queries where the result is in a different format?).

The problem originally occurred with a PHPBB system which had been running for several months without problems, then, for no apparent reason, the PHPBB pages had nonsense values for lists of who was logged on.  I've tried upgrading PHPBB, upgrading MySQL from 4.0.22 to 4.0.24 but the problem persists.  I've tried a clean install of MySQL on a different machine, and isolated a minimal database that exhibits the problem.

This appears to be similar to bug #3563.

How to repeat:
I've tried a clean install of MySQL 4.0.24 on a different Windows PC, and then
ran in the following file using

mysql -u root -p < tmtest3.sql

Here's the contents of tmtest3.sql:

---------------------------------%<------------------------------------

--
DROP DATABASE IF EXISTS `tmtest`;
CREATE DATABASE `tmtest`;
USE tmtest;

-- --------------------------------------------------------

-- 
-- Table structure for table `phpbb_sessions`
-- 

DROP TABLE IF EXISTS `phpbb_sessions`;
CREATE TABLE IF NOT EXISTS `phpbb_sessions` (
  `session_id` char(32) NOT NULL default '',
  `session_user_id` mediumint(8) NOT NULL default '0',
  `session_start` int(11) NOT NULL default '0',
  `session_time` int(11) NOT NULL default '0',
  `session_ip` char(8) NOT NULL default '',
  `session_page` int(11) NOT NULL default '0',
  `session_logged_in` tinyint(1) NOT NULL default '0',
  PRIMARY KEY  (`session_id`),
  KEY `session_user_id` (`session_user_id`),
  KEY `session_id_ip_user_id` (`session_id`,`session_ip`,`session_user_id`)
) TYPE=MyISAM;

-- 
-- Dumping data for table `phpbb_sessions`
-- 

INSERT INTO `phpbb_sessions` VALUES ('8e245ee4780bbcc7e93107a95cc062ad', -1,
1113297782, 1113299824, 'ac140f66', 0, 0);
INSERT INTO `phpbb_sessions` VALUES ('cae531234975c0955d45e65feeb744f1', 10,
1113299233, 1113299294, 'c01b79d3', 0, 1);
INSERT INTO `phpbb_sessions` VALUES ('d527a3c78b78bb8fa745f9fc1c68725f', -1,
1113297536, 1113297536, 'c01b79d3', 0, 0);
INSERT INTO `phpbb_sessions` VALUES ('3934ff002b70828e616baeb1026efb99', -1,
1113297529, 1113297529, 'c01b79d3', 0, 0);
INSERT INTO `phpbb_sessions` VALUES ('4be3602ee497e87ba499f590bdd2a580', -1,
1113301675, 1113301675, 'c01b79d3', 0, 0);

-- --------------------------------------------------------

-- 
-- Table structure for table `phpbb_users`
-- 

DROP TABLE IF EXISTS `phpbb_users`;
CREATE TABLE IF NOT EXISTS `phpbb_users` (
  `user_id` mediumint(8) NOT NULL default '0',
  `user_active` tinyint(1) default '1',
  `username` varchar(25) NOT NULL default '',
  `user_password` varchar(32) NOT NULL default '',
  `user_session_time` int(11) NOT NULL default '0',
  `user_session_page` smallint(5) NOT NULL default '0',
  `user_lastvisit` int(11) NOT NULL default '0',
  `user_regdate` int(11) NOT NULL default '0',
  `user_level` tinyint(4) default NULL,
  `user_posts` mediumint(8) unsigned NOT NULL default '0',
  `user_timezone` decimal(5,2) NOT NULL default '0.00',
  `user_style` tinyint(4) default NULL,
  `user_lang` varchar(255) default NULL,
  `user_dateformat` varchar(14) NOT NULL default 'd M Y H:i',
  `user_new_privmsg` smallint(5) unsigned NOT NULL default '0',
  `user_unread_privmsg` smallint(5) unsigned NOT NULL default '0',
  `user_last_privmsg` int(11) NOT NULL default '0',
  `user_emailtime` int(11) default NULL,
  `user_viewemail` tinyint(1) default NULL,
  `user_attachsig` tinyint(1) default NULL,
  `user_allowhtml` tinyint(1) default '1',
  `user_allowbbcode` tinyint(1) default '1',
  `user_allowsmile` tinyint(1) default '1',
  `user_allowavatar` tinyint(1) NOT NULL default '1',
  `user_allow_pm` tinyint(1) NOT NULL default '1',
  `user_allow_viewonline` tinyint(1) NOT NULL default '1',
  `user_notify` tinyint(1) NOT NULL default '1',
  `user_notify_pm` tinyint(1) NOT NULL default '0',
  `user_popup_pm` tinyint(1) NOT NULL default '0',
  `user_rank` int(11) default NULL,
  `user_avatar` varchar(100) default NULL,
  `user_avatar_type` tinyint(4) NOT NULL default '0',
  `user_email` varchar(255) default NULL,
  `user_icq` varchar(15) default NULL,
  `user_website` varchar(100) default NULL,
  `user_from` varchar(100) default NULL,
  `user_sig` text,
  `user_sig_bbcode_uid` varchar(10) default NULL,
  `user_aim` varchar(255) default NULL,
  `user_yim` varchar(255) default NULL,
  `user_msnm` varchar(255) default NULL,
  `user_occ` varchar(100) default NULL,
  `user_interests` varchar(255) default NULL,
  `user_actkey` varchar(32) default NULL,
  `user_newpasswd` varchar(32) default NULL,
  PRIMARY KEY  (`user_id`),
  KEY `user_session_time` (`user_session_time`)
) TYPE=MyISAM;

-- 
-- Dumping data for table `phpbb_users`
-- 

INSERT INTO `phpbb_users` VALUES (-1, 0, 'Anonymous', '', 0, 0, 0, 1099926690,
0, 5, 0.00, NULL, '', '', 0, 0, 0, NULL, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, NULL,
'', 0, '', '', '', '', '', NULL, '', '', '', '', '', '', '');
INSERT INTO `phpbb_users` VALUES (2, 1, 'administrator',
'9dd4e461268c8034f5c8564e155c67a6', 1113300886, 0, 1113299180, 1099926690, 1, 1,
0.00, 1, 'english', 'd M Y h:i a', 0, 0, 1113228988, NULL, 1, 0, 0, 1, 1, 1, 1,
1, 0, 1, 1, 1, '', 0, 'ss@cc.c', '', '', '', '', '', '', '', '', '', '', '',
'');
INSERT INTO `phpbb_users` VALUES (10, 1, 'trevorm',
'9dd4e461268c8034f5c8564e155c67a6', 1113299294, 0, 1113235227, 1100014577, 0,
13, 0.00, 1, 'english', 'D M d, Y g:i a', 0, 0, 1107439856, NULL, 0, 0, 0, 1, 1,
1, 1, 0, 0, 0, 1, NULL, 'Avatar_Pack_1/102.gif', 3,
'trevor.marshall@thales-tts.com', '', '', '', '', '', '', '', '', '', '', '',
NULL);

-- --------------------------------------------------------

--
-- Query with no ORDER BY - is OK
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id;

--
-- Repeat
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id;

--
-- Repeat
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id;

--
-- Repeat
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id;

--
-- Repeat
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id;

--
-- Repeat with ORDER BY
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id
ORDER BY u.user_id;

--
-- Repeat with ORDER BY
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id
ORDER BY u.user_id;

--
-- Repeat with ORDER BY
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id
ORDER BY u.user_id;

--
-- Repeat with ORDER BY
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id
ORDER BY u.user_id;

--
-- Repeat with ORDER BY
--

SELECT u.user_id, s.session_user_id
FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id
ORDER BY u.user_id;

---------------------------------%<------------------------------------

Since this file contains the database structure, data and queries it should be possible to reproduce the problem using just this.

Thanks in advance for any suggestions

Suggested fix:
I can remove the 'ORDER BY' clause from most of the PHPBB code, but that seems like the wrong way to fix it.
[19 Apr 2005 0:11] Miguel Solorzano
I wasn't able to repeat the behavior reported. Could you please
provide your my.ini file ?

Thanks in advance.
[19 Apr 2005 10:34] Trevor Marshall
OK, I don't have a 'my.ini' file, but I do have a 'C:\WINNT\my.cnf'. I may have misunderstood the installation documentation, but it looks to me that for older versions (<4.1.5) the file was 'my.cnf'.  If that's not the case then that may be where the problem lies.  Anyway, we do have a 'my.cnf, it looks like it was just copied from one of the examples, but here it is anyway:

# Example MySQL config file for medium systems.
#
# This is for a system with little memory (32M - 64M) where MySQL plays
# an important part, or systems up to 128M where MySQL is used together with
# other programs (such as a web server)
#
# You can copy this file to
# /etc/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options (in this
# installation this directory is /usr/local/var) or
# ~/.my.cnf to set user-specific options.
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.

# The following options will be passed to all MySQL clients
[client]
#password	= your_password
port		= 3306
socket		= /tmp/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
port		= 3306
socket		= /tmp/mysql.sock
skip-locking
key_buffer = 16M
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
myisam_sort_buffer_size = 8M

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
# 
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
log-bin

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id	= 1

# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
#    the syntax is:
#
#    CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
#    MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
#    where you replace <host>, <user>, <password> by quoted strings and
#    <port> by the master's port number (3306 by default).
#
#    Example:
#
#    CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
#    MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
#    start replication for the first time (even unsuccessfully, for example
#    if you mistyped the password in master-password and the slave fails to
#    connect), the slave will create a master.info file, and any later
#    change in this file to the variables' values below will be ignored and
#    overridden by the content of the master.info file, unless you shutdown
#    the slave server, delete master.info and restart the slaver server.
#    For that reason, you may want to leave the lines below untouched
#    (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id       = 2
#
# The replication master for this slave - required
#master-host     =   <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user     =   <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password =   <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port     =  <port>
#
# binary logging - not required for slaves, but recommended
#log-bin

# Point the following paths to different dedicated disks
#tmpdir		= /tmp/		
#log-update 	= /path-to-dedicated-directory/hostname

# Uncomment the following if you are using BDB tables
#bdb_cache_size = 4M
#bdb_max_lock = 10000

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /usr/local/var/
#innodb_data_file_path = ibdata1:10M:autoextend
#innodb_log_group_home_dir = /usr/local/var/
#innodb_log_arch_dir = /usr/local/var/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 16M
#innodb_additional_mem_pool_size = 2M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 5M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

Thanks again.
[20 Apr 2005 9:15] Trevor Marshall
MySQL Server Configuration file

Attachment: my.cnf (application/octet-stream, text), 4.70 KiB.

[20 Apr 2005 9:16] Trevor Marshall
I've added 'my.cnf' as a file attached to this bug, hope that helps.
[20 Apr 2005 19:27] Miguel Solorzano
Thank you for the feedback, however I wasn't able to repeat the
behavior reported.
[21 Apr 2005 8:29] Trevor Marshall
Thanks for trying.  However, I still have the problem, and I can repeat it here at will.  The problem occurs on at least two of our company Windows PCs.  I've tried the same data on my home Linux system and there's no problem there, so it's possible that the problem is outside MySQL.  The phpBB system that showed up the problem ran fine until about ten days ago, and then started showing some garbage data; when I tried the minimal database (as posted here) on my office PC the problem showed up immediately.

Just to add some more information:  the additional data in the result of the query seems to accumulate from when the MySQL server is started.  You can stop the client and restart it, and the additional data is still there.  We had a power outage yesterday and all the computers restarted, and the first query from the example with ORDER BY after the restart was fine.  The second query had one extra set of results, the third two extra sets and so on.  Results here:

----------------------------------%<-----------------------------------

mysql> select u.user_id, s.session_user_id FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id ORDER BY u.user_id
    -> ;
+---------+-----------------+
| user_id | session_user_id |
+---------+-----------------+
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      10 |              10 |
+---------+-----------------+
5 rows in set (0.03 sec)

mysql> select u.user_id, s.session_user_id FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id ORDER BY u.user_id;
+---------+-----------------+
| user_id | session_user_id |
+---------+-----------------+
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      10 |              10 |
|      10 |              10 |
+---------+-----------------+
10 rows in set (0.00 sec)

mysql> select u.user_id, s.session_user_id FROM phpbb_users u, phpbb_sessions s
WHERE u.user_id = s.session_user_id ORDER BY u.user_id;
+---------+-----------------+
| user_id | session_user_id |
+---------+-----------------+
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      -1 |              -1 |
|      10 |              10 |
|      10 |              10 |
|      10 |              10 |
+---------+-----------------+
15 rows in set (0.00 sec)

----------------------------------%<-----------------------------------

It looks very like a buffer not being flushed between queries.

I understand that you can't repeat the problem, but if you've any suggestions for anything else I can try, either to gather more data or try a fix then I'll give it a go.

Thanks again for all the help so far.
[26 Apr 2005 23:17] Sergei Golubchik
Can you show the output of 'SHOW VARIABLES' please ?
[27 Apr 2005 15:03] Trevor Marshall
output from SHOW VARIABLES

Attachment: sqlvars.txt (text/plain), 3.51 KiB.

[27 Apr 2005 15:05] Trevor Marshall
I've uploaded the output from SHOW VARIABLES as 'sqlvars.txt' attached to this bug.

Hope that helps.
[31 May 2005 16:19] Jorge del Conde
I was unable to reproduce this bug with 4.0.23, 24 & 25bk under both, Win32 & Linux.
[27 Jan 2006 16:43] Trevor Marshall
I think I now know what caused this.  Looks like mixing MySQL with Cygwin requires a bit more care.  

A colleague had the same problem, but only after installing Cygwin on the PC. After some investigation they found running 
mysql-nt --help 
showed "cygwin stuff" in the value for tmpdir.  Starting mysqld as 
mysql-nt --standalone --console --tmpdir=C:\tmp 
fixed it.  

So on my computer I added
tmpdir=C:\Temp
(where C:\Temp actually exists)
and ORDER BY works fine now.  Taking the 'cygwin stuff' out of tmpdir certainly seems to fix the problem - and would account for why we had the problem on our boxes - nearly all of which have Cygwin installed - and nobody else could repeat it.

So I guess I can close this bug now.