Bug #23924 general_log truncates queries with character set introducers.
Submitted: 3 Nov 2006 9:25 Modified: 27 Mar 2008 19:12
Reporter: Alexander Barkov Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Logging Severity:S3 (Non-critical)
Version:5.1.15-BK OS:Linux (Linux)
Assigned to: Alexander Barkov CPU Architecture:Any

[3 Nov 2006 9:25] Alexander Barkov
Description:
Queries having character set introducers are
truncated when written to tabke "mysql.general_log"

How to repeat:
1. Start "mysqld --log" to force logging into table "mysql.general_log"

2. Then run: 

   mysql -uroot mysql < bug.sql

   (bug.sql attached)

3. See datadir/mysql/general_log.CSV

2006-11-03 13:18:22","root[root] @ localhost []","5","0","Query","set names binary"
"2006-11-03 13:18:22","root[root] @ localhost []","5","0","Query","select _koi8r'"
"2006-11-03 13:18:22","root[root] @ localhost []","5","0","Quit",""

Note, the secdond line is truncated near the start of national character string.

Suggested fix:
National strings must be somehow encoded before writting into the log.
[3 Nov 2006 9:26] Alexander Barkov
How-to-repeat script

Attachment: bug.sql (application/octet-stream, text), 72 bytes.

[3 Nov 2006 9:28] Alexander Barkov
This bug is related to:

http://bugs.mysql.com/bug.php?id=19443
[3 Nov 2006 9:31] Alexander Barkov
Another related bug:

http://bugs.mysql.com/bug.php?id=11986
[3 Dec 2006 8:57] Valeriy Kravchuk
Thank you for a problem report. Sorry, but I was not able to repeat the behaviour described with latest 5.1.14-BK on Linux. No truncation of national characters.
[4 Dec 2006 4:49] Alexander Barkov
Queries with national characters don't get logged

Attachment: bug.sql (application/octet-stream, text), 132 bytes.

[4 Dec 2006 4:52] Alexander Barkov
I tried with a later version - the query with Cyrillic characters 
even does not get logged. See the second version of bug.sql attached.

mysql -uroot test < bug.sql

$mysql test -uroot < bug.sql
test
test
я┌п╣я│я┌
тест
event_time      user_host       thread_id       server_id       command_type argument
2006-12-04 08:48:51     root[root] @ localhost []       6       0       Query  set names binary
2006-12-04 08:48:51     root[root] @ localhost []       6       0       Query  select _koi8r'test'
2006-12-04 08:48:51     root[root] @ localhost []       6       0       Query  flush logs
2006-12-04 08:48:51     root[root] @ localhost []       6       0       Query  select * from general_log

Note, only one SELECT query - with latin characters - got logged.
The second SELECT query with Cyrillic letter was not logged.
[30 Dec 2006 12:23] Valeriy Kravchuk
Verified just as described with latest 5.1.15-BK on Linux (I do not have koi8 character set in that terminal window, but anyway results are obvious):

openxs@suse:~/dbs/5.1> bin/mysql -uroot test < /tmp/bug23924.sql
test
test
С'РчС?С'
ФЕУФ
event_time      user_host       thread_id       server_id       command_type argument
2006-12-30 12:19:19     root[root] @ localhost []       1       0       Query set names binary
2006-12-30 12:19:19     root[root] @ localhost []       1       0       Query
select _koi8r'test'
2006-12-30 12:19:19     root[root] @ localhost []       1       0       Query
flush logs
2006-12-30 12:19:19     root[root] @ localhost []       1       0       Query
select * from general_log
[20 Feb 2008 19:36] Omer Barnir
workaround: use file logging
[29 Feb 2008 11:50] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/43213

ChangeSet@1.2530, 2008-02-29 15:46:02+04:00, bar@mysql.com +7 -0
  Bug#23924 general_log truncates queries with character set introducers.
  Problem: logging of utf8-incompatible binary strings didn't work
  Fix: hex-encoding of incompatible sequences.
[29 Feb 2008 12:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/43217

ChangeSet@1.2530, 2008-02-29 16:41:02+04:00, bar@mysql.com +7 -0
  Bug#23924 general_log truncates queries with character set introducers.
  Problem: logging of utf8-incompatible binary strings didn't work
  Fix: hex-encoding of incompatible sequences.
[29 Feb 2008 12:52] Sergei Glukhov
ok to push
[29 Feb 2008 14:00] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/43221

ChangeSet@1.2530, 2008-02-29 17:56:50+04:00, bar@mysql.com +7 -0
  Bug#23924 general_log truncates queries with character set introducers.
  Problem: logging of utf8-incompatible binary strings didn't work
  Fix: hex-encoding of incompatible sequences.
[27 Mar 2008 11:17] Bugs System
Pushed into 5.1.24-rc
[27 Mar 2008 17:49] Bugs System
Pushed into 6.0.5-alpha
[27 Mar 2008 19:12] Paul DuBois
Noted in 5.1.24, 6.0.5 changelogs.

Logging of statements to log tables was incorrect for statements that
contained utf8-incompatible binary strings. Incompatible sequences
are hex-encoded now.
[30 Mar 2008 0:11] Jon Stephens
Also noted in 5.1.23-ndb-6.3.11 changelog.