Description:
Environment:
CentOS 5 i386
5.0.22 master
5.0.51a and 5.1.24 slaves
When a slave is configured to rewrite the DB name, and an ALTER TABLE ... DISABLE/ENABLE KEYS statement is wrapped in a version-filtering comment (i.e. /*40000 ...*/), the thread executing the statement on the slave corrupts memory and the server becomes crippled, but does not necessarily crash entirely.
Statements like these cause the problem:
/*!41000 ALTER TABLE Rotation DISABLE KEYS */;
/*!40000 ALTER TABLE Rotation DISABLE KEYS */;
/*!50000 ALTER TABLE Rotation ENABLE KEYS */;
These seem ok:
/*!40000 ALTER TABLE Rotation ADD COLUMN blah2 INTEGER */;
/*!40000 INSERT INTO Rotation VALUES (102, 1,'1') */;
ALTER TABLE Rotation DISABLE KEYS;
So it would seem to be the combination of the version comment and the disable/enable keys. I haven't found any other statements that cause the problem, but i did not test extensively.
I built a debug version of mysql 5.0.51a and when I run one of the test cases above, I get a bunch of memory overrun messages in the error log :
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_alloc.c:357'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:152'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:111'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'sql_base.cc:1905'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:152'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_alloc.c:201'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:134'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1046'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1059'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1079'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1080'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:312'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:339'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1079'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1080'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:111'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_alloc.c:201'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:873'
Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:152'
fyi,
log_event.cc:1532: Query_log_event::Query_log_event()
if (!(start= data_buf = (Log_event::Byte*) my_malloc(catalog_len + 1 +
time_zone_len + 1 +
data_len + 1 +
QUERY_CACHE_FLAGS_SIZE +
db_len + 1,
MYF(MY_WME))))
I tried attaching a debugger, but I'm not having a lot of luck tracking down the problem, because the behaviour is not consistent. This was also the case on our front-end production boxes -- some went down, others were "responsive" (could connect but queries would lock up). Sometimes when I try to attach a debugger and step through, I end up with an underrun message instead, like the below.
Error: Memory allocated at (null):262144 was underrun, discovered at my_alloc.c:357
How to repeat:
I am also attaching the following for reference:
- my.cnf files I used for master/slave
- a mysql trace I did from one of my reproductions
- original error log entries from production server and stack trace
This specific problem is not a huge issue as there is an obvious workaround, but it would be interesting to see if the bug that causes this crash affects anything else.
Suggested fix:
Unable to find cause for the bug.
Description: Environment: CentOS 5 i386 5.0.22 master 5.0.51a and 5.1.24 slaves When a slave is configured to rewrite the DB name, and an ALTER TABLE ... DISABLE/ENABLE KEYS statement is wrapped in a version-filtering comment (i.e. /*40000 ...*/), the thread executing the statement on the slave corrupts memory and the server becomes crippled, but does not necessarily crash entirely. Statements like these cause the problem: /*!41000 ALTER TABLE Rotation DISABLE KEYS */; /*!40000 ALTER TABLE Rotation DISABLE KEYS */; /*!50000 ALTER TABLE Rotation ENABLE KEYS */; These seem ok: /*!40000 ALTER TABLE Rotation ADD COLUMN blah2 INTEGER */; /*!40000 INSERT INTO Rotation VALUES (102, 1,'1') */; ALTER TABLE Rotation DISABLE KEYS; So it would seem to be the combination of the version comment and the disable/enable keys. I haven't found any other statements that cause the problem, but i did not test extensively. I built a debug version of mysql 5.0.51a and when I run one of the test cases above, I get a bunch of memory overrun messages in the error log : Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_alloc.c:357' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:152' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:111' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'sql_base.cc:1905' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:152' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_alloc.c:201' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:134' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1046' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1059' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1079' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1080' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:312' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:339' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1079' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:1080' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:111' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_alloc.c:201' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'table.cc:873' Error: Memory allocated at log_event.cc:1532 was overrun, discovered at 'my_open.c:152' fyi, log_event.cc:1532: Query_log_event::Query_log_event() if (!(start= data_buf = (Log_event::Byte*) my_malloc(catalog_len + 1 + time_zone_len + 1 + data_len + 1 + QUERY_CACHE_FLAGS_SIZE + db_len + 1, MYF(MY_WME)))) I tried attaching a debugger, but I'm not having a lot of luck tracking down the problem, because the behaviour is not consistent. This was also the case on our front-end production boxes -- some went down, others were "responsive" (could connect but queries would lock up). Sometimes when I try to attach a debugger and step through, I end up with an underrun message instead, like the below. Error: Memory allocated at (null):262144 was underrun, discovered at my_alloc.c:357 How to repeat: I am also attaching the following for reference: - my.cnf files I used for master/slave - a mysql trace I did from one of my reproductions - original error log entries from production server and stack trace This specific problem is not a huge issue as there is an obvious workaround, but it would be interesting to see if the bug that causes this crash affects anything else. Suggested fix: Unable to find cause for the bug.