MySQL Bugs: #44373: handler::column_bitmaps_signal not called before handler::rnd

Bug #44373	handler::column_bitmaps_signal not called before handler::rnd_pos
Submitted:	20 Apr 2009 20:57	Modified:	11 Sep 2012 19:36
Reporter:	Zardosht Kasheff (OCA)	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: Storage Engine API	Severity:	S2 (Serious)
Version:	5.1.30, 5.1, azalea bzr	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
Storage engines that setup their own column bitmaps rely on handler::column_bitmaps_signal to be called in order to update their local copies.

For the following query in Q4 in test-ATIS (in the sql-bench tests):
"SELECT from_airport,to_airport,fare.fare_class,night,one_way_cost,rnd_trip_cost,class_days FROM compound_class,fare WHERE compound_class.fare_class=fare.fare_class AND one_way_cost <= 825 AND one_way_cost >= 280 AND from_airport='SFO' AND to_airport='DFW' GROUP BY from_airport,to_airport,fare.fare_class,night,one_way_cost,rnd_trip_cost,class_days ORDER BY one_way_cost"

handler::index_init on the key "from_airport" of the table "fare" is called. Before handler::index_end is ever called, handler::rnd_pos is called to retrieve some rows. The value of table->read_set in the calls to rnd_pos are different than those of the last column_bitmaps_signal call.

According to comments in handler.h on top of handler::column_bitmaps_signal, this seems like a bug. The change should have resulted in a call to column_bitmaps_signal so that the storage engine could adjust accordingly.

How to repeat:
compare table->read_set from last column_bitmaps_signal call to table->read_set of the first handler::rnd_pos call when running the query above in test-ATIS

Suggested fix:
handler::column_bitmaps_signal should be called to notify of the storage engine of the change.

Thank you for the report.

If I set breakpoint in handler::column_bitmaps_signal and ha_myisam::rnd_pos ha_myisam::rnd_pos  never called:

Breakpoint 4, handler::column_bitmaps_signal (this=0xeaa9228) at handler.cc:2455
2455      DBUG_ENTER("column_bitmaps_signal");
(gdb) c
Continuing.

Breakpoint 4, handler::column_bitmaps_signal (this=0xeaa3b28) at handler.cc:2455
2455      DBUG_ENTER("column_bitmaps_signal");
(gdb) c
Continuing.

Breakpoint 4, handler::column_bitmaps_signal (this=0xeaa3b28) at handler.cc:2455
2455      DBUG_ENTER("column_bitmaps_signal");
(gdb) c
Continuing.

Breakpoint 1, handler::index_init (this=0xeaa3b28, idx=2, sorted=false) at ../../sql/handler.h:1776
1776      virtual int index_init(uint idx, bool sorted) { active_index= idx; return 0; }
(gdb) c
Continuing.

Breakpoint 1, handler::index_init (this=0xeaa9228, idx=0, sorted=false) at ../../sql/handler.h:1776
1776      virtual int index_init(uint idx, bool sorted) { active_index= idx; return 0; }
(gdb) c
Continuing.

Breakpoint 2, handler::index_end (this=0xeaa3b28) at ../../sql/handler.h:1777
1777      virtual int index_end() { active_index= MAX_KEY; return 0; }
(gdb) c
Continuing.

Breakpoint 2, handler::index_end (this=0xeaa9228) at ../../sql/handler.h:1777
1777      virtual int index_end() { active_index= MAX_KEY; return 0; }
(gdb) c
Continuing.

Breakpoint 4, handler::column_bitmaps_signal (this=0xeaacb28) at handler.cc:2455
2455      DBUG_ENTER("column_bitmaps_signal");
(gdb) c
Continuing.

Breakpoint 4, handler::column_bitmaps_signal (this=0xeaacb28) at handler.cc:2455
2455      DBUG_ENTER("column_bitmaps_signal");
(gdb) c
Continuing.

Please specify how to modify/implement column_bitmaps_signal so rnd_pos is called.

I think you may have misunderstood the problem. The problem is that given rnd_pos is called, column_bitmaps_signal is NOT called to notify the storage engine of a different bitmap.

Thank you for the feedback.

But in my case rnd_pos was not called. I tested with today bzr sources.

I think the problem reproduces with any test that calls rnd_pos, so there must be some test for MyISAM that executes rnd_pos.

With our storage engine, this reproduces the issue as well:

create table foo ( a int, b int, c int);
insert into foo values (1,1,1),(1,2,2),(1,3,3),(2,1,1),(2,2,2),(2,3,3),(3,1,1),(3,2,2),(3,3,3);
insert into foo values (4,4,1),(4,5,2),(4,6,3),(5,4,1),(5,5,2),(5,6,3),(6,4,1),(6,5,2),(6,6,3);
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
select count(*) from foo;

alter table foo add index (a), add index (b);
select * from foo where a=2 and b=2;

The select uses an intersect on keys a and b, which result in rnd_pos calls. The last call to column_bitmaps_signal has the bitmap set at 2. The first call to rnd_pos has the bitmap set at 7. This is the problem. column_bitmaps_signal should have been called resetting the bitmap to 7.

Thank you for the feedback.

I still can not repeat described behavior with MyISAM: no call to rnd_pos without call column_bitmaps_signal  before. Please, provide repeatable test case. Thanks in advance.

Have you verified that the bitmaps table->read_set and table->write_set are the same in both calls? The bug is not that column_bitmaps_signal is not called, the bug is that it is not called with the right bitmap

Thank you for the feedback.

Yes, they both are same for me.

The following test from above reproduces the issue on windows, with the engine being InnoDB:

create table foo ( a int, b int, c int);
insert into foo values
(1,1,1),(1,2,2),(1,3,3),(2,1,1),(2,2,2),(2,3,3),(3,1,1),(3,2,2),(3,3,3);
insert into foo values
(4,4,1),(4,5,2),(4,6,3),(5,4,1),(5,5,2),(5,6,3),(6,4,1),(6,5,2),(6,6,3);
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
insert into foo select * from foo;
select count(*) from foo;

alter table foo add index (a), add index (b);
select * from foo where a=2 and b=2;

In the select call, column_bitmaps_signal is called 9 times before the first rnd_pos. In the 9th call to column_bitmaps_signal, table->read_set is 2. In the first call to rnd_pos, table->read_set is 7. This is the problem. There should be a call to columns_bitmap_signal between those two where table->read_set is 7.

Thank you for the feedback.

Verified as described.

Wasn't able to reproduce this one. Maybe something's changed meanwhile.
Sveta, can you check it once more?

Alexey,

have you checked table->read_set->bitmap value?

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

I did not see an answer to Sveta's question, so I do not know what i need to comment, besides that I think the bug should still be open

Feedback for Sveta's question.

Setting back to "Verified" since devs seems missed this bug.