Bug #63214 Compile time error when page size is set to 32KB
Submitted: 11 Nov 2011 20:16 Modified: 18 Feb 2016 22:11
Reporter: Nizameddin Ordulu Email Updates:
Status: Closed Impact on me:
Category:MySQL Server: InnoDB Plugin storage engine Severity:S2 (Serious)
Version: OS:Any
Assigned to: Kevin Lewis CPU Architecture:Any
Tags: fsp0fsp.c, innodb

[11 Nov 2011 20:16] Nizameddin Ordulu
The following piece of code in xdes_calc_descriptor_page() in fsp0fsp.c causes compile time error when I try to compile innodb with UNIV_PAGE_SIZE_SHIFT = 15.

 656#ifndef DOXYGEN /* Doxygen gets confused of these */
 659#  error
 660# endif

What is this check trying to do? Can you tell me the format for fsp pages and how they work? Following is a comment inside dict_build_table_def_step() in dict0crea.c. 

 270    /* We create a new single-table tablespace for the table.
 271    We initially let it be 4 pages:
 272    - page 0 is the fsp header and an extent descriptor page,
 273    - page 1 is an ibuf bitmap page,
 274    - page 2 is the first inode page,
 275    - page 3 will contain the root of the clustered index of the
 276    table we create here. */

Can you explain the page format for fsp header, inode and ibuf bitmap pages?

How to repeat:
Try to change page size to 32KB instead of 16KB.

Suggested fix:
Not a problem.
[15 Nov 2011 13:34] Marko Mäkelä
Every PAGE_SIZE bytes, there are two special pages: the allocation bitmap page and the insert buffer bitmap page. If the page size is 16384 bytes, these would be pages 0,1,16384,16385,32768,32769 and so on. Each bitmap page describes a block of PAGE_SIZE pages.

The allocation bitmap page at page 0 additionally serves as the tablespace header page.

The insert buffer bitmap (pages 1, 16385, 32769, and so on) contains 4 bits for every page:

IBUF_BITMAP_IBUF: Says whether a page belongs to the change buffer tree.
Always 0 if space_id>0
IBUF_BITMAP_BUFFERED: Consulted on page read. Says whether ibuf merge is needed before returning the page to the user.
IBUF_BITMAP_FREE (0..3): Low-bound estimate of the space available for buffered inserts. (Page overflow is not allowed on insert buffer merge.)

An InnoDB file segment inode is a collection of allocated pages. It is mainly used for allocation bookkeeping. For example, each index B-tree has two file segments (pointers are in the root page): one for the non-leaf pages and another for leaf pages and BLOBs. Do you need more info on this?

Last but not least, the allocation bitmap page (pages 0, 16384, 32768, 49162, 65536, etc., assuming 16k page size).

The allocation bitmap page is divided into extent descriptors, each of which covers FSP_EXTENT_SIZE (or 64) pages. I do not know the exact reason for this. Perhaps Heikki wanted to extend tablespaces in full megabytes (64*16k) at a time and thought that this needs to be reflected on the allocation bitmap page. 

An extent descriptor looks like this:

XDES_ID		0	id (64 bits), 0 if the segment is unused
XDES_FLST_NODE	8	next,prev file list pointer (2*6 bytes)
XDES_STATE	20	state (4 bytes for 2 bits used): XDES_FREE, XDES_FREE_FRAG, XDES_FULL_FRAG, XDES_FSEG
XDES_BITMAP	24	bitmap (2 bits/page * 64 pages)

In the bitmap, there are these bits for each page:

#define	XDES_FREE_BIT		0	/* Index of the bit which tells if
					the page is free */
#define	XDES_CLEAN_BIT		1	/* NOTE: currently not used!
					Index of the bit which tells if
					there are old versions of tuples
					on the page */

If I were to design this from scratch, I would merge the two bitmap pages and get rid of XDES_CLEAN_BIT and IBUF_BITMAP_IBUF and the extra allocated bits. The change buffer would be in dedicated tablespace(s). The XDES_ID seems to be a per-tablespace entity. Because a tablespace can contain at most 32 bits, allocating 32 bits for XDES_ID would be more than enough. Typically, there should be something like 2*n_indexes file segments in a tablespace, plus some special file segments in the system tablespace. MySQL allows something like 32 or 64 indexes per table.

You should probably look at xdes_get_descriptor_with_space_hdr() and its callers to get an idea how this works. Feel free to ask further questions.
[18 Feb 2016 22:11] Kevin Lewis
Posted by developer:
This wa fixed with the introduction of 32k and 64k page size.