Bug #105914 some undo logs may be truncated, but were not be purged
Submitted: 16 Dec 2021 6:48 Modified: 17 Dec 2021 3:25
Reporter: jie xu Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S3 (Non-critical)
Version:5.7 OS:Any
Assigned to: CPU Architecture:Any

[16 Dec 2021 6:48] jie xu
Description:

--------------------code------------------
/********************************************************************//**
Removes unnecessary history data from a rollback segment. */
static
void
trx_purge_truncate_rseg_history(
/*============================*/
	trx_rseg_t*		rseg,		/*!< in: rollback segment */
	const purge_iter_t*	limit)		/*!< in: truncate offset */
{
  ...

  if (undo_trx_no >= limit->trx_no) {

	/* limit space_id should match the rollback segment
	space id to avoid freeing of the page belongs to
	different rollback segment for the same trx_no. */
	if (undo_trx_no == limit->trx_no
		 && rseg->space == limit->undo_rseg_space) {

		trx_undo_truncate_start(
			rseg, hdr_addr.page,
			hdr_addr.boffset, limit->undo_no);
	}

	mutex_exit(&(rseg->mutex));
	mtr_commit(&mtr);

	return;
  }

  ...

  if ((mach_read_from_2(seg_hdr + TRX_UNDO_STATE) == TRX_UNDO_TO_PURGE)
	    && (mach_read_from_2(log_hdr + TRX_UNDO_NEXT_LOG) == 0)) {

	/* We can free the whole log segment */

	mutex_exit(&(rseg->mutex));
	mtr_commit(&mtr);

	/* calls the trx_purge_remove_log_hdr()
	inside trx_purge_free_segment(). */
	trx_purge_free_segment(rseg, hdr_addr, noredo);

  } else {
	/* Remove the log hdr from the rseg history. */
	trx_purge_remove_log_hdr(rseg_hdr, log_hdr, &mtr);

	mutex_exit(&(rseg->mutex));
	mtr_commit(&mtr);
  }

  ...

}
--------------------code------------------
When truncate rseg history; if trx_no bigger than limit->trx_no, can't truncate this trx_no's undo logs.

--------------------code------------------
/** Iterate over selected UNDO tablespace and check if all the rsegs
that resides in the tablespace are free.
@param[in]	limit		truncate_limit
@param[in,out]	undo_trunc	undo truncate tracker */
static
void
trx_purge_initiate_truncate(
	purge_iter_t*	limit,
	undo::Truncate*	undo_trunc)
{
   ...

   for (trx_undo_t* undo =
	  UT_LIST_GET_FIRST(rseg->update_undo_cached);
	undo != NULL && all_free;
	undo = UT_LIST_GET_NEXT(undo_list, undo)) {

	if (limit->trx_no < undo->trx_id) {
		all_free = false;
	} else {
		cached_undo_size += undo->size;
	}
   }

   for (trx_undo_t* undo =
	 UT_LIST_GET_FIRST(rseg->insert_undo_cached);
	 undo != NULL && all_free;
	 undo = UT_LIST_GET_NEXT(undo_list, undo)) {

	if (limit->trx_no < undo->trx_id) {
		all_free = false;
	} else {
		cached_undo_size += undo->size;
	}
   }

   ...

   if (!all_free) {
      /* rseg still holds active data.*/
      return;
   }
   
   ...
}
--------------------code------------------
When truncate undo file; if trx_id bigger than limit->trx_no in undo cache, can't truncate this undo file.

But if such a trx exists, this trx's trx_no bigger than limit->trx_no, and trx's trx_id less than limit->trx_no; and this undo file may be truncated, but this trx's undo logs may not be purged.

How to repeat:
no repeat

Suggested fix:
When truncate undo file; if trx_no bigger than limit->trx_no in undo cache, can't truncate this undo file, not trx_id.
[16 Dec 2021 6:57] jie xu
This bug was fixed in MySQL 8.0, but hasn't be fixed in MySQL 5.7.

This bug is Bug #27114068 in MySQL 8.0.
https://github.com/mysql/mysql-server/commit/19746679399649c1898ff165e51edaf5229b89c0
[16 Dec 2021 13:19] MySQL Verification Team
Hi Mr. xu,

Thank you for your bug report.

When the bug is fixed in 8.0 and not in 5.7, then it is probably made by the decision of the team in charge of this particular topic. Hence, we are verifying your report as a bug, but the decision on whether it will be fixed or not will come from the department in charge.

When a decision is made, this report will be updated.

Verified as reported.
[17 Dec 2021 3:25] jie xu
Thank you!