Bug #121039 Bug: Alter_stage::aggregate() loses the log_index phase work,
Submitted: 30 Jul 9:04
Reporter: Alice Alice Email Updates:
Status: Open Impact on me:
None 
Category:MySQL Server: Performance Schema Severity:S4 (Feature request)
Version:8.0 OS:Any
Assigned to: CPU Architecture:Any

[30 Jul 9:04] Alice Alice
Description:
  aggregate() accumulates each sub-builder's per-stage counters into the parent stage's PFS progress.
  Two things combine to make the log_index (log apply) phase work disappear, so WORK_COMPLETED visibly
  decreases after aggregation:

  1. The end stage is skipped. The inner loop does if (stage.first == &srv_stage_alter_table_end) { 
  continue; }, so the end stage's counter is never added.
  2. Counter semantics + the skip interact badly for log_index. change_phase() snapshots the previous
  stage's final WORK_COMPLETED/WORK_ESTIMATED into the new stage's counter (m_stages[i].counter = 
  final value of stage[i-1]). So, for a sub-builder's m_stages = [read_pk, merge_sort, insert, 
  log_index, end]:
    - log_index.counter holds the insert final value (snapshot when entering log_index), not the
  log_index final value.
    - end.counter holds the log_index final value (snapshot when entering end).

  Because the end stage is skipped, the log_index final value (the actual log-apply work) is never
  accumulated. The log_index branch adds only log_index.counter (the insert final value), so the
  aggregated WORK_COMPLETED loses the log-apply work and regresses (e.g. 47 → 23 for a single-index
  build; 2 × insert_final for an N-index build).

How to repeat:
  CREATE TABLE t(a INT, b VARCHAR(100));
  -- insert ~1000 rows
  SET SESSION innodb_ddl_threads=1;
  ALTER TABLE t ADD INDEX idx(a), ALGORITHM=INPLACE;
  -- during the ALTER, query WORK_COMPLETED across the phase transition:
  SELECT EVENT_NAME, WORK_COMPLETED, WORK_ESTIMATED
    FROM performance_schema.events_stages_current
    WHERE EVENT_NAME LIKE 'stage/innodb/alter%';
  -- before aggregate(): WORK_COMPLETED grows to ~47 (log apply index)
  -- after aggregate()  : WORK_COMPLETED drops to ~23 (insert final value)

  The drop happens at the Loader destructor boundary, when aggregate() runs over the sub-builders and
  the parent stage becomes the observed one.

Suggested fix:
  Capture the end stage's counter (which holds the log_index final value) before the main loop, and
  use it for the log_index branch instead of log_index's own (incomplete) counter:

  for (auto alter_stage : alter_stages) {
    alter_stage->begin_phase_end();

    /* end.counter snapshots the log_index final value; the main loop skips end,
    so capture it here for the log_index branch below. */
    Counter log_index_final{};
    for (auto &st : alter_stage->m_stages) {
      if (st.first == &srv_stage_alter_table_end) {
        log_index_final = st.second.second;
        break;
      }
    }

    for (auto stage : alter_stage->m_stages) {
      if (stage.first == &srv_stage_alter_table_end) {
        continue;
      }
      ...
      const auto &counter = stage.second.second;
      uint64_t cf = counter.first;
      uint64_t ce = counter.second;

      if (stage.first == &srv_stage_alter_table_log_index) {
        /* log_index.counter holds only the insert final value; use end.counter
        (the log_index final value) so progress does not regress. */
        cf = log_index_final.first;
        ce = log_index_final.second;
      }

      c += cf;
      e += ce;
      ...
    }
  }

  The end stage is still skipped in the main loop (its counter is not added separately), avoiding
  double counting; only the log_index branch substitutes the captured value.