Bug #94565 group_replication_member_stats do not have every columns listed in docs
Submitted: 5 Mar 2019 15:45 Modified: 18 Apr 2019 6:57
Reporter: Yoann La Cancellera Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Group Replication Severity:S4 (Feature request)
Version:5.7.25 OS:Ubuntu
Assigned to: CPU Architecture:Any
Tags: documentation, observability

[5 Mar 2019 15:45] Yoann La Cancellera
Description:
This could be in documentation category, but this is more a feature request than a documentation bug

According to https://dev.mysql.com/doc/refman/5.7/en/group-replication-replication-group-member-stats.h..., I should get :

mysql> select * from performance_schema.replication_group_member_stats\G;
*************************** 1. row ***************************
                              CHANNEL_NAME: group_replication_applier
                                   VIEW_ID: 15517946505895476:13
                                 MEMBER_ID: 80c2611b-3f4f-11e9-bc27-0242ac120002
               COUNT_TRANSACTIONS_IN_QUEUE: 0
                COUNT_TRANSACTIONS_CHECKED: 166245
                  COUNT_CONFLICTS_DETECTED: 22517
        COUNT_TRANSACTIONS_ROWS_VALIDATING: 112532
        TRANSACTIONS_COMMITTED_ALL_MEMBERS: 80c2611b-3f4f-11e9-bc27-0242ac120002:1-12,
8bdecced-3f4f-11e9-bc27-0242ac120002:1-118049:1000013-1003006
            LAST_CONFLICT_FREE_TRANSACTION: 8bdecced-3f4f-11e9-bc27-0242ac120002:228878
COUNT_TRANSACTIONS_REMOTE_IN_APPLIER_QUEUE: 0
         COUNT_TRANSACTIONS_REMOTE_APPLIED: 0
         COUNT_TRANSACTIONS_LOCAL_PROPOSED: 0
         COUNT_TRANSACTIONS_LOCAL_ROLLBACK: 0
1 row in set (0.00 sec)

Instead, I get (missing last 4 columns) on mysql 5.7.25  : 
mysql> select * from performance_schema.replication_group_member_stats\G;
*************************** 1. row ***************************
                      CHANNEL_NAME: group_replication_applier
                           VIEW_ID: 15517946505895476:13
                         MEMBER_ID: 80c2611b-3f4f-11e9-bc27-0242ac120002
       COUNT_TRANSACTIONS_IN_QUEUE: 0
        COUNT_TRANSACTIONS_CHECKED: 166245
          COUNT_CONFLICTS_DETECTED: 22517
COUNT_TRANSACTIONS_ROWS_VALIDATING: 112532
TRANSACTIONS_COMMITTED_ALL_MEMBERS: 80c2611b-3f4f-11e9-bc27-0242ac120002:1-12,
8bdecced-3f4f-11e9-bc27-0242ac120002:1-118049:1000013-1003006
    LAST_CONFLICT_FREE_TRANSACTION: 8bdecced-3f4f-11e9-bc27-0242ac120002:228878
1 row in set (0.00 sec)

We have a cluster with 3 nodes (single-primary), one crashed, a new primary was elected.
Since our crashed node is in recovery, we experience a lot of rollbacks (during COMMIT with error 3101) and I suspect that the flow control has been triggered

Problem is, we know flow control happen when a queue exceeds 25k waiting transactions, but we cannot see queue sizes despite what is written is documentation
We cannot see when flow control is activated

We cannot see how late is a node either, because of lacking queue size stats. 

Is it a bug ? A documentation failure ? 

Anyway, we could use some logs/table to observe flow control behaviour

Is a backport of these 4 missing columns planned ?

How to repeat:
Use dbdeployer to deploy a GR cluster 
https://github.com/datacharmer/dbdeployer

dbdeployer deploy replication -t=group 5.7.25

./sandboxes/group_msb_5_7_25/use_all 'select * from performance_schema.replication_group_member_stats\G'

Suggested fix:
- Backport of 8.0.14 performance_schema tables

- Fix documentation

- Flow Control logs feature
[6 Mar 2019 7:14] MySQL Verification Team
Hello Yoann,

Thank you for the report and feedback.
This looks like doc issue to me, verifying after comparing https://dev.mysql.com/doc/refman/5.7/en/replication-group-member-stats-table.html and https://dev.mysql.com/doc/refman/8.0/en/replication-group-member-stats-table.html

regards,
Umesh
[17 Apr 2019 14:13] Margaret Fisher
Posted by developer:
 
Thanks for pointing this out! The incorrect fields have now been removed from the 5.7 documentation.
[18 Apr 2019 6:57] Yoann La Cancellera
Thanks ! 
Any plan to backport these columns to 5.7 ?