Bug #108824 GCS_DEBUG_TRACE log not possible to rotate/archive without downtime
Submitted: 19 Oct 2022 10:21 Modified: 19 Oct 2022 10:51
Reporter: Przemyslaw Malkowski Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Group Replication Severity:S3 (Non-critical)
Version:8.0.31 OS:Any
Assigned to: CPU Architecture:Any
Tags: group replicaiton

[19 Oct 2022 10:21] Przemyslaw Malkowski
Description:
The $datadir/GCS_DEBUG_TRACE file gets created upon Group Replication initialization, i.e. after running:
start group_replication;

Now, it is natural that you may want to rotate the file if a long enough debug session is needed (via group_replication_communication_debug_options).

So, what we intuitively would do is: stop debug mode, rm/mv the file and enable it back. But, that does not do the job. Also, enabling debug mode does not return any error if the GCS_DEBUG_TRACE file is not present, even though it will not re-create it and so debug mode is ineffective.

So, to rotate the file, you have to restart group replication on the node...

How to repeat:
node1 > select @@version,@@group_replication_communication_debug_options;
+-----------+-------------------------------------------------+
| @@version | @@group_replication_communication_debug_options |
+-----------+-------------------------------------------------+
| 8.0.31    | GCS_DEBUG_BASIC                                 |
+-----------+-------------------------------------------------+
1 row in set (0.00 sec)

$ ls -lh data/GCS_DEBUG_TRACE
-rw-r----- 1 przemek przemek 63K Oct 19 12:15 data/GCS_DEBUG_TRACE

node1 > set global group_replication_communication_debug_options=GCS_DEBUG_NONE;
Query OK, 0 rows affected (0.00 sec)

$ mv data/GCS_DEBUG_TRACE data/GCS_DEBUG_TRACE.1

node1 > flush logs;
Query OK, 0 rows affected (0.05 sec)

node1 > set global group_replication_communication_debug_options=GCS_DEBUG_BASIC;
Query OK, 0 rows affected (0.00 sec)

node1 > flush logs;
Query OK, 0 rows affected (0.05 sec)

$ ls -lh data/GCS_DEBUG_TRACE
ls: cannot access 'data/GCS_DEBUG_TRACE': No such file or directory

$ touch data/GCS_DEBUG_TRACE

node1 > set global group_replication_communication_debug_options=GCS_DEBUG_NONE;
Query OK, 0 rows affected (0.00 sec)

node1 > set global group_replication_communication_debug_options=GCS_DEBUG_BASIC;
Query OK, 0 rows affected (0.00 sec)

$ ls -lh data/GCS_DEBUG_TRACE
-rw-rw-r-- 1 przemek przemek 0 Oct 19 12:18 data/GCS_DEBUG_TRACE

Suggested fix:
Allow GCS_DEBUG_TRACE rotation without restarting the replication.
[19 Oct 2022 10:51] MySQL Verification Team
Hello Przemyslaw,

Thank you for the report. 

Thanks,
Umesh