Description:
When you enable full backups `backupProfile/backupSchedules/ storage PVC` in innodbcluster resource mysqloperator creates cronjob with job/pod template.
Backup process is:
cronjob-cb -> job -> pod -> create mysqlbackup -> job (backup) -> pod (exectues backup)
mysqlbackup - related job/pod stays there forever no autodeletion.
At node replacement, all backup pods are triggered again at the time of node replacement. At the time of node replacement database had values X and all pods copied data from the time of node replacement to old directories which means they were overwriting older backups with newer data values.
How to repeat:
`backupProfile/backupSchedules/ storage PVC`
keep mysqlbackup resources (5 of them)
trigger node replacement and observe which jobs/pods will execute again
Suggested fix:
Remove mysqlbackup resources after completed succesfully