MySQL Bugs: #117768: kopf lock cause pod not ready

Bug #117768	kopf lock cause pod not ready
Submitted:	21 Mar 16:27	Modified:	31 Mar 7:32
Reporter:	Bing Ma (OCA)	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Operator	Severity:	S3 (Non-critical)
Version:		OS:	Any
Assigned to:	MySQL Verification Team	CPU Architecture:	Any

Description:
### pod READINESS GATES
mcamel-common-mgr-cluster-0                                 3/3     Running   0               45m     10.233.101.237   g-master1   <none>           1/2
mcamel-common-mgr-cluster-1                                 3/3     Running   0               45m     10.233.66.107    g-master2   <none>           1/2
mcamel-common-mgr-cluster-2                                 3/3     Running   1 (5h44m ago)   6h34m   10.233.67.152    g-master3   <none>           2/2

### operator logs
[2025-03-21 16:16:20,058] kopf.objects         [ERROR   ] Handler 'on_pod_create' failed temporarily: mcamel-common-mgr-cluster busy. lock_owner=mcamel-common-mgr-cluster-1 owner_context=n/a lock_created_at=2025-03-21T15:35:17.084765

[2025-03-21 16:16:09,118] kopf.objects         [WARNING ] Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2025-03-21T15:47:04.247884', 'stopped': None, 'delayed': '2025-03-21T16:16:19.039547', 'purpose': 'create', 'retries': 169, 'success': False, 'failure': False, 'message': 'mcamel-common-kpanda-mgr busy. lock_owner=mcamel-common-kpanda-mgr-1 owner_context=n/a lock_created_at=2025-03-21T14:57:22.123302', 'subrefs': None}}}, None),)

### pod yaml

apiVersion: v1
kind: Pod
metadata:
  annotations:
    cni.projectcalico.org/containerID: 0152c24f926fea4cef5967dc97719fea40141eec7557b4d3724fe841f27e7e6d
    cni.projectcalico.org/podIP: 10.233.101.237/32
    cni.projectcalico.org/podIPs: 10.233.101.237/32
    kopf.zalando.org/on_pod_create: '{"started":"2025-03-21T16:03:08.648261","delayed":"2025-03-21T16:17:00.862259","purpose":"create","retries":81,"success":false,"failure":false,"message":"mcamel-common-mgr-cluster
      busy. lock_owner=mcamel-common-mgr-cluster-1 owner_context=n/a lock_created_at=2025-03-21T15:35:17.084765"}'
  creationTimestamp: "2025-03-21T15:29:51Z"
  finalizers:
  - mysql.oracle.com/membership
  - kopf.zalando.org/KopfFinalizerMarker
  generateName: mcamel-common-mgr-cluster-
  labels:
    app.kubernetes.io/component: database
    app.kubernetes.io/created-by: mysql-operator
    app.kubernetes.io/instance: mysql-innodbcluster-mcamel-common-mgr-cluster-mysql-server
    app.kubernetes.io/managed-by: mysql-operator
    app.kubernetes.io/name: mysql-innodbcluster-mysql-server
    apps.kubernetes.io/pod-index: "0"
    component: mysqld
    controller-revision-hash: mcamel-common-mgr-cluster-5f54687d78
    k8slens-edit-resource-version: v1
    mysql.oracle.com/cluster: mcamel-common-mgr-cluster
    mysql.oracle.com/instance-type: group-member
    statefulset.kubernetes.io/pod-name: mcamel-common-mgr-cluster-0
    tier: mysql
  name: mcamel-common-mgr-cluster-0

How to repeat:
I don't know how to repeat this

Hi,

I cannot reproduce this but generally temporary errors are something you have to retry (or the system you use to execute / deploy). 

I would say this is not a bug but without being able to reproduce this I can't be sure.

What version are you running? Have you tried using latest?