| Bug #107322 | The sample may be wrong | ||
|---|---|---|---|
| Submitted: | 18 May 2022 7:31 | Modified: | 22 Jun 2022 4:04 |
| Reporter: | enn bo | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Operator | Severity: | S7 (Test Cases) |
| Version: | 8.0.29-2.0.4 | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
| Tags: | Sidecar configured | ||
[19 May 2022 12:18]
MySQL Verification Team
Hi, We have some interim questions for you ...... Have you created the three instances on three different computers or three virtual machines and ran the first command (kubectl) with success ???? We are waiting on your feedback.
[20 May 2022 2:19]
enn bo
Yes, I ran kubectl successfully, it started 3 pods on two nodes (our cluster nodes are much larger than 3, each node is a server), the logs of all three nodes show "Sidecar of mycluster-X is not yet configured” My kubernetes version number is 1.16.7
[20 May 2022 13:32]
MySQL Verification Team
Hi Mr. bo, Please let us know the configuration of your InnoDB cluster, how are nodes connected, its architecture and was it running while you were applying the commands that you describe. Waiting on your feedback.
[21 May 2022 9:26]
enn bo
I've said that I can't run the example, here's the innodbcluster config:
```
apiVersion: mysql.oracle.com/v2
kind: InnoDBCluster
metadata:
name: mycluster
spec:
secretName: mypwds
tlsUseSelfSigned: true
instances: 3
version: 8.0.29
router:
instances: 1
version: 8.0.29
datadirVolumeClaimTemplate:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 4Gi
mycnf: |
[mysqld]
max_connections=162
```
Then I created this instance in the test namespace. Here is a description of one of the Pods:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 64s default-scheduler Successfully assigned test/mycluster-0 to tbj3-paas-k8s4
Error Logging 64s kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
Warning Logging 64s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2022-05-21T09:16:40.501856', 'stopped': None, 'delayed': '2022-05-21T09:16:50.535880', 'purpose': 'create', 'retries': 1, 'success': False, 'failure': False, 'message': 'Sidecar of mycluster-0 is not yet configured', 'subrefs': None}}}, None),)
Normal Logging 64s kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None
Normal Pulling 63s kubelet Pulling image "mysql/mysql-operator:8.0.29-2.0.4"
Normal Created 62s kubelet Created container fixdatadir
Normal Started 62s kubelet Started container fixdatadir
Normal Pulled 62s kubelet Successfully pulled image "mysql/mysql-operator:8.0.29-2.0.4"
Normal Pulling 61s kubelet Pulling image "mysql/mysql-operator:8.0.29-2.0.4"
Normal Pulled 60s kubelet Successfully pulled image "mysql/mysql-operator:8.0.29-2.0.4"
Normal Created 60s kubelet Created container initconf
Normal Started 60s kubelet Started container initconf
Warning Logging 54s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'dummy': '2022-05-21T09:16:50.538221'}, None),)
Normal Logging 54s kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None
Error Logging 54s kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
Warning Logging 54s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2022-05-21T09:16:40.501856', 'stopped': None, 'delayed': '2022-05-21T09:17:00.681133', 'purpose': 'create', 'retries': 2, 'success': False, 'failure': False, 'message': 'Sidecar of mycluster-0 is not yet configured', 'subrefs': None}}}, None),)
Normal Pulling 44s (x3 over 59s) kubelet Pulling image "mysql/mysql-server:8.0.29"
Warning Logging 44s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'dummy': '2022-05-21T09:17:00.683258'}, None),)
Warning Logging 44s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2022-05-21T09:16:40.501856', 'stopped': None, 'delayed': '2022-05-21T09:17:10.818979', 'purpose': 'create', 'retries': 3, 'success': False, 'failure': False, 'message': 'Sidecar of mycluster-0 is not yet configured', 'subrefs': None}}}, None),)
I think this is a useful piece of information：“Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured”
[21 May 2022 9:28]
enn bo
Add some describe, it looks a bit repetitive：
Normal Logging 44s kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None
Error Logging 44s kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
Normal Created 43s (x3 over 58s) kubelet Created container initmysql
Normal Pulled 43s (x3 over 58s) kubelet Successfully pulled image "hub.intra.mlamp.cn/public/mysql/mysql-server:8.0.29"
Warning Failed 43s (x3 over 58s) kubelet Error: failed to start container "initmysql": Error response from daemon: linux spec capabilities: Unknown capability drop: "AUDIT_READ"
Warning BackOff 42s (x5 over 54s) kubelet Back-off restarting failed container
Warning Logging 34s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2022-05-21T09:16:40.501856', 'stopped': None, 'delayed': '2022-05-21T09:17:20.958252', 'purpose': 'create', 'retries': 4, 'success': False, 'failure': False, 'message': 'Sidecar of mycluster-0 is not yet configured', 'subrefs': None}}}, None),)
Error Logging 34s kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
Warning Logging 34s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'dummy': '2022-05-21T09:17:10.820358'}, None),)
Normal Logging 34s kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None
Warning Logging 24s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'dummy': '2022-05-21T09:17:20.960637'}, None),)
Normal Logging 23s kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None
Warning Logging 23s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2022-05-21T09:16:40.501856', 'stopped': None, 'delayed': '2022-05-21T09:17:31.098090', 'purpose': 'create', 'retries': 5, 'success': False, 'failure': False, 'message': 'Sidecar of mycluster-0 is not yet configured', 'subrefs': None}}}, None),)
Error Logging 23s kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
Normal Logging 13s kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None
Warning Logging 13s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'dummy': '2022-05-21T09:17:31.100490'}, None),)
Warning Logging 13s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2022-05-21T09:16:40.501856', 'stopped': None, 'delayed': '2022-05-21T09:17:41.239130', 'purpose': 'create', 'retries': 6, 'success': False, 'failure': False, 'message': 'Sidecar of mycluster-0 is not yet configured', 'subrefs': None}}}, None),)
[24 May 2022 20:18]
MySQL Verification Team
Hi, Thanks for the report. Can you try using newer kubectl? I just tried this with v1.21.0 and I had no issues ? kind regards
[22 Jun 2022 4:04]
Philip Olson
Posted by developer: Fixed as of the upcoming MySQL Operator 8.0.30 release, and here's the proposed changelog entry from the documentation team: Removed the AUDIT_READ securityContext capability; a feature introduced in Linux Kernel 3.16 which is too new for some K8s installations, such as Enterprise Linux 7. Thank you for the bug report.
[7 Dec 2022 12:36]
Y G
Hi. I've encountered the same problem on minikube, i just followed the sample configuration at https://dev.mysql.com/doc/mysql-operator/en/mysql-operator-innodbcluster-simple-kubectl.ht... mysql-operator version is 8.0.31-2.0.7 kubernetes version is 1.25.2 kubectl describe pod mycluster-0 output: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 20m default-scheduler Successfully assigned default/mycluster-0 to minikube Error Logging 20m kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured Normal Logging 20m kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None Normal Pulling 20m kubelet Pulling image "mysql/mysql-operator:8.0.31-2.0.7" Normal Created 20m kubelet Created container fixdatadir Normal Pulled 20m kubelet Successfully pulled image "mysql/mysql-operator:8.0.31-2.0.7" in 4.603942662s Normal Pulling 20m kubelet Pulling image "mysql/mysql-operator:8.0.31-2.0.7" Normal Started 20m kubelet Started container fixdatadir Normal Pulled 20m kubelet Successfully pulled image "mysql/mysql-operator:8.0.31-2.0.7" in 5.679272068s Normal Created 20m kubelet Created container initconf Normal Started 20m kubelet Started container initconf Normal Pulling 20m kubelet Pulling image "mysql/mysql-server:8.0.31" Normal Pulled 19m kubelet Successfully pulled image "mysql/mysql-server:8.0.31" in 3.660939128s Normal Created 19m kubelet Created container initmysql Normal Started 19m kubelet Started container initmysql Normal Pulling 19m kubelet Pulling image "mysql/mysql-operator:8.0.31-2.0.7" Normal Created 19m kubelet Created container sidecar Normal Pulled 19m kubelet Successfully pulled image "mysql/mysql-operator:8.0.31-2.0.7" in 5.980011863s Normal Started 19m kubelet Started container sidecar Error Logging 19m kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
[14 Dec 2022 7:45]
Dileep AP
I also get the same error. K8s Version - v1.23 OS - Ubuntu 20.04.5 LTS mysql-operator - 8.0.31 44s Normal Logging pod/mysql-cluster-1 POD CREATED: pod=mysql-cluster-1 ContainersReady=False Ready=False gate[configured]=None 44s Error Logging pod/mysql-cluster-1 Handler 'on_pod_create' failed temporarily: Sidecar of mysql-cluster-1 is not yet configured 44s Normal Logging pod/mysql-cluster-0 POD CREATED: pod=mysql-cluster-0 ContainersReady=False Ready=False gate[configured]=None 44s Error Logging pod/mysql-cluster-0 Handler 'on_pod_create' failed temporarily: Sidecar of mysql-cluster-0 is not yet configured 14s Normal Logging pod/mysql-cluster-2 POD CREATED: pod=mysql-cluster-2 ContainersReady=False Ready=False gate[configured]=None 14s Error Logging pod/mysql-cluster-2 Handler 'on_pod_create' failed temporarily: Sidecar of mysql-cluster-2 is not yet configured xxx@xxxx24:~/dileep/mysqlorgi/mysql-innodbcluster$ kubectl get all -n mysql-cluster NAME READY STATUS RESTARTS AGE pod/mysql-cluster-2 0/2 Init:CrashLoopBackOff 7 (2m20s ago) 13m pod/mysql-cluster-1 0/2 Init:CrashLoopBackOff 7 (2m23s ago) 13m pod/mysql-cluster-0 0/2 Init:CrashLoopBackOff 7 (2m20s ago) 13m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/mysql-cluster-instances ClusterIP None <none> 3306/TCP,33060/TCP,33061/TCP 13m service/mysql-cluster ClusterIP 10.152.183.98 <none> 3306/TCP,33060/TCP,6446/TCP,6448/TCP,6447/TCP,6449/TCP 13m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/mysql-cluster-router 0/0 0 0 13m NAME DESIRED CURRENT READY AGE replicaset.apps/mysql-cluster-router-777bf96649 0 0 0 13m NAME READY AGE statefulset.apps/mysql-cluster 0/3 13m xxx@xxxx24:~/dileep/mysqlorgi/mysql-innodbcluster$
[28 Aug 2023 14:12]
Samantha Jones
I have encountered a persistent issue when attempting to set up an InnoDBCluster in my Kubernetes environment. I followed the sample configuration provided until here:
yaml
Copy code
apiVersion: mysql.oracle.com/v2
kind: InnoDBCluster
metadata:
name: mycluster
spec:
secretName: mypwds
tlsUseSelfSigned: true
instances: 3
version: 8.0.29
router:
instances: 1
version: 8.0.29
datadirVolumeClaimTemplate:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 4Gi
mycnf: |
[mysqld]
max_connections=162
I then deployed this instance in the test namespace, and the logs of all three nodes consistently report the error message: "Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-X is not yet configured". This issue seems to persist despite multiple retries.
I believe this information might be relevant:
sql
Copy code
Normal Logging 44s kopf POD CREATED: pod=mycluster-0 ContainersReady=False Ready=False gate[configured]=None
Error Logging 44s kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
Normal Created 43s (x3 over 58s) kubelet Created container initmysql
Normal Pulled 43s (x3 over 58s) kubelet Successfully pulled image "hub.intra.mlamp.cn/public/mysql/mysql-server:8.0.29" https://trustanalytica.com/online/best-urgent-essay-writing-services
Warning Failed 43s (x3 over 58s) kubelet Error: failed to start container "initmysql": Error response from daemon: linux spec capabilities: Unknown capability drop: "AUDIT_READ"
Warning BackOff 42s (x5 over 54s) kubelet Back-off restarting failed container
Warning Logging 34s kopf Patching failed with inconsistencies: (('remove', ('status', 'kopf'), {'progress': {'on_pod_create': {'started': '2022-05-21T09:16:40.501856', 'stopped': None, 'delayed': '2022-05-21T09:17:20.958252', 'purpose': 'create', 'retries': 4, 'success': False, 'failure': False, 'message': 'Sidecar of mycluster-0 is not yet configured', 'subrefs': None}}}, None),)
Error Logging 34s kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-0 is not yet configured
I hope this additional information can help in diagnosing and resolving this issue. Your assistance is greatly appreciated.
[29 Aug 2023 15:10]
Andrey Hristov
Posted by developer: The problem exists in operator 8.0.29 and got fixed in 8.0.30-2.05. Please upgrade to Operator 8.0.34-2.0.11
[28 Nov 2023 1:47]
sheng mei
I have met the same issue in latest version 8.2.0-2.1.1 mysql server is 8.2.0.
my OS is CentOS Linux release 7.6.1810 (Core) kernel version is 3.10.0-957.el7.x86_64, the k8s version is v1.24.13.
POD describe as below:
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 16m default-scheduler Successfully assigned mysql-operator/mycluster-2 to worker4
Normal SuccessfulAttachVolume 16m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-f1f07348-01f6-441e-8299-90af3f04a974"
Normal Logging 16m kopf POD CREATED: pod=mycluster-2 ContainersReady=False Ready=False gate[configured]=None
Error Logging 16m kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-2 is not yet configured
Normal Created 16m kubelet Created container fixdatadir
Normal Pulling 16m kubelet Pulling image "registry.example.com:8443/mysql-operator/community-operator:8.2.0-2.1.1"
Normal Pulled 16m kubelet Successfully pulled image "registry.example.com:8443/mysql-operator/community-operator:8.2.0-2.1.1" in 81.510448ms
Normal Started 16m kubelet Started container fixdatadir
Normal Created 16m kubelet Created container initconf
Normal Pulling 16m kubelet Pulling image "registry.example.com:8443/mysql-operator/community-operator:8.2.0-2.1.1"
Normal Pulled 16m kubelet Successfully pulled image "registry.example.com:8443/mysql-operator/community-operator:8.2.0-2.1.1" in 55.733362ms
Normal Started 16m kubelet Started container initconf
Normal Pulled 16m kubelet Successfully pulled image "registry.example.com:8443/mysql-operator/mysql-server:8.2.0" in 53.543097ms
Normal Pulled 16m kubelet Successfully pulled image "registry.example.com:8443/mysql-operator/mysql-server:8.2.0" in 78.212583ms
Normal Pulling 16m (x3 over 16m) kubelet Pulling image "registry.example.com:8443/mysql-operator/mysql-server:8.2.0"
Normal Created 16m (x3 over 16m) kubelet Created container initmysql
Normal Started 16m (x3 over 16m) kubelet Started container initmysql
Normal Pulled 16m kubelet Successfully pulled image "registry.example.com:8443/mysql-operator/mysql-server:8.2.0" in 53.030075ms
Normal Logging 16m kopf POD CREATED: pod=mycluster-2 ContainersReady=False Ready=False gate[configured]=None
Error Logging 16m kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-2 is not yet configured
Normal Logging 15m kopf POD CREATED: pod=mycluster-2 ContainersReady=False Ready=False gate[configured]=None
Error Logging 15m kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-2 is not yet configured
Error Logging 15m kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-2 is not yet configured
Normal Logging 15m kopf POD CREATED: pod=mycluster-2 ContainersReady=False Ready=False gate[configured]=None
Error Logging 14m kopf Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-2 is not yet configured
Normal Logging 14m kopf POD CREATED: pod=mycluster-2 ContainersReady=False Ready=False gate[configured]=None
Normal Logging 14m kopf POD CREATED: pod=mycluster-2 ContainersReady=False Ready=False gate[configured]=None
can someone help me to fix the issue? thanks very much.
[28 Nov 2023 3:30]
sheng mei
Supplement mysql log： 2023-11-28T11:31:23.96833615+08:00 stdout F 2023-11-28T03:31:23.961413Z 0 [Note] [MY-010098] [Server] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled 2023-11-28T11:31:23.968356348+08:00 stdout F 2023-11-28T03:31:23.961506Z 0 [Note] [MY-013932] [Server] BuildID[sha1]=f6c339ddbc13c4a34c0ac687cd9adf25ca26a1ed 2023-11-28T11:31:23.968362074+08:00 stdout F 2023-11-28T03:31:23.961516Z 0 [Note] [MY-010949] [Server] Basedir set to /opt/bitnami/mysql/. 2023-11-28T11:31:23.968367016+08:00 stdout F 2023-11-28T03:31:23.961528Z 0 [System] [MY-010116] [Server] /opt/bitnami/mysql/bin/mysqld (mysqld 8.2.0) starting as process 1 2023-11-28T11:31:23.968372083+08:00 stdout F 2023-11-28T03:31:23.967268Z 0 [Warning] [MY-010091] [Server] Can't create test file /bitnami/mysql/data/mysqld_tmp_file_case_insensitive_test.lower-test 2023-11-28T11:31:23.968376957+08:00 stdout F 2023-11-28T03:31:23.967281Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /bitnami/mysql/data/ is case insensitive 2023-11-28T11:31:23.968381922+08:00 stdout F 2023-11-28T03:31:23.967616Z 0 [Warning] [MY-010122] [Server] One can only use the --user switch if running as root 2023-11-28T11:31:23.968447667+08:00 stdout F mysqld: File './mycluster.index' not found (OS errno 30 - Read-only file system) 2023-11-28T11:31:23.968463526+08:00 stdout F 2023-11-28T03:31:23.968407Z 0 [ERROR] [MY-010119] [Server] Aborting 2023-11-28T11:31:23.968576554+08:00 stdout F 2023-11-28T03:31:23.968514Z 0 [Note] [MY-010120] [Server] Binlog end 2023-11-28T11:31:23.968757984+08:00 stdout F 2023-11-28T03:31:23.968698Z 0 [System] [MY-010910] [Server] /opt/bitnami/mysql/bin/mysqld: Shutdown complete (mysqld 8.2.0) Source distribution. 2023-11-28T11:31:23.968859769+08:00 stdout F 2023-11-28T03:31:23.968806Z 0 [Note] [MY-015021] [Server] MySQL Server: Components Shutdown - start. 2023-11-28T11:31:23.970019276+08:00 stdout F 2023-11-28T03:31:23.969953Z 0 [Note] [MY-015022] [Server] MySQL Server: Components Shutdown - end (with return value = 0). 2023-11-28T11:31:23.970155378+08:00 stdout F 2023-11-28T03:31:23.970101Z 0 [System] [MY-015016] [Server] MySQL Server - end.
[1 Feb 2024 9:57]
Arun Lal
I have installed the InnoDB Cluster on Kubernetes, for the first few minutes there were no issues, but after that, the pods started terminating.
NAME READY STATUS RESTARTS AGE
mycluster-0 2/2 Terminating 0 3h21m
mycluster-1 0/2 Terminating 8 (53m ago) 3h21m
mycluster-2 2/2 Terminating 0 3h21m
mycluster-router-77b8ddfb4-2x7zk 1/1 Terminating 1 (3h19m ago) 3h20m
mycluster-router-77b8ddfb4-crvst 0/1 CrashLoopBackOff 15 (2m11s ago) 43m
mysql-client-pod 1/1 Running 0 18m
mysql-operator-ff46567db-2wvkf 1/1 Terminating 0 3h22m
mysql-operator-ff46567db-vwmld 1/1 Running 0 43m
I am showing one of the Pod logs.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 60m (x175 over 3h20m) kubelet Liveness probe failed: command "/livenessprobe.sh" timed out
Normal Logging 56m kopf Could not connect to mycluster-1.mycluster-instances.mysql-operator.svc.cluster.local:3306: error=MySQL Error (2005): mysqlsh.connect_dba: Unknown MySQL server host 'mycluster-1.mycluster-instances.mysql-operator.svc.cluster.local' (-2)
Normal Logging 56m kopf diag instance mycluster-1 --> InstanceDiagStatus.OFFLINE quorum=None gtid_executed=None
Normal Logging 56m kopf mycluster-1.mycluster-instances.mysql-operator.svc.cluster.local:3306: pod.phase=Running deleting=True
Normal Logging 56m kopf Could not connect to mycluster-2.mycluster-instances.mysql-operator.svc.cluster.local:3306: error=MySQL Error (2005): mysqlsh.connect_dba: Unknown MySQL server host 'mycluster-2.mycluster-instances.mysql-operator.svc.cluster.local' (-2)
Normal Logging 56m kopf diag instance mycluster-2 --> InstanceDiagStatus.UNKNOWN quorum=None gtid_executed=None
Normal Logging 56m kopf mycluster-2.mycluster-instances.mysql-operator.svc.cluster.local:3306: pod.phase=Running deleting=False
Normal Logging 56m kopf mycluster: all={<MySQLPod mycluster-1>, <MySQLPod mycluster-2>, <MySQLPod mycluster-0>} members={<MySQLPod mycluster-2>, <MySQLPod mycluster-1>, <MySQLPod mycluster-0>} online={<MySQLPod mycluster-0>} offline={<MySQLPod mycluster-1>} unsure={<MySQLPod mycluster-2>}
Normal Logging 56m kopf diag instance mycluster-0 --> InstanceDiagStatus.ONLINE quorum=True gtid_executed=72c7fc4e-c0cb-11ee-aa18-ea86c48973af:1-8,
8a92c481-c0cb-11ee-96f3-ea86c48973af:1-1442
Normal Logging 56m kopf Group view of mycluster-0.mycluster-instances.mysql-operator.svc.cluster.local:3306 has dict_keys(['mycluster-0.mycluster-instances.mysql-operator.svc.cluster.local:3306', 'mycluster-1.mycluster-instances.mysql-operator.svc.cluster.local:3306', 'mycluster-2.mycluster-instances.mysql-operator.svc.cluster.local:3306']) but these are not ONLINE: {'mycluster-2.mycluster-instances.mysql-operator.svc.cluster.local:3306', 'mycluster-1.mycluster-instances.mysql-operator.svc.cluster.local:3306'}
Error Logging 56m kopf Handler 'on_pod_delete' failed temporarily: Cluster status results inconsistent
Normal Logging 56m kopf diag instance mycluster-1 --> InstanceDiagStatus.ONLINE quorum=True gtid_executed=72c7fc4e-c0cb-11ee-aa18-ea86c48973af:1-8,
8a92c481-c0cb-11ee-96f3-ea86c48973af:1-1444
Normal Logging 56m kopf diag instance mycluster-0 --> InstanceDiagStatus.ONLINE quorum=True gtid_executed=72c7fc4e-c0cb-11ee-aa18-ea86c48973af:1-8,
8a92c481-c0cb-11ee-96f3-ea86c48973af:1-1446
Normal Logging 56m kopf mycluster: all={<MySQLPod mycluster-1>, <MySQLPod mycluster-2>, <MySQLPod mycluster-0>} members={<MySQLPod mycluster-1>, <MySQLPod mycluster-2>, <MySQLPod mycluster-0>} online={<MySQLPod mycluster-1>, <MySQLPod mycluster-2>, <MySQLPod mycluster-0>} offline=set() unsure=set()
Normal Logging 56m kopf Removing mycluster-1.mycluster-instances.mysql-operator.svc.cluster.local:3306 from cluster
Normal Logging 56m kopf cluster probe: status=ClusterDiagStatus.ONLINE online=[<MySQLPod mycluster-0>, <MySQLPod mycluster-1>, <MySQLPod mycluster-2>]
Normal Logging 56m kopf remove_instance: mycluster-1 peer=mycluster-0 options={}
Normal Logging 56m kopf Connected to mycluster-0
Normal Logging 55m kopf Removed finalizer for pod mycluster-1
Normal Logging 55m kopf diag instance mycluster-1 --> InstanceDiagStatus.UNMANAGED quorum=None gtid_executed=72c7fc4e-c0cb-11ee-aa18-ea86c48973af:1-8,
8a92c481-c0cb-11ee-96f3-ea86c48973af:1-1450
I don't understand what is happening, and I followed the official documentation for this setup.
link: https://dev.mysql.com/doc/mysql-operator/en/mysql-operator-installation.html
[18 Dec 2024 20:50]
behzad shaghaghi
hi we have this issue in 8.4.1-2.1.4 version
Description: I have followed the sample configuration until here: apiVersion: mysql.oracle.com/v2 kind: InnoDBCluster metadata: name: mycluster spec: secretName: mypwds tlsUseSelfSigned: true instances: 3 router: instances: 1 Then these instances have not been successfully started, and the following errors are reported in all three instances: Handler 'on_pod_create' failed temporarily: Sidecar of mycluster-X is not yet configured I checked the documentation and didn't find where to configure the sidecar How to repeat: Use kubectl to deploy the operator, then deploy this: https://dev.mysql.com/doc/mysql-operator/en/mysql-operator-innodbcluster-simple-kubectl.ht...