Bug #116957 kubernetes operator report an exception, the code doesn't seem to have any error handling
Submitted: 12 Dec 2024 2:55 Modified: 10 Jan 14:14
Reporter: Bing Ma (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Operator Severity:S3 (Non-critical)
Version:8.3.0-2.1.2 OS:Linux
Assigned to: CPU Architecture:Any

[12 Dec 2024 2:55] Bing Ma
Description:
the operator report an exception

Traceback (most recent call last):
  File "/usr/lib64/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/usr/lib/mysqlsh/python-packages/mysqloperator/controller/group_monitor.py", line 219, in run
    cluster.ensure_connected()
  File "/usr/lib/mysqlsh/python-packages/mysqloperator/controller/group_monitor.py", line 55, in ensure_connected
    self.connect_to_primary()
  File "/usr/lib/mysqlsh/python-packages/mysqloperator/controller/group_monitor.py", line 71, in connect_to_primary
    session, is_primary = self.find_primary()
  File "/usr/lib/mysqlsh/python-packages/mysqloperator/controller/group_monitor.py", line 108, in find_primary
    session = self.try_connect(pod)
  File "/usr/lib/mysqlsh/python-packages/mysqloperator/controller/group_monitor.py", line 134, in try_connect
    session = mysqlx.get_session(pod.xendpoint_co)
  File "/usr/lib/mysqlsh/python-packages/mysqloperator/controller/innodbcluster/cluster_api.py", line 2188, in xendpoint_co
    self.admin_account = self.get_cluster().get_admin_account()
AttributeError: 'NoneType' object has no attribute 'get_admin_account'

How to repeat:
My testing colleague found the following error:

GroupMonitor: Error fetching notice: 
  dest=fb-mgr-0926-1.fb-mgr-0926-instances.default.svc.cluster.local:33060 
  error=MySQL Error (2006): Session.run_sql: MySQL server has gone away

GroupMonitor: Trying to connect to a member of cluster default/fb-mgr-0926
Could not get cluster default/fb-mgr-0926: (404)

Exception in thread group-monitor:
  Reason: Not Found
  HTTP response headers: 
    HTTPHeaderDict({
      'Audit-Id': '867ed596-391a-46b0-8260-8f2afa12a9a5',
      'Cache-Control': 'no-cache, private',
      'Content-Type': 'application/json',
      'X-Kubernetes-Pf-Flowschema-Uid': '917c84f8-333b-4ebd-8f5a-d8862f540cd9',
      'X-Kubernetes-Pf-Prioritylevel-Uid': '4040c529-af82-4aab-807e-90ae27696687',
      'Date': 'Wed, 25 Sep 2024 03:31:07 GMT',
      'Content-Length': '254'
    })
  HTTP response body: 
    {
      "kind": "Status",
      "apiVersion": "v1",
      "metadata": {},
      "status": "Failure",
      "message": "innodbclusters.mysql.oracle.com \"fb-mgr-0926\" not found",
      "reason": "NotFound",
      "details": {
        "name": "fb-mgr-0926",
        "group": "mysql.oracle.com",
        "kind": "innodbclusters"
      },
      "code": 404
    }

Suggested fix:
This exception should be caught
[10 Jan 14:14] MySQL Verification Team
Thank you for the report