| Bug #117073 | Cannot specify resource requests/limits for backup job, resulting in OOM Killed | ||
|---|---|---|---|
| Submitted: | 30 Dec 2024 11:27 | Modified: | 9 Jan 17:11 |
| Reporter: | Martin Fleurke (OCA) | Email Updates: | |
| Status: | Verified | Impact on me: | |
| Category: | MySQL Operator | Severity: | S2 (Serious) |
| Version: | 9.1.0-2.2.2 | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
[7 Jan 13:41]
Martin Fleurke
The problem is fixed with: https://github.com/mysql/mysql-operator/pull/38
[8 Jan 0:41]
OCA Admin
Contribution submitted via Github - fix for Bug #117073 - Cannot specify resource requests/limits for bac… (*) Contribution by Martin Fleurke (Github amfleurke, mysql-operator/pull/38#issuecomment-2575582289): I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.
Contribution: git_patch_2263931554.txt (text/plain), 7.53 KiB.
Description: When you specify a backupProfile for an Innodbclusters resource, you cannot specify the resource.requests or .limits for the backup pod/container If your cluster has low defaults (via a LimitRange), e.g. 128Mi memory, the pod will get OOMKilled by kubernetes. How to repeat: Deploy a limitRange to your kubernetes luster: apiVersion: v1 kind: LimitRange metadata: name: mynotbypassablelimitrange spec: limits: - default: cpu: 100m memory: 128Mi defaultRequest: cpu: 100m memory: 128Mi type: Container Deploy a cluster with a backup profile and schedule, e.g.: backupProfiles: - name: myprofile dumpInstance: dumpOptions: excludeSchemas: ["information_schema","mysql", "mysql_innodb_cluster_metadata", "performance_schema", "sys"] storage: s3: prefix: "mytestcluster" config: mysecret bucketName: mybucket profile: "user1" endpoint: http://minio:9000 backupSchedules: - name: schedule-for-dump schedule: "56 1 * * *" timeZone: "Europe/Amsterdam" deleteBackupData: false backupProfileName: myprofile enabled: true This will result in a cronjob creating a job creating a job with pods that have memory limit of 128Mi and 100m CPU. Suggested fix: besides backupProfiles: - podLabels: .. podAnnotations: ... also have podSpec: ... similar to podSpec for the innoDBcluster pods. or, at least support: backupProfiles: - resources: ... e.g. backupProfiles: - resources: requests: memory: 1Gi cpu: 1 limits: memory: 1Gi cpu: 2