Bug #118528 Connector/J Memory Leak Problem
Submitted: 26 Jun 2025 8:43 Modified: 12 Mar 0:01
Reporter: ünal polat Email Updates:
Status: Open Impact on me:
None 
Category:Connector / J Severity:S3 (Non-critical)
Version:9.1.0 OS:Any
Assigned to: Assigned Account CPU Architecture:Any
Tags: Connector/J, memory-leak

[26 Jun 2025 8:43] ünal polat
Description:
Hi,
After upgrading Spring Boot 3.4.x, mysql version upgraded to 9.1.0 from 8.3.0.

In mysql 9.1.0, OpenTelemetry features comes in mysql connector/j library:
https://dev.mysql.com/doc/connector-j/en/connector-j-opentelemetry.html

We use OpenTelemetry to observe our application, and the default openTelemetry prop for mysql is PREFERRED, and mysql connector/j OpenTelemetry features become available due to usage of OpenTelemetry in our applications.

When we heap dump any application that have OpenTelemetry, we found a memory leak in com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.ConnectionFinalizerPhantomReference

When we examine this class objects, we found the root of the leak:

ConnectionFinalizerPhantomReference -> (com.mysql.cj.jdbc.ConnectionImpl) referent -> (com.mysql.cj.NativeSession) session -> (com.mysql.cj.otel.OpenTelemetryHandler) telemetryHandler -> (java.utilWeakHashMap) spans

This spans map never emptied and only used to store com.mysql.cj.telemetry.TelemetrySpan - io.opentelemetry.api.trace.Span relation. The only thing I saw, there is a linkTargest list object and spans map only used to add or remove from that list.

Therefore, the spans map becomes bigger and bigger when time passes. This causes huge memory usages by this map in our applicatons. Somehow the garbage collector did not clean this object.

The linkTargest list object does not have so much area in heap. I think that is because of the algortim when closing connections. You can see that behavior in com.mysql.cj.jdbc.ConnectionImpl#doClose method via calling this.session.getTelemetryHandler().removeLinkTarget(this.connectionSpan).

This kinda empty the list object but the spans map remains same although the connection closed.

How to repeat:
1. Use Mysql 9.1.0 for mysql-connector-j library.
2. Make sure io.opentelemetry.api.GlobalOpenTelemetry class exist in your application.
3. Make sure com.mysql.cj.otel.OpenTelemetryHandler initialized when new session created via com.mysql.cj.NativeSession#NativeSession
4. Just run the program.
5. Wait enough to see the growing size of spans map. It may vary on your application request count. 

Suggested fix:
I suggest when we closing ConnectionImpl, the linkTargets are removing via

this.session.getTelemetryHandler().removeLinkTarget(this.connectionSpan)

After that, we can remove this connectionSpan from spans too. In other words, the removing link target logic is:

public void removeLinkTarget(TelemetrySpan span) {
    Span otelSpan = (Span)this.spans.get(span);
    if (otelSpan != null) {
      this.linkTargets.remove(otelSpan);
    }

why not we remove otelSpan from spans map?
[8 Jul 2025 19:55] Filipe Silva
Hi Ünal Polat,

Thank you for your interest in MySQL Connector/J and for taking the time to report this issue.

As it stands, the current implementation doesn't require the spans map to be explicitly cleared. The use of a WeakHashMap in combination with the garbage collector is intended to handle this automatically.

The fix you suggested doesn’t have a meaningful effect, as it would only remove a single entry from the map—the one corresponding to the span promoted as a link target (i.e., the connection creation span). All other spans are short-lived and, once the operation is complete, no strong references remain. As such, they should be eligible for garbage collection and automatically removed from the map.

Are you certain that the garbage collector is not cleaning up the spans map? In my observations, this behavior appears to be working as expected.

Would you be able to provide a minimal test case that reproduces the issue without involving Spring Boot? Ideally, it would include only Connector/J, OpenTelemetry, and a bit of test code to help isolate the behavior. Also, please use the latest MySQL Connector/J version.
[9 Aug 2025 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[11 Mar 19:34] Saurabh Anand
Facing the same issue. 

OpenTelemetryHandler maintains a WeakHashMap field named spans. When a connection is closed via Connection.close() or try-with-resources, this map is not cleared. 

After closure, 1 entry remains permanently in the map and is never reclaimed, neither by close() nor by repeated System.gc() calls.

This is a deterministic per-connection memory leak. In long-running applications, full GC cycles are infrequent by design; the WeakHashMap entries that GC can collect sit unreleased for extended periods, and the 1 entry GC cannot collect accumulates permanently, making the "GC will handle it" assumption incorrect for production workloads.
[11 Mar 19:36] Saurabh Anand
Adding Simple Java code to reproduce the issue, via attached file.

Proposed solution :

Three coordinated changes are required. A single spans.remove() in removeLinkTarget() is insufficient because spans accumulates entries beyond the connectionSpan, all telemetry state for a connection must be released at close time.

1. TelemetryHandler interface — add close() default no-op:
default void close() {
    // no-op
}

2. OpenTelemetryHandler — implement close():
@Override
public void close() {
    this.spans.clear();
    this.linkTargets.clear();
}

3. ConnectionImpl.doClose() — call handler.close() before replacing with no-op:
TelemetryHandler handler = this.session.getTelemetryHandler();
handler.removeLinkTarget(this.connectionSpan);
handler.close();
this.session.setTelemetryHandler(NoopTelemetryHandler.getInstance());

This ensures all telemetry state is released deterministically at connection close, before the handler reference is dropped. No GC dependency, no WeakHashMap eviction race.

Tested with local patch and it releases the resources at connection.close(). Happy to submit a patch if required.
[11 Mar 19:49] Saurabh Anand
Github link for code to reproduce the leak: https://github.com/iSaurabhAnand/connectorJ-Leak/blob/main/src/main/java/com/example/OTelL...