MySQL Bugs: #15089: BLOB data should not remain in memory

Bug #15089	BLOB data should not remain in memory
Submitted:	21 Nov 2005 5:24	Modified:	20 May 9:32
Reporter:	Gili
Status:	Verified
Category:	Server	Severity:	S2 (Serious)
Version:	3.1.10, 4.1, 5.0, 5.1, 6.0 bzr	OS:	Any (Linux, Windows XP sp2)
Assigned to:		Target Version:
Tags:	affects_connectors
Triage:	Triaged: D5 (Feature request)

Description:
Currently com.mysql.Blob stores the entire BLOB data in memory at all times. This defeats
the entire purpose of BLOB which is made to hold huge amounts of data.

I need this issue fixed as soon as possible because my application makes heavy use of
BLOBs and quickly runs out of memory.

Expected behavior: BLOB methods should retrieve data sub-sets from the server on-demand
and not keep a local copy around (nor the entire BLOB value, since often only a small
subset is actually requested). If you want a local cache, feel free to use SoftReference
for that purpose.

If you have a workaround in mind I am all ears :)

How to repeat:
Loads Blobs from DB, one at a time, modify them slightly, and write them back into the
DB.

The problem is that I want to have an Image object in memory without necessarily having
its data (blob) in memory for the same lifetime.

Due to the evils of Hibernate, I cannot always unload the enclosing Image from memory
when I want to.

Suggested fix:
Use SoftReferences for local cache. Only retrieve requested subsets of BLOB, not entire
value.

This is a limitation with the server, neither the MySQL network protocol, nor the server
itself allow a client to retrieve a BLOB piecemeal in an _efficient_ fashion.

(The JDBC driver can do this if "emulateLocators" is set to true in the configuration
properties, but this still requires the _server_ to load the ENTIRE LOB for each request
of a "chunk" of it from the input stream returned to the application by the JDBC driver).

Yes, this is exists and it is known.

Correct state for this bug is "Won't fix" or "Not a Bug" though.

This partially fixed in patch for bug #38002.

Bug #48085 was marked as duplicate of this one.