Bug #52313 selects retrieving many blob rows take too long freeing allocated RAM

• Submitted: 23 Mar 2010 19:18
• Modified: 2 Sep 2010 15:15
• Reporter: Andrew Hutchings
• Status: Closed
• Category: MySQL Cluster: Cluster (NDB) storage engine
• Severity: S3 (Non-critical)
• Version: mysql-5.1-telco-6.3
• OS: Any
• Assigned to: Frazer Clement
• CPU Architecture: Any

[23 Mar 2010 19:18] Andrew Hutchings

Description:
When running a select on a table with a cluster blob memory appears to be allocated for the blobs but is not freed until the end of the select.

This can cause problems for things such as mysqldump on a large table where many gigabytes of RAM may be required to dump a table.

How to repeat:
1. create table test.t1 (a int auto_increment primary key, b text) engine=ndb
2. insert lots of rows (in my case I used about 200,000 rows each with 1Kbyte blobs)
3. mysqldump test > test.sql
4. watch virt and res usage for mysqld as the dump progresses.

[23 Mar 2010 19:20] Andrew Hutchings

Reproduced in 7.0.9 - 7.0.13

[2 Sep 2010 9:50] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/117407

3128 Frazer Clement	2010-09-02
      bug#52313 selects retrieving many blob rows take too long freeing allocated RAM
      
      Release operations between Scan nextResult() calls.
      This frees internal operations created to read Blob part table rows.
      This avoids excessive memory consumption when scanning a table containing Blobs.
      
      A new bug (56488) NdbApi : Improve Blob/internal operation management has been 
      created to implement a more general solution to this and other problems at
      the NdbApi level.

[2 Sep 2010 10:09] Bugs System

Pushed into mysql-5.1-telco-6.3 5.1.47-ndb-6.3.38 (revid:[e-mail]-20100902095057-z5g280fpv7s6y7yh) (version source revid:[e-mail]-20100902095057-z5g280fpv7s6y7yh) (merge vers: 5.1.47-ndb-6.3.38) (pib:21)

[2 Sep 2010 10:09] Bugs System

Pushed into mysql-5.1-telco-7.0 5.1.47-ndb-7.0.19 (revid:[e-mail]-20100902095335-a9o49eb71mc76uxu) (version source revid:[e-mail]-20100902095335-a9o49eb71mc76uxu) (merge vers: 5.1.47-ndb-7.0.19) (pib:21)

[2 Sep 2010 10:10] Bugs System

Pushed into mysql-5.1-telco-6.2 5.1.47-ndb-6.2.19 (revid:[e-mail]-20100902094749-rn5ibli1jm2m9pgb) (version source revid:[e-mail]-20100902094749-rn5ibli1jm2m9pgb) (merge vers: 5.1.47-ndb-6.2.19) (pib:21)

[2 Sep 2010 10:21] Frazer Clement

Also pushed to 7.1.8

[2 Sep 2010 15:15] Jon Stephens

Documented bugfix in the NDB-6.2.19, 6.3.38, 7.0.19, and 7.1.8 changelogs, as follows:

        When running a SELECT on an NDB table with BLOB or TEXT columns,
        memory was allocated for the columns but was not freed until the
        end of the SELECT. This could cause problems when dumping tables
        with such columns and having many rows.

Closed.

[25 Oct 2012 7:38] Cyril SCETBON

Hi,

can we just update our mysql server or do we need to update the whole cluster ?

thanks