MySQL Bugs: #90874: ClusterJ exception "The method is not valid in current blob state"

Bug #90874	ClusterJ exception "The method is not valid in current blob state"
Submitted:	15 May 2018 17:01	Modified:	7 Sep 2018 22:41
Reporter:	Andrew Tikhonov	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster/J	Severity:	S2 (Serious)
Version:	7.4.13	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	clusterJ BLOB TEXT exception

Description:
If a table contains a BLOB or TEXT field, and the table is queried using clusterJ library for a record that doesn't exist, the following exception is thrown.

Strangely, if the table is not populated, exception is not thrown. Need to populate table with a few records.

Workarounds are welcome.

SEVERE: Error in NdbJTie: returnCode -1, code 4,265, mysqlCode -1, status 2, classification 1, message The method is not valid in current blob state .
com.mysql.clusterj.ClusterJDatastoreException: Error in NdbJTie: returnCode -1, code 4,265, mysqlCode -1, status 2, classification 1, message The method is not valid in current blob state .
	at com.mysql.clusterj.tie.Utility.throwError(Utility.java:1145)
	at com.mysql.clusterj.tie.Utility.throwError(Utility.java:1131)
	at com.mysql.clusterj.tie.BlobImpl.handleError(BlobImpl.java:179)
	at com.mysql.clusterj.tie.BlobImpl.getLength(BlobImpl.java:86)
	at com.mysql.clusterj.tie.NdbRecordBlobImpl.readData(NdbRecordBlobImpl.java:97)
	at com.mysql.clusterj.tie.NdbRecordOperationImpl.loadBlobValues(NdbRecordOperationImpl.java:950)
	at com.mysql.clusterj.tie.NdbRecordUniqueKeyOperationImpl$1.run(NdbRecordUniqueKeyOperationImpl.java:50)
	at com.mysql.clusterj.tie.ClusterTransactionImpl.performPostExecuteCallbacks(ClusterTransactionImpl.java:579)
	at com.mysql.clusterj.tie.ClusterTransactionImpl.executeNoCommit(ClusterTransactionImpl.java:207)
	at com.mysql.clusterj.tie.NdbRecordOperationImpl.resultData(NdbRecordOperationImpl.java:453)
	at com.mysql.clusterj.tie.NdbRecordOperationImpl.resultData(NdbRecordOperationImpl.java:444)
	at com.mysql.clusterj.core.query.QueryDomainTypeImpl.getResultData(QueryDomainTypeImpl.java:314)
	at com.mysql.clusterj.core.query.QueryDomainTypeImpl.getResultList(QueryDomainTypeImpl.java:181)
	at com.mysql.clusterj.core.query.QueryImpl.getResultList(QueryImpl.java:144)
	at persistence.dao.NdbClusterBLOBTest.doTest(NdbClusterBLOBTest.java:68)
	at persistence.dao.NdbClusterBLOBTest.main(NdbClusterBLOBTest.java:90)
Exception in thread "main" com.mysql.clusterj.ClusterJDatastoreException: Error in NdbJTie: returnCode -1, code 4,265, mysqlCode -1, status 2, classification 1, message The method is not valid in current blob state .

How to repeat:
    CREATE TABLE IF NOT EXISTS `testblob` (
      `id` bigint(20) NOT NULL,
      `session` bigint(20) DEFAULT NULL COMMENT 'SESSION ID',
      `payload` BLOB DEFAULT NULL COMMENT 'JSON PROFILE',
      PRIMARY KEY (`id`),
      UNIQUE KEY `testblob_UNIQUE` (`session`),
      INDEX `id_idx` (`id`),
      INDEX `session_idx` (`session`) USING BTREE
    ) ENGINE=ndbcluster DEFAULT CHARSET=latin1;

    insert into testblob (`id`,`session`,`payload`) values (1,11111111,'test payload 1');
    insert into testblob (`id`,`session`,`payload`) values (2,22222222,'test payload 2');
    insert into testblob (`id`,`session`,`payload`) values (3,33333333,'test payload 3');

@PersistenceCapable(table = "testblob")
@JsonPropertyOrder({"id", "", "session", "payload"})
@Indices({
        @Index(name = "session_idx", columns = {@Column(name = "session")})
})
public interface NdbClusterRecord {

    @PrimaryKey
    @JsonSerialize(using = ToStringSerializer.class)
    @Column(name = "id")
    long getId();
    void setId(long id);

    @JsonSerialize(using = ToStringSerializer.class)
    @Column(name = "session")
    long getSession();
    void setSession(long session);

    @Column(name = "payload")
    byte[] getPayload();
    void setPayload(byte[] payload);
}

    private void doTest() {

        final Properties props = new Properties();
        props.put("com.mysql.clusterj.connectstring", "localhost:1186");
        props.put("com.mysql.clusterj.database", "db");
        props.put("com.mysql.clusterj.connect.retries", "1");
        props.put("com.mysql.clusterj.connect.delay", "5");
        props.put("com.mysql.clusterj.connect.verbose", "1");
        props.put("com.mysql.clusterj.connect.timeout.before", "5");
        props.put("com.mysql.clusterj.connect.timeout.after", "5");
        props.put("com.mysql.clusterj.connection.pool.size", "1");

        SessionFactory sessionFactory = ClusterJHelper.getSessionFactory(props);

        Session session = sessionFactory.getSession();

        try {
            //QueryBuilder qb = session.getQueryBuilder();

            QueryDomainType<NdbClusterRecord> domain =
                    session.getQueryBuilder().createQueryDefinition(NdbClusterRecord.class);

            Query<NdbClusterRecord> query = session.createQuery(domain);

            query.setParameter("session", 12345678L);
            //query.setParameter("session", 22222222L);

            domain.where(domain.get("session").equal(domain.param("session")));

            List<NdbClusterRecord> results = query.getResultList();

            System.out.println("size:" + results.size());
            for (NdbClusterRecord r : results) {
                System.out.println("r:" + r.getId() + " session:" +
                        r.getSession() + " payload:" + new String(r.getPayload()));
            }

        } finally {
            session.close();
        }
    }

Suggested fix:
method readData needs to make sure the data is present before reading the length of it. Reading the length of not existing data is considered by the underlying layer an invalid operation and results in error code 4265.

    public void readData() {
        int var1 = this.getLength().intValue();
        if (logger.isDetailEnabled()) {
            logger.detail("reading: " + var1 + " bytes.");
        }

        this.data = new byte[var1];
        this.readData(this.data, var1);
    }

    public Long getLength() {
        long[] var1 = new long[1];
        int var2 = this.ndbBlob.getLength(var1);
        handleError(var2, this.ndbBlob);
        return var1[0];
    }

suggested, verified fix:

    public void loadBlobValues() {
        if (getStatus() == 0) {
            for (NdbRecordBlobImpl ndbRecordBlobImpl: activeBlobs) {
                ndbRecordBlobImpl.readData();
            }
        }
    }

Dear Bogdan, if you need more info as to how to reproduce the issue or details about the fix - please you're welcome to ask. 

Regards,
Andrew

Hi Andrew,

Verified as described. Thanks for the submission

kind regards
Bogdan

Thank you Bogdan! Did you verify the fix as well ?

Regards,
Andrew

Hi Andrew, 

no, I leave this for the dev team :) since you did not sign the OCA so I don't think they can use the code as is. 

all best
Bogdan

If you are interested, in order to submit contributions you must first sign the Oracle Contribution Agreement (OCA).
For additional information please check http://www.oracle.com/technetwork/community/oca-486395.html.
If you have any questions, please contact the MySQL community team (https://dev.mysql.com/community/).

thanks
Bogdan

Thanks Bogdan, that's fine. As long as they are aware of the solution, it's okay. How long do you think it will take to create a patch or release a fixed version of clusterJ ?

Regards,
Andrew

Hi Andrew, 

I cannot make prediction when the dev team will work on this, should be soon since the issue is verified and solution is fairly simple but I can't give you a date.

all best
Bogdan

Dear Bogdan, 

Is there any news ? Can you maybe ask someone in design team if they're gonna be looking at the problem any time soon ? And if so whether they understand and agree with the proposed fix ? I can help to explain and furnish more details to the solution if needed.

We're still interested in solution.

Thanks a lot in advance.

Regards,
Andrew

Posted by developer:
 
The bug is in the NdbApi.

Create the following table and run the attached ndbapi_blob.cpp test.

Fetching an existing row is not an issue. 
But fetching a row that doesn't exist turn out to be a problem. (This is also obvious from the test case attached in 'How To Repeat').

I was wrong. This is indeed a ClusterJ Bug. It should not go reading the blob values when the key fetch returned 0 rows. The c++ ndbapi application uploaded here has a simialr issue. I'll fix up the ClusterJ code.

Posted by developer:
 
Added the following entry to the NDB 8.0.13, 7.6.8, 7.5.12, and 7.4.22 changelogs:

"When a table containing a BLOB or a TEXT field was being queried with ClusterJ for a record that did not exist, an exception (The method is not valid in current blob state) was thrown."