Bug #51622 Extend NDBAPI to allow parallel asynchronous NdbScanOperation
Submitted: 1 Mar 2010 21:17
Reporter: Matthew Montgomery Email Updates:
Status: Open Impact on me:
Category:MySQL Cluster: NDB API Severity:S4 (Feature request)
Version:mysql-5.1-telco-6.2 OS:Any
Assigned to: Assigned Account CPU Architecture:Any

[1 Mar 2010 21:17] Matthew Montgomery
Per Frazer:

In a multi-threaded API node where threads 1 and 2 are not interacting with Ndb directly but they are using thread 3 as a 'DB worker' thread.

Thread 3 is defining PK/UK operations, sending them asynchronously and polling for results. Thread 3 can be extended to also define scan operations and send them asynchronously. Once a scan operation has been executed, the results are iterated with calls to nextResult().
The nextResult() API will return rows in the currently 'cached' batches to the caller. When there are no more rows available in the current 'cached' batches, nextResult() needs to ask the data nodes for more batches, which it will do if the fetchAllowed parameter is true. This is a synchronous operation. It will get the next batch(es) of rows for the scan operation it is called upon.

It will not :
- Request any more batches for other scan operations
- Execute any defined PK / UK operations
- Return control to the caller until the batch(es) are available.

How to repeat:
Having Thread 3 iterate results will cause Thread3 to be unresponsive while it is making a blocking call to nextResult() for one of it's scans.

Suggested fix:
We would need to define some asynchronous Api extensions for use with scans so that :
- A request for more batch(es) can be sent asynhronously
- pollNdb() can 'wakeup' when the requested batch(es) are available.

This would allow :
- Thread 3 to request and wait for other work while scan results are being iterated
- More than 1 scan to be advanced in parallel by a single thread.
[7 Sep 2010 12:13] Jørgen Austvik
Bernd: Please set priority