Description:
This bug report/feature request is really about the whole X protocol documentation but I will focus on the initial setup and authentication.
The MySQL documentation for the X protocol feels rather incomplete and is tied into the MySQL server code. That's not ideal, as if the code changes so too may the specification. The X protocol specification should be a network specification independent of the code that implements it.
Current documentation is incomplete.
How to repeat:
I was looking at the "capabilities" part, used to initialise a session and the documentation seems rather poor. You can optionally send GetCapabalities and get a Capabilities message back but the description on the Capabilities message https://dev.mysql.com/doc/dev/mysql-server/latest/mysqlx__connection_8proto.html is absolutely vague as it seems to be generated from the proto files so adds no "context" to actual usage, or expected or potential capabilities that a client may be offered.
https://dev.mysql.com/doc/dev/mysql-server/latest/mysqlx_protocol_lifecycle.html provides an example of the lifecycle and mentions the capabilities messages but there's no specific specification of:
* which capabilities (as a minimum) a driver writer should expect to be told about, and if this varies from MySQL version. e.g. MYSQL41 vs SHA256_MEMORY? or which method should be used if both are offered by the server and supported by the client.
There is a reference to this in the lifecycle doc, https://dev.mysql.com/doc/dev/mysql-server/latest/mysqlx_protocol_lifecycle.html, but that's not the same as making a specification.
With regard to "capabilities":
* which "capabilities" should be offered? What are their specific values, and given it seems capabilities are sent as key/values what types/values are expected for different keys?
* I should NOT have to read the code to see this, as code might change and that would implicitly change the specification.
* Now you have MySQL 5.7 and 8.0, are there differences in the capabilities offered by these versions in their default configurations? It would be good to be more explicit about this.
* If I were to want to extend the capabilities of a server, how would I do that? If I were want to register a new capability in the protocol what would the process be to handle that? While it's clear that Oracle has at the moment been the only entity to build the protocol if you want others to use it, and I think the protocol does have potential benefits over the current protocol, even if currently there is little usage, this sort of thing should be documented. If you want to allow "free" capabilities, maybe suggesting the implementer names them something like "X_...." but may trigger conflicts with different implementations.
* What should a client do with a capability that it does not understand? Ignore it I would expect? Be explicit.
* What should a server do with a capability that it does not understand requested by the client?
* The capability "flow" shown in the lifecycle document cover the optimistic flow yet do not cover the failure scenarios very clearly. How should errors be reported and handled and what behaviour is expected of the client or server after such an error?
There is also talk about pipelining
* again related to pipelining there are no examples showing or describing the behaviour expected should an error occur. This would be helpful.
A bug/incomplete description in the flows here https://dev.mysql.com/doc/dev/mysql-server/latest/mysqlx_protocol_lifecycle.html (Stages of Session Setup) shows a { "tls": 0 } response but I assume that should be a Capabilities{ "tls": 0, ....other capabilities....} message. Please clarify, and provide a full example message.
It would be good to make a full list of messages, perhaps, in a similar way to https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html where global variables are defined, as this provides a way to describe each message and any specific values different fields should have, or if/when there are changes to the specification, such as "MYSQL41" being one of the authentication methods but I would imagine "SHA256_MEMORY" was added later for 8.0, and then backported.
Suggested fix:
The suggested fixes are suggested in the description.
Improving the documentation will help people implement the protocol in languages other than those supported by Oracle (e.g. the client side and MySQL Server).
I considered looking at this again but have been disappointed that the documentation alone is not a good source of information for some of the reasons outlined above and I expect I would need to read the client/server source code to do this completely, something which should not be necessary.
This may also explain why usage of the X protocol is still somewhat limited even if the protocol itself should be more efficient and faster due to the pipelining the protocol provides. Some benchmarking to show these advantages might also help convince people using it is good.
