Bug #88091 getTables in mysql connector 6.x+ differs from mysql connector 5.1.41
Submitted: 13 Oct 2017 22:02 Modified: 16 Oct 2017 16:32
Reporter: Pala Dox Email Updates:
Status: Duplicate Impact on me:
None 
Category:Connector / J Severity:S1 (Critical)
Version:6.0.6 and 8.0.8 OS:Any
Assigned to: CPU Architecture:Any

[13 Oct 2017 22:02] Pala Dox
Description:
Hi, since trying out mysql connector java 6.x and 8.x we found it did not work correctly when applying schema. It was like it forgot it applied the schema.

Ie i got this error

Exception in thread "main" com.google.gwtorm.server.OrmException: insert failure on schema_version
	at com.google.gwtorm.schema.sql.SqlDialect.convertError(SqlDialect.java:159)
	at com.google.gwtorm.schema.sql.DialectMySQL.convertError(DialectMySQL.java:230)
	at com.google.gwtorm.jdbc.JdbcAccess.convertError(JdbcAccess.java:484)
	at com.google.gwtorm.jdbc.JdbcAccess.insert(JdbcAccess.java:169)
	at com.google.gerrit.server.schema.SchemaCreator.create(SchemaCreator.java:89)
	at com.google.gerrit.server.schema.SchemaUpdater.update(SchemaUpdater.java:111)
	at com.google.gerrit.pgm.init.BaseInit$SiteRun.upgradeSchema(BaseInit.java:388)
	at com.google.gerrit.pgm.init.BaseInit.run(BaseInit.java:145)
	at com.google.gerrit.pgm.util.AbstractProgram.main(AbstractProgram.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.google.gerrit.launcher.GerritLauncher.invokeProgram(GerritLauncher.java:203)
	at com.google.gerrit.launcher.GerritLauncher.mainImpl(GerritLauncher.java:108)
	at com.google.gerrit.launcher.GerritLauncher.main(GerritLauncher.java:63)
	at Main.main(Main.java:24)
Caused by: java.sql.BatchUpdateException: Table 'reviewdb4.schema_version' doesn't exist
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.cj.core.util.Util.handleNewInstance(Util.java:185)
	at com.mysql.cj.core.util.Util.getInstance(Util.java:168)
	at com.mysql.cj.core.util.Util.getInstance(Util.java:175)
	at com.mysql.cj.jdbc.exceptions.SQLError.createBatchUpdateException(SQLError.java:212)
	at com.mysql.cj.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1545)
	at com.mysql.cj.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1210)
	at com.mysql.cj.jdbc.StatementImpl.executeBatch(StatementImpl.java:1020)
	at com.google.gwtorm.schema.sql.SqlDialect.executeBatch(SqlDialect.java:435)
	at com.google.gwtorm.jdbc.JdbcAccess.execute(JdbcAccess.java:455)
	at com.google.gwtorm.jdbc.JdbcAccess.insertAsBatch(JdbcAccess.java:213)
	at com.google.gwtorm.jdbc.JdbcAccess.insert(JdbcAccess.java:164)
	... 13 more
Caused by: java.sql.SQLSyntaxErrorException: Table 'reviewdb4.schema_version' doesn't exist
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:112)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:116)
	at com.mysql.cj.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1867)
	at com.mysql.cj.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2029)
	at com.mysql.cj.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:1970)
	at com.mysql.cj.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:4996)
	at com.mysql.cj.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1506)
	... 19 more

I belive this is caused by https://github.com/mysql/mysql-connector-j/commit/3d3d0c157491d98b077429e0a5e23331dc69b0b0... though not 100% sure as i carn't revert it do to lots of merge conflicts.

On gerrit's project we use gwtorm. see https://github.com/GerritCodeReview/gwtorm/blob/master/src/main/java/com/google/gwtorm/sch...

but it seems it was changed from int to long mysql connector side.

Mysql connector 5.1.x+ worked. But seemed to break in 6.x.

Looking at mariadb one, it is using int instead of long for executeBatch.

How to repeat:
download

https://www.gerritcodereview.com/download/gerrit-2.15-rc1.war

mkdir <site> then mkdir <site>/lib/

then download the mysql connector 5.1 from https://mvnrepository.com/artifact/mysql/mysql-connector-java/5.1.44 and put it in the lib folder

then mkdir <site>/etc/

then create gerrit.config in that folder

and put this content in it

[database]
    driver = com.mysql.jdbc.Driver
	hostname = <db_host>
	database = <db_name>
	username = <db_user>
	url = jdbc:mysql://localhost/<db_name>?nullNamePatternMatchesAll=true
         #?nullNamePatternMatchesAll=true&useSSL=false
	type = jdbc

then run

 java -jar <gerrit war file> init -d <site>

ie something like

 java -jar gerrit.war init -d review_site3

follow steps, but write n for no when it says download mysql connector as we have already manually done that in the lib folder.

then follow onscreen instructions.

then once it is done you will find it executed fine.

Now repeat this again by clearing the lib folder of mysql connector and delete db and recreate it.

Download https://mvnrepository.com/artifact/mysql/mysql-connector-java/8.0.8-dmr into the lib folder and redo the process.
[13 Oct 2017 22:03] Pala Dox
See our https://bugs.chromium.org/p/gerrit/issues/detail?id=7331 bug.
[14 Oct 2017 18:34] Pala Dox
I think i found the problem, but not sure what the fix is.

When i added a print system call to see what it is finding

it is showing all the tables in mysql connector 6.x+ whereas in 5.x it is finding only the ones from the db.

nullNamePatternMatchesAll was  changed to false in 6.x by default.
[14 Oct 2017 20:40] Pala Dox
getTables
[14 Oct 2017 22:02] Pala Dox
changing version column.
[16 Oct 2017 12:49] Chiranjeevi Battula
Hello Pala Dox,

Thank you for the bug report.
This is most likely duplicate of Bug #87826, please see Bug #87826

Thanks,
Chiranjeevi.
[16 Oct 2017 12:54] Pala Dox
Oh, i have set that option to true. Was there more then a default value change done to that config?
[16 Oct 2017 12:57] Pala Dox
I found the getTables in mysql connector 5.1.x was qucker and found the correct tables compared to 6.x+ which was very slow and found alot of tables.
[16 Oct 2017 14:09] Pala Dox
See https://phabricator.wikimedia.org/P6123 for comparison.
[16 Oct 2017 15:49] Pala Dox
@Chiranjeevi Battula hi, i doint think this is a duplicate of that because, setting that config to true still causes the problem.
[16 Oct 2017 16:32] Pala Dox
It may be getColumns.

As looking at https://github.com/mysql/mysql-connector-j/commit/8d34e91534b3fe68aef9eca7ddff9602ef4c5e0d

DatabaseMetaData getColumns is changed.