Bug #62577 XA connection fails with ClassCastException
Submitted: 29 Sep 2011 22:46 Modified: 8 Jul 2014 22:11
Reporter: dan f Email Updates:
Status: Closed Impact on me:
None 
Category:Connector / J Severity:S1 (Critical)
Version:5.1.17 OS:Linux (CentOS release 5.7 (Final) )
Assigned to: Alexander Soklakov CPU Architecture:Any
Tags: ClassCastException, connect, xa

[29 Sep 2011 22:46] dan f
Description:
on startup jboss fails with ClassCastException:

Caused by: java.lang.ClassCastException: $Proxy388 cannot be cast to com.mysql.jdbc.ConnectionImpl
	at com.mysql.jdbc.jdbc2.optional.MysqlXADataSource.wrapConnection(MysqlXADataSource.java:74)
	at com.mysql.jdbc.jdbc2.optional.MysqlXADataSource.getXAConnection(MysqlXADataSource.java:61)
	at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.getXAManagedConnection(XAManagedConnectionFactory.java:449)
	... 77 more

How to repeat:
this works on another machine with the exact same configuration, but which doesn't connect to multiple databases, working machine just connects to a single 5.1.58 mysql. this instance connects to two databases, one mysql 5.5.11 and one 5.1.58. also, this works fine if i roll back to connector/j 5.1.12. it's a hard fail on 5.1.16 and 5.1.17.

Suggested fix:
no workaround found so far.
[29 Sep 2011 22:46] dan f
full java stacktrace

Attachment: myj-5.1.17.txt (text/plain), 8.36 KiB.

[3 Oct 2011 6:55] Valeriy Kravchuk
Looks like a duplicate of Bug #41321. Please, check.

What "other version of driver" do you mean?
[4 Oct 2011 13:22] dan f
i don't believe this is the same problem, i'm not using loadbalance'd jdbc url. my url is: jdbc:mysql://10.1.100.30,10.1.100.32/station?autoReconnect=true and i'm using the xa driver com.mysql.jdbc.jdbc2.optional.MysqlXADataSource. the other driver version was the version mentioned in the previous message, 5.1.12 works fine. 5.1.16-17 both fail. also, it works if the multiple connections go to the same machine. oh, i think i just figured out the problem. the XA transaction probably only works across a single machine? would an XA transaction work across two connections to different databases on different machines, probably not? but why does it work then in 5.1.12 and not in 5.1.17? just thinking out loud now.
[5 Oct 2011 7:43] Tonci Grgin
Hi Dan.

I'm most interested in full stack trace showing more of what's happening on c/J side (only a line or two in your trace). I believe , if there is a bug here, it's in JDBC4ConnectionWrapper:isWrapperFor but I can't confirm it without more info.

I trust you followed examples in testsuite.simple.DataSourceTest.java and XATest.java as well as read 
http://dev.mysql.com/doc/refman/5.5/en/connector-j-reference-configuration-properties.html and 
http://dev.mysql.com/doc/refman/5.5/en/xa.html#.
[5 Oct 2011 7:52] Tonci Grgin
Dan, it would be awesome if you could pinpoint exact version of c/J where this starts to fail.
[11 Oct 2011 22:32] dan f
this appears to be a bug in the URL processing, this URL works:

jdbc:mysql://192.168.1.20/wall2?autoReconnect=true

but this one doesn't in 5.1.17

jdbc:mysql://192.168.1.20,192.168.1.3/wall2?autoReconnect=true

they both work in 5.1.12
[11 Oct 2011 22:33] dan f
java stack trace

Attachment: mysql_xa_classcast.txt (text/plain), 17.21 KiB.

[12 Oct 2011 0:44] Mark Matthews
You are using a load-balanced connection under the hood. Starting in 5.1.13, URLs of the form jdbc:mysql://host1,host2 are the load-balanced connection being used, configured in a failover scenario.

In some cases, it seems the connection proxied for this is not implementing the right interfaces to be used as an XAConnection.
[8 Nov 2011 4:41] dan f
so bug #41321 says the only way to use the driver with XA is to use the non-loadbalance URL and you are saying as of 5.1.13 the standard URL is using the loadbalance URL under the covers--so now there is no way to use XA with Connector/J at all?
[18 Sep 2012 9:36] Bogdan Degtyariov
Same problem happens with Connector/J 5.1.21.
I have verified it with JBoss 5.0.1.

Just as Mark said, even if loadbalance prefix is omitted in the connect URL (jdbc:mysql://) the underlying connection is created as LoadBalancedMySQLConnection instance, which cannot be casted to ConnectionImpl.
[18 Sep 2012 18:27] Todd Farmer
To clarify, load-balancing is used under the hood when multiple hosts are defined.  That's why the it's possible to create a connection without this error when only a single host is defined.  So, this bug means that XA and multi-host (failover, replication, load-balanced) configurations are incompatible at present.
[26 Dec 2012 11:35] scarlett xu
So will this bug be fixed in the 5.1.2* versions now?
or if we need to use the multiple host, we have to use the version back to 5.1.12 version?
[17 Dec 2013 10:01] Manjunatha Nakshathri
Hello All, 

Is this fixed? What version of connector i should use for connecting to the MySQL Cluster. Even with the connector version 12, we are facing issues like : 

INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) JBAS010404: Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver
(version 5.1)

Can you please help us.
[17 Dec 2013 10:08] Alexander Soklakov
Hi Manjunatha,

Please clarify your question. Do you have troubles only with XA in load-balanced configuration? Or you can't connect to Cluster at all?

Please provide Jboss, Cluster, Connector/J versions used and show the stacktrace you get.
[25 Jun 2014 6:38] Pankajan Chanthirasegaran
Because of this bug, we cannot do load balancing/fail-over for XA Connections. I have tested with version 5.1.12 which is the last version this worked. The bug exists even in the latest 5.1.31. I did not find any workaround for this which forced me to install HAProxy to do the fail-over and load balancing. 

It is surprising that this problem is there for over 2 years. Will it be fixed or should be have to rely on third party software to load balance the mysql servers?
[25 Jun 2014 10:42] Alexander Soklakov
Hi Pankajan,

Thanks for reminding about this bug. We could fix ClassCastException.
As for possibility to use our load-balancing/failing-over you should clarify the scenario you want to implement. Our multi-host connections use pools of ConnectionImpl and, with load-balancing, provide the switch between them on usual commit() eg, or, with failover, gives you the life second connection if the first is failed. In anyway, each underlying connection has it's own session thus you have to recreate all session-specific data, like prepared statements or also XA transactions.
[8 Jul 2014 22:11] Daniel So
Added the following entry to the Connector/J 5.1.32 changelog:

"XA connections failed with a class cast exception for a load-balanced configuration with multiple hosts. This was because some XA-related classes used com.mysql.jdbc.ConnectionImpl in method parameters during calls. This fix makes the classes use com.mysql.jdbc.Connection instead in those cases."
[8 Jul 2014 22:12] Daniel So
Posted by developer:
[14 Aug 2014 18:31] David Martín
Still having the same issue using 5.1.32 Connector/J (platform independent):

Caused by: java.lang.ClassCastException: com.sun.proxy.$Proxy58 cannot be cast to com.mysql.jdbc.ConnectionImpl
	at com.mysql.jdbc.jdbc2.optional.MysqlXADataSource.wrapConnection(MysqlXADataSource.java:71)
	at com.mysql.jdbc.jdbc2.optional.MysqlXADataSource.getXAConnection(MysqlXADataSource.java:50)
	at com.atomikos.jdbc.AtomikosXAConnectionFactory.createPooledConnection(AtomikosXAConnectionFactory.java:60)
	at com.atomikos.datasource.pool.ConnectionPool.createPooledConnection(ConnectionPool.java:116)
	at com.atomikos.datasource.pool.ConnectionPool.growPool(ConnectionPool.java:225)
	at com.atomikos.datasource.pool.ConnectionPool.retrieveFirstAvailableConnectionAndGrowPoolIfNecessary(ConnectionPool.java:175)
	at com.atomikos.datasource.pool.ConnectionPool.findOrWaitForAnAvailableConnection(ConnectionPool.java:160)
	at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:151)
	at com.atomikos.jdbc.AbstractDataSourceBean.getConnection(AbstractDataSourceBean.java:342)
	at com.atomikos.jdbc.AbstractDataSourceBean.getConnection(AbstractDataSourceBean.java:394)
	at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:70)
	at org.hibernate.engine.jdbc.internal.JdbcServicesImpl$ConnectionProviderJdbcConnectionAccess.obtainConnection(JdbcServicesImpl.java:242)
	at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:117)
	at org.hibernate.service.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:75)
	at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:159)
	at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:131)
	at org.hibernate.cfg.SettingsFactory.buildSettings(SettingsFactory.java:71)
	at org.hibernate.cfg.Configuration.buildSettingsInternal(Configuration.java:2277)
	at org.hibernate.cfg.Configuration.buildSettings(Configuration.java:2273)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1742)
	at org.hibernate.ejb.EntityManagerFactoryImpl.<init>(EntityManagerFactoryImpl.java:94)
	at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:905)
	at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:890)
	at org.hibernate.ejb.HibernatePersistence.createContainerEntityManagerFactory(HibernatePersistence.java:74)
	at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:268)
	at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:310)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1514)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1452)
	... 109 more

Spring datasource configuration:
	<bean id="XADataSource" 
		  class="com.atomikos.jdbc.AtomikosDataSourceBean" 
		  init-method="init" 
		  destroy-method="close"
		  lazy-init="true">
	    <property name="xaDataSourceClassName" value="com.mysql.jdbc.jdbc2.optional.MysqlXADataSource"/>
	    <property name="uniqueResourceName" value="XADataSource"/>    
	    <property name="minPoolSize" value="3" />
	    <property name="maxPoolSize" value="25" />
	    <property name="maxIdleTime" value="5" />
	    <property name="maxLifetime" value="20" />   
	    <property name="xaProperties">
	        <props>
	            <prop key="url">jdbc:mysql://localhost:3306,qadb01:3306/schema</prop>
	            <prop key="user">${username}</prop>
	            <prop key="password">${password}</prop>
	            <prop key="pinGlobalTxToPhysicalConnection">true</prop>
	            <prop key="autoReconnect">true</prop>
	            <prop key="failOverReadOnly">true</prop>
	        </props>
	    </property>
	</bean>

what does the new version solve?
[15 Aug 2014 7:34] Alexander Soklakov
Hi David,

Thanks for update, we missed one occurrence of ConnectionImpl in case pinGlobalTxToPhysicalConnection set to true. I opened Bug#73594 for this regression.

C/J 5.1.32 should work with default pinGlobalTxToPhysicalConnection=false.
[15 Aug 2014 9:55] David Martín
Thanks Alexander for your quick response.

I have tested using pinGlobalTxToPhysicalConnection=false and the class cast exception disappears, but the JTA implementation we use (Atomikos) needs this flag set to true to establish the XA connection with MySQL (see http://www.atomikos.com/Documentation/KnownProblems#MySQL_does_not_support_TMJOIN)

Looking forward for Bug#73594 fix. 
In relation with this bug, I can not access to see its content and follow its resolution. Is the bug number correct?

Regards
[15 Aug 2014 11:06] Alexander Soklakov
Bug#73594 should be visible now.