Transaction exception in concurrent environment

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Transaction exception in concurrent environment

Maik Musall
Hi,

Hugi and I converted a large concurrent application from EOF to Cayenne over the last few months, and we tried taking it into production today for the first time. Mostly went fine, but we hit one problem:

Exception in thread "Thread-57" org.apache.cayenne.CayenneRuntimeException: [v.4.0.B2-SNAPSHOT Sep 11 2017 17:37:06] Commit Exception
        at org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
        at org.apache.cayenne.access.DataContext.commitChangesToParent(DataContext.java:682)
        at com.selbstdenker.canusa.powerd.PDObjectContext.saveChanges(PDObjectContext.java:171)
        at com.selbstdenker.canusa.powerd.eo.PDCSimpleWebserviceRequestQueue.sendRequest(PDCSimpleWebserviceRequestQueue.java:349)
        at com.selbstdenker.canusa.powerd.eo.PDCSimpleWebserviceRequestQueue$QueueRunner.run(PDCSimpleWebserviceRequestQueue.java:136)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Transaction must have 'STATUS_ACTIVE' to add a connection. Current status: STATUS_COMMITTED
        at org.apache.cayenne.tx.BaseTransaction.connectionAdded(BaseTransaction.java:246)
        at org.apache.cayenne.tx.CayenneTransaction.connectionAdded(CayenneTransaction.java:49)
        at org.apache.cayenne.tx.BaseTransaction.addConnection(BaseTransaction.java:231)
        at org.apache.cayenne.tx.BaseTransaction.getOrCreateConnection(BaseTransaction.java:203)
        at org.apache.cayenne.access.DataNode$TransactionDataSource.getConnection(DataNode.java:446)
        at org.apache.cayenne.dba.oracle.OraclePkGenerator.longPkFromDatabase(OraclePkGenerator.java:166)
        at org.apache.cayenne.dba.JdbcPkGenerator.generatePk(JdbcPkGenerator.java:215)
        at org.apache.cayenne.access.DataDomainInsertBucket.createPermIds(DataDomainInsertBucket.java:162)
        at org.apache.cayenne.access.DataDomainInsertBucket.appendQueriesInternal(DataDomainInsertBucket.java:73)
        at org.apache.cayenne.access.DataDomainSyncBucket.appendQueries(DataDomainSyncBucket.java:78)
        at org.apache.cayenne.access.DataDomainFlushAction.preprocess(DataDomainFlushAction.java:185)
        at org.apache.cayenne.access.DataDomainFlushAction.flush(DataDomainFlushAction.java:143)
        at org.apache.cayenne.access.DataDomain.onSyncFlush(DataDomain.java:633)
        at org.apache.cayenne.access.DataDomain.onSyncNoFilters(DataDomain.java:603)
        at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
        at org.apache.cayenne.commitlog.CommitLogFilter.onSync(CommitLogFilter.java:85)
        at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
        at org.apache.cayenne.tx.TransactionFilter$1.perform(TransactionFilter.java:73)
        at org.apache.cayenne.tx.TransactionFilter$1.perform(TransactionFilter.java:70)
        at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:87)
        at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:51)
        at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:40)
        at org.apache.cayenne.tx.TransactionFilter.onSync(TransactionFilter.java:70)
        at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
        at org.apache.cayenne.access.DataDomain.onSync(DataDomain.java:590)
        at org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:742)

There is only one application instance with one Cayenne stack. I do have other cases where the exact same code path did not produce problems, both before and after this exception happened. There's a lot of concurrency in this application, so it might only happen when this leads to a new connection being opened?

Any help is greatly appreciated.

Thanks
Maik

Reply | Threaded
Open this post in threaded view
|

Re: Transaction exception in concurrent environment

Andrus Adamchik
Transactions are thread-bound (via InheritableThreadLocal), so there generally should not be any issues with concurrency. Still I can think of two reasons:

* There's a manual transaction management code that does something unusual.
* A parent thread starts a transaction, and then spawns a bunch of child threads that cause a concurrency issue (we should really rethink the "Inheritable" part of "InheritableThreadLocal" for tx.).

Andrus

> On Sep 21, 2017, at 3:53 PM, Musall, Maik <[hidden email]> wrote:
>
> Hi,
>
> Hugi and I converted a large concurrent application from EOF to Cayenne over the last few months, and we tried taking it into production today for the first time. Mostly went fine, but we hit one problem:
>
> Exception in thread "Thread-57" org.apache.cayenne.CayenneRuntimeException: [v.4.0.B2-SNAPSHOT Sep 11 2017 17:37:06] Commit Exception
>        at org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
>        at org.apache.cayenne.access.DataContext.commitChangesToParent(DataContext.java:682)
>        at com.selbstdenker.canusa.powerd.PDObjectContext.saveChanges(PDObjectContext.java:171)
>        at com.selbstdenker.canusa.powerd.eo.PDCSimpleWebserviceRequestQueue.sendRequest(PDCSimpleWebserviceRequestQueue.java:349)
>        at com.selbstdenker.canusa.powerd.eo.PDCSimpleWebserviceRequestQueue$QueueRunner.run(PDCSimpleWebserviceRequestQueue.java:136)
>        at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalStateException: Transaction must have 'STATUS_ACTIVE' to add a connection. Current status: STATUS_COMMITTED
>        at org.apache.cayenne.tx.BaseTransaction.connectionAdded(BaseTransaction.java:246)
>        at org.apache.cayenne.tx.CayenneTransaction.connectionAdded(CayenneTransaction.java:49)
>        at org.apache.cayenne.tx.BaseTransaction.addConnection(BaseTransaction.java:231)
>        at org.apache.cayenne.tx.BaseTransaction.getOrCreateConnection(BaseTransaction.java:203)
>        at org.apache.cayenne.access.DataNode$TransactionDataSource.getConnection(DataNode.java:446)
>        at org.apache.cayenne.dba.oracle.OraclePkGenerator.longPkFromDatabase(OraclePkGenerator.java:166)
>        at org.apache.cayenne.dba.JdbcPkGenerator.generatePk(JdbcPkGenerator.java:215)
>        at org.apache.cayenne.access.DataDomainInsertBucket.createPermIds(DataDomainInsertBucket.java:162)
>        at org.apache.cayenne.access.DataDomainInsertBucket.appendQueriesInternal(DataDomainInsertBucket.java:73)
>        at org.apache.cayenne.access.DataDomainSyncBucket.appendQueries(DataDomainSyncBucket.java:78)
>        at org.apache.cayenne.access.DataDomainFlushAction.preprocess(DataDomainFlushAction.java:185)
>        at org.apache.cayenne.access.DataDomainFlushAction.flush(DataDomainFlushAction.java:143)
>        at org.apache.cayenne.access.DataDomain.onSyncFlush(DataDomain.java:633)
>        at org.apache.cayenne.access.DataDomain.onSyncNoFilters(DataDomain.java:603)
>        at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
>        at org.apache.cayenne.commitlog.CommitLogFilter.onSync(CommitLogFilter.java:85)
>        at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
>        at org.apache.cayenne.tx.TransactionFilter$1.perform(TransactionFilter.java:73)
>        at org.apache.cayenne.tx.TransactionFilter$1.perform(TransactionFilter.java:70)
>        at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:87)
>        at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:51)
>        at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:40)
>        at org.apache.cayenne.tx.TransactionFilter.onSync(TransactionFilter.java:70)
>        at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
>        at org.apache.cayenne.access.DataDomain.onSync(DataDomain.java:590)
>        at org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:742)
>
> There is only one application instance with one Cayenne stack. I do have other cases where the exact same code path did not produce problems, both before and after this exception happened. There's a lot of concurrency in this application, so it might only happen when this leads to a new connection being opened?
>
> Any help is greatly appreciated.
>
> Thanks
> Maik
>