• Bug
  • Status: Open
  • 2 Major
  • Resolution:
  • DSO:L1
  • prodmgmt
  • Reporter: eellis00
  • November 07, 2008
  • 0
  • Watchers: 0
  • March 19, 2010

Attachments

Description

com.tc.object.tx.UnlockedSharedObjectException: ******************************************************************************* Attempt to access a shared object outside the scope of a shared lock. All access to shared objects must be within the scope of one or more shared locks defined in your Terracotta configuration. Please alter the locks section of your Terracotta configuration so that this access is auto-locked or protected by a named lock.

For more information on this issue, please visit our Troubleshooting Guide at: http://terracotta.org/kit/troubleshooting

Caused by Thread: http-8085-Processor69  in  VM(19)
Shared Object Type: java.util.SimpleTimeZone \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

    at com.tc.object.tx.ClientTransactionManagerImpl.getTransaction(ClientTransactionManagerImpl.java:303)
    at com.tc.object.tx.ClientTransactionManagerImpl.fieldChanged(ClientTransactionManagerImpl.java:598)
    at com.tc.object.TCObjectImpl.objectFieldChanged(TCObjectImpl.java:305)
    at com.tc.object.TCObjectImpl.longFieldChanged(TCObjectImpl.java:349)
    at java.util.SimpleTimeZone.__tc_setcacheYear(SimpleTimeZone.java)
    at java.util.SimpleTimeZone.getOffset(SimpleTimeZone.java:675)
    at java.util.SimpleTimeZone.getOffsets(SimpleTimeZone.java:550)
    at java.util.SimpleTimeZone.getOffset(SimpleTimeZone.java:522)
    at java.util.SimpleTimeZone.inDaylightTime(SimpleTimeZone.java:834)
    at sun.util.calendar.ZoneInfo.getOffsets(ZoneInfo.java:282)
    at sun.util.calendar.ZoneInfo.getOffsetsByWall(ZoneInfo.java:233)
    at java.util.GregorianCalendar.computeTime(GregorianCalendar.java:2450)
    at java.util.Calendar.updateTime(Calendar.java:2463)
    at java.util.Calendar.getTimeInMillis(Calendar.java:1082)
    at java.util.Calendar.getTime(Calendar.java:1055)
    at net.sourceforge.jtds.jdbc.DateTime.toTimestamp(DateTime.java:355)
    at net.sourceforge.jtds.jdbc.Support.convert(Support.java:379)
    at net.sourceforge.jtds.jdbc.JtdsResultSet.getTimestamp(JtdsResultSet.java:1160)
    at net.sourceforge.jtds.jdbc.JtdsResultSet.getTimestamp(JtdsResultSet.java:1312)
    at org.apache.tomcat.dbcp.dbcp.DelegatingResultSet.getTimestamp(DelegatingResultSet.java:261)
    at com.ibatis.sqlmap.engine.type.DateTypeHandler.getResult(DateTypeHandler.java:44)
    at com.ibatis.sqlmap.engine.mapping.result.BasicResultMap.getPrimitiveResultMappingValue(BasicResultMap.java:565)
    at com.ibatis.sqlmap.engine.mapping.result.BasicResultMap.getResults(BasicResultMap.java:319)

java.util.Locale java.text.SimpleDateFormat java.text.DateFormat java.text.Format java.text.DateFormatSymbols java.text.DecimalFormat java.text.NumberFormat java.text.DecimalFormatSymbols java.text.DigitList java.util.TimeZone sun.util.calendar.Gregorian sun.util.calendar.BaseCalendar sun.util.calendar.AbstractCalendar sun.util.calendar.CalendarSystem sun.util.calendar.Gregorian$Date sun.util.calendar.BaseCalendar$Date sun.util.calendar.CalendarDate sun.util.calendar.ZoneInfo java.lang.String$CaseInsensitiveComparator org.xml.sax.helpers.DefaultHandler java.util.SimpleTimeZone

I’ll try this out and see if it works

			<!-- SimpleTimeZone  -->
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.get\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.set\*(..)</method-expression>
				<lock-level>write</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.use\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.in\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.clone(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.has\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.equals(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.toString(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>

Comments

Eric Ellis 2008-11-26

Here is the modified config that appears to work in production w/o exceptions:

			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.get\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.set\*(..)</method-expression>
				<lock-level>write</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.use\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.in\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.clone(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.has\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.equals(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.toString(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>

Eric Ellis 2008-11-26

Opps, version control tagging issue. Here’s the update:

			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.getOffset(..)</method-expression>
				<lock-level>write</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.getDSTSavings()</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.getRawOffset()</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.set\*(..)</method-expression>
				<lock-level>write</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.use\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.in\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.clone(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.has\*(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.equals(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>
			<autolock auto-synchronized="true">
				<method-expression>\* java.util.SimpleTimeZone.toString(..)</method-expression>
				<lock-level>read</lock-level>
			</autolock>

Eric Ellis 2008-11-26

a minor fix:

\* java.util.SimpleTimeZone.getOffset\*(..) write

Eric Ellis 2008-12-01

These locks are causing a deadlock across the entire Terracotta cluster… I’ll just have to live with the concurrent mod exceptions for now.

at sun.util.calendar.ZoneInfo.__tc_wrapped_getOffsets(ZoneInfo.java:225) - waiting to lock [0x00002aaaf2891b38] (a sun.util.calendar.ZoneInfo) at sun.util.calendar.ZoneInfo.getOffsets(ZoneInfo.java) at java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2024) at java.util.GregorianCalendar.computeFields(GregorianCalendar.java:1996) at java.util.Calendar.setTimeInMillis(Calendar.java:1104) at java.util.Calendar.setTime(Calendar.java:1070) at org.apache.log4j.helpers.ISO8601DateFormat.format(ISO8601DateFormat.java:65) at java.text.DateFormat.format(DateFormat.java:316) at org.apache.log4j.helpers.PatternParser$DatePatternConverter.convert(PatternParser.java:436) at org.apache.log4j.helpers.PatternConverter.format(PatternConverter.java:56) at org.apache.log4j.PatternLayout.format(PatternLayout.java:495) at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:292) at org.apache.log4j.RollingFileAppender.subAppend(RollingFileAppender.java:225) at org.apache.log4j.WriterAppender.append(WriterAppender.java:150) at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:221) - locked [0x00002aaaf2896e68] (a org.apache.log4j.RollingFileAppender) at com.tc.logging.DelegatingAppender.doAppend(DelegatingAppender.java:47) at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:57) at org.apache.log4j.Category.callAppenders(Category.java:194) - locked [0x00002aaaf2878a20] (a org.apache.log4j.Logger) at org.apache.log4j.Category.forcedLog(Category.java:379) at org.apache.log4j.Category.info(Category.java:654) at com.tc.logging.TCLoggerImpl.info(TCLoggerImpl.java:71) at com.tc.logging.BaseMessageDecoratorTCLogger.info(BaseMessageDecoratorTCLogger.java:46) at com.tc.object.tx.RemoteTransactionManagerImpl.commit(RemoteTransactionManagerImpl.java:234) at com.tc.object.tx.ClientTransactionManagerImpl.commitInternal(ClientTransactionManagerImpl.java:456) at com.tc.object.tx.ClientTransactionManagerImpl.commit(ClientTransactionManagerImpl.java:421) at com.tc.object.tx.ClientTransactionManagerImpl.commit(ClientTransactionManagerImpl.java:360) at com.tc.object.bytecode.ManagerImpl.monitorExit(ManagerImpl.java:520) at com.tc.object.bytecode.ManagerUtil.monitorExit(ManagerUtil.java:478) at java.util.concurrent.locks.ReentrantReadWriteLock$DsoLock.unlock(ReentrantReadWriteLock/java:67) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock/java) at java.util.concurrent.ConcurrentHashMap$Segment.unlock(ConcurrentHashMap.java) at java.util.concurrent.ConcurrentHashMap$Segment.put(ConcurrentHashMap.java:443) at java.util.concurrent.ConcurrentHashMap.put(Unknown Source) at com.zdm.jforum.cache.DataStoreCacheEngine.add(DataStoreCacheEngine.java:106) at net.jforum.repository.RolesRepository.addGroupRoles(RolesRepository.java:105) at net.jforum.dao.generic.security.GenericGroupSecurityDAO.loadRolesByUserGroups(GenericGroupSecurityDAO.java:301) at net.jforum.repository.SecurityRepository.load(SecurityRepository.java:152) at net.jforum.repository.SecurityRepository.load(SecurityRepository.java:95) at net.jforum.repository.SecurityRepository.load(SecurityRepository.java:113) at net.jforum.repository.SecurityRepository.get(SecurityRepository.java:233) at net.jforum.repository.SecurityRepository.canAccess(SecurityRepository.java:207) at net.jforum.repository.SecurityRepository.canAccess(SecurityRepository.java:180) at net.jforum.entities.UserSession.__tc_wrapped_isModerator(UserSession.java:342) at net.jforum.entities.UserSession.isModerator(UserSession.java) at net.jforum.exceptions.ExceptionWriter.handleExceptionData(ExceptionWriter.java:98) at net.jforum.JForum.handleException(JForum.java:303) at net.jforum.JForum.service(JForum.java:222) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at net.jforum.util.legacy.clickstream.ClickstreamFilter.doFilter(ClickstreamFilter.java:59) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at com.tc.tomcat55.session.SessionValve55.tcInvoke(SessionValve55.java:63) at com.tc.tomcat55.session.SessionValve55.invoke(SessionValve55.java:50) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:619)

Sreenivasan Iyer 2008-12-08

Basically Hibernate auto-generates entities with Calendar fields to store date-time information. Therefore, one is forced to add these to the boot-jar and Terracotta’s treatment of SimpleTimeZone and/or java.util.Calendar appears to be causing problems - (The problem appears to be manifesting as Deadlocks in Terracotta use of Log4J Logging, although the root cause may point to how locking is implemented when SimpleTimeZone and/or Calendar get clustered…)…