• Bug
  • Status: Resolved
  • 2 Major
  • Resolution: Won't Fix
  • interfaces
  • Reporter: tgautier
  • April 11, 2008
  • 0
  • Watchers: 1
  • February 12, 2014
  • February 12, 2014

Attachments

Description

trying to load a map and getting an oome in the client. then I get this:

********************************** ERROR *********************************** * Terracotta client shutting down due to error java.lang.OutOfMemoryError: * Java heap space ****************************************************************************

java.lang.OutOfMemoryError: Java heap space at java.lang.StringCoding$CharsetSE.encode(StringCoding.java:334) at java.lang.StringCoding.encode(StringCoding.java:378) at java.lang.String.getBytes(String.java:812) at com.tc.object.compression.StringCompressionUtil.stringToUncompressedBin(StringCompressionUtil.java:44) at com.tc.object.dna.impl.BaseDNAEncodingImpl.writeCompressedString(BaseDNAEncodingImpl.java:334) at com.tc.object.dna.impl.BaseDNAEncodingImpl.encode(BaseDNAEncodingImpl.java:231) at com.tc.object.dna.impl.DNAWriterImpl.addLogicalAction(DNAWriterImpl.java:103) at com.tc.object.change.event.LogicalChangeEvent.write(LogicalChangeEvent.java:22) at com.tc.object.change.TCChangeBufferImpl.writeEventsToDNA(TCChangeBufferImpl.java:108) at com.tc.object.change.TCChangeBufferImpl.writeTo(TCChangeBufferImpl.java:92) at com.tc.object.tx.TransactionBatchWriter$TransactionBuffer.writeChanges(TransactionBatchWriter.java:491) at com.tc.object.tx.TransactionBatchWriter$TransactionBuffer.writeFirst(TransactionBatchWriter.java:555) at com.tc.object.tx.TransactionBatchWriter$TransactionBuffer.write(TransactionBatchWriter.java:449) at com.tc.object.tx.TransactionBatchWriter.addTransaction(TransactionBatchWriter.java:284) at com.tc.object.tx.TransactionSequencer.addTransactionToBatch(TransactionSequencer.java:80) at com.tc.object.tx.TransactionSequencer.addTxnInternal(TransactionSequencer.java:110) at com.tc.object.tx.TransactionSequencer.addTransaction(TransactionSequencer.java:89) at com.tc.object.tx.RemoteTransactionManagerImpl.commit(RemoteTransactionManagerImpl.java:226) at com.tc.object.tx.ClientTransactionManagerImpl.commitInternal(ClientTransactionManagerImpl.java:451) at com.tc.object.tx.ClientTransactionManagerImpl.commit(ClientTransactionManagerImpl.java:416) at com.tc.object.tx.ClientTransactionManagerImpl.commit(ClientTransactionManagerImpl.java:355) at com.tc.object.bytecode.ManagerImpl.monitorExit(ManagerImpl.java:526) at com.tc.object.bytecode.ManagerUtil.monitorExit(ManagerUtil.java:501) at Main.load(Main.java:46) at Main.run(Main.java:71) at Main.main(Main.java:79)

regular java quits with an oome. dso-java pauses and does not exit the vm

Comments

Taylor Gautier 2008-04-11

to repro:

save all attached files to new directory then:

$ javac *.java $ start-tc-server.sh $ dso-java Main

Tim Eck 2008-04-11

It’s still a bug if so, but I suspect the remote transaction manager is trying to flush uncommited txns in a shutdown hook and hanging. A thread dump would shed some light on the issue

Taylor Gautier 2008-04-11

^\Full thread dump Java HotSpot(TM) Client VM (1.5.0_13-119 mixed mode):

“Statistics Emitter Timer” daemon prio=5 tid=0x01026c90 nid=0x9e6800 in Object.wait() [0xb1caa000..0xb1caad90] at java.lang.Object.wait(Native Method) - waiting on <0x05f3f328> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x05f3f328> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462)

“Job_Executor3” daemon prio=5 tid=0x010265c0 nid=0x9e5a00 in Object.wait() [0xb1c29000..0xb1c29d90] at java.lang.Object.wait(Native Method) - waiting on <0x05f723e0> (a [I) at com.sun.jmx.remote.opt.util.JobExecutor.run(JobExecutor.java:87) - locked <0x05f723e0> (a [I)

“Job_Executor2” daemon prio=5 tid=0x01023b00 nid=0x9e4c00 in Object.wait() [0xb1ba8000..0xb1ba8d90] at java.lang.Object.wait(Native Method) - waiting on <0x05f2fa00> (a [I) at com.sun.jmx.remote.opt.util.JobExecutor.run(JobExecutor.java:87) - locked <0x05f2fa00> (a [I)

“Job_Executor1” daemon prio=5 tid=0x010229f0 nid=0x9e7a00 in Object.wait() [0xb1b27000..0xb1b27d90] at java.lang.Object.wait(Native Method) - waiting on <0x05f29e60> (a [I) at com.sun.jmx.remote.opt.util.JobExecutor.run(JobExecutor.java:87) - locked <0x05f29e60> (a [I)

“Job_Executor0” daemon prio=5 tid=0x01020f40 nid=0x87f000 in Object.wait() [0xb0e8e000..0xb0e8ed90] at java.lang.Object.wait(Native Method) - waiting on <0x05f21c50> (a [I) at com.sun.jmx.remote.opt.util.JobExecutor.run(JobExecutor.java:87) - locked <0x05f21c50> (a [I)

“Thread-13” daemon prio=5 tid=0x0101f560 nid=0x9e3e00 in Object.wait() [0xb1aa6000..0xb1aa6d90] at java.lang.Object.wait(Native Method) - waiting on <0x05ed2e00> (a com.tc.management.remote.protocol.terracotta.TunnelingEventHandler) at java.lang.Object.wait(Object.java:474) at com.tc.management.remote.protocol.terracotta.TunnelingEventHandler.accept(TunnelingEventHandler.java:84) - locked <0x05ed2e00> (a com.tc.management.remote.protocol.terracotta.TunnelingEventHandler) at com.tc.management.remote.protocol.terracotta.TunnelingMessageConnectionServer.accept(TunnelingMessageConnectionServer.java:31) at com.sun.jmx.remote.generic.SynchroMessageConnectionServerImpl.accept(SynchroMessageConnectionServerImpl.java:47) at javax.management.remote.generic.GenericConnectorServer$Receiver.run(GenericConnectorServer.java:340)

“Timer-3” daemon prio=5 tid=0x0101e600 nid=0x9e3000 in Object.wait() [0xb1a25000..0xb1a25d90] at java.lang.Object.wait(Native Method) - waiting on <0x05f04740> (a java.util.TaskQueue) at java.lang.Object.wait(Object.java:474) at java.util.TimerThread.mainLoop(Timer.java:483) - locked <0x05f04740> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462)

“Low Memory Detector” daemon prio=5 tid=0x01018cb0 nid=0x9e1400 runnable [0x00000000..0x00000000]

“CompilerThread0” daemon prio=9 tid=0x01018280 nid=0x9e0600 waiting on condition [0x00000000..0xb18a27d8]

“Signal Dispatcher” daemon prio=9 tid=0x01017dc0 nid=0x9df800 waiting on condition [0x00000000..0x00000000]

“WorkerThread(dmi_stage,0)” daemon prio=5 tid=0x010170f0 nid=0x9dbc00 in Object.wait() [0xb17a0000..0xb17a0d90] at java.lang.Object.wait(Native Method) - waiting on <0x05edf3e8> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edf3e8> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(receive_root_id_stage,0)” daemon prio=5 tid=0x01016cb0 nid=0x9dae00 in Object.wait() [0xb171f000..0xb171fd90] at java.lang.Object.wait(Native Method) - waiting on <0x05edc340> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edc340> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(lock_statistics_enable_disable_stage,0)” daemon prio=5 tid=0x01016870 nid=0x9da000 in Object.wait() [0xb169e000..0xb169ed90] at java.lang.Object.wait(Native Method) - waiting on <0x05edf528> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edf528> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(batch_txn_ack_stage,0)” daemon prio=5 tid=0x01016430 nid=0x9ca800 in Object.wait() [0xb161d000..0xb161dd90] at java.lang.Object.wait(Native Method) - waiting on <0x05edf5f8> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edf5f8> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(receive_object_stage,0)” daemon prio=5 tid=0x01015ff0 nid=0x9c9a00 in Object.wait() [0xb159c000..0xb159cd90] at java.lang.Object.wait(Native Method) - waiting on <0x05edf6c8> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edf6c8> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(lock_statistics_response_stage,0)” daemon prio=5 tid=0x01015bb0 nid=0x9c8c00 in Object.wait() [0xb151b000..0xb151bd90] at java.lang.Object.wait(Native Method) - waiting on <0x05edf798> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edf798> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(receive_transaction_complete_stage,0)” daemon prio=5 tid=0x01015770 nid=0x8dc600 in Object.wait() [0xb149a000..0xb149ad90] at java.lang.Object.wait(Native Method) - waiting on <0x05edf868> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edf868> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(object_id_request_response_stage,0)” daemon prio=5 tid=0x01015330 nid=0x8db800 in Object.wait() [0xb1419000..0xb1419d90] at java.lang.Object.wait(Native Method) - waiting on <0x05edf938> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edf938> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(lock_response_stage,0)” daemon prio=5 tid=0x01014ef0 nid=0x8daa00 in Object.wait() [0xb1398000..0xb1398d90] at java.lang.Object.wait(Native Method) - waiting on <0x05edc3a0> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edc3a0> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(jmxremote_tunnel_stage,0)” daemon prio=5 tid=0x01014ab0 nid=0x934400 in Object.wait() [0xb1317000..0xb1317d90] at java.lang.Object.wait(Native Method) - waiting on <0x05edfa78> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edfa78> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(receive_transaction_stage,0)” daemon prio=5 tid=0x01014670 nid=0x933600 in Object.wait() [0xb1296000..0xb1296d90] at java.lang.Object.wait(Native Method) - waiting on <0x05edfb48> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edfb48> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(client_coordination_stage,0)” daemon prio=5 tid=0x01014230 nid=0x932800 in Object.wait() [0xb1215000..0xb1215d90] at java.lang.Object.wait(Native Method) - waiting on <0x05edfc18> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edfc18> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“WorkerThread(hydrate_message_stage,0)” daemon prio=5 tid=0x01013e60 nid=0x879000 in Object.wait() [0xb1194000..0xb1194d90] at java.lang.Object.wait(Native Method) - waiting on <0x05edfce8> (a java.lang.Object) at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(Unknown Source) - locked <0x05edfce8> (a java.lang.Object) at com.tc.util.concurrent.TCBoundedLinkedQueue.poll(TCBoundedLinkedQueue.java:29) at com.tc.async.impl.StageQueueImpl.poll(StageQueueImpl.java:103) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:129)

“TC Memory Monitor” daemon prio=5 tid=0x01013060 nid=0x877400 waiting on condition [0xb1092000..0xb1092d90] at java.lang.Thread.sleep(Native Method) at com.tc.runtime.TCMemoryManagerImpl$MemoryMonitor.run(TCMemoryManagerImpl.java:132) at java.lang.Thread.run(Thread.java:613)

“Reaper” daemon prio=5 tid=0x01012cb0 nid=0x9d0400 in Object.wait() [0xb1011000..0xb1011d90] at java.lang.Object.wait(Native Method) - waiting on <0x05ed2de0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) - locked <0x05ed2de0> (a java.lang.ref.ReferenceQueue$Lock) at com.tc.object.ClientObjectManagerImpl$2.run(ClientObjectManagerImpl.java:1192)

“H2 Log Writer STATISTICS-BUFFER” daemon prio=5 tid=0x010122d0 nid=0x9cf600 waiting on condition [0xb0f90000..0xb0f90d90] at java.lang.Thread.sleep(Native Method) at org.h2.store.WriterThread.run(WriterThread.java:147)

“H2 File Lock Watchdog /Users/tgautier/src/forge/cookbook/string/statistics-192.168.110.102/statistics-buffer.lock.db” daemon prio=9 tid=0x01011b10 nid=0x9ce800 waiting on condition [0xb0f0f000..0xb0f0fd90] at java.lang.Thread.sleep(Native Method) at org.h2.store.FileLock$1.run(FileLock.java:198) at java.lang.Thread.run(Thread.java:613)

“com.tc.util.ToggleableReferenceManager$QueueProcessor” daemon prio=5 tid=0x010110e0 nid=0x87e200 in Object.wait() [0xb0e0d000..0xb0e0dd90] at java.lang.Object.wait(Native Method) - waiting on <0x05df45b0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) - locked <0x05df45b0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) at com.tc.util.ToggleableReferenceManager$QueueProcessor.run(ToggleableReferenceManager.java:129)

“DSO Lock Object.wait() timer” daemon prio=5 tid=0x01010ac0 nid=0x997400 in Object.wait() [0xb0d8c000..0xb0d8cd90] at java.lang.Object.wait(Native Method) - waiting on <0x05df4638> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x05df4638> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462)

“RemoteTransactionManager Flusher” daemon prio=5 tid=0x010104f0 nid=0x996600 in Object.wait() [0xb0d0b000..0xb0d0bd90] at java.lang.Object.wait(Native Method) - waiting on <0x05df46e0> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x05df46e0> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462)

“HealthChecker” daemon prio=5 tid=0x0100fbf0 nid=0x87d400 waiting on condition [0xb0c8a000..0xb0c8ad90] at java.lang.Thread.sleep(Native Method) at java.lang.Thread.sleep(Thread.java:276) at com.tc.util.concurrent.ThreadUtil.reallySleep(ThreadUtil.java:24) at com.tc.util.concurrent.ThreadUtil.reallySleep(ThreadUtil.java:16) at com.tc.net.protocol.transport.ConnectionHealthCheckerImpl$HealthCheckerMonitorThreadEngine.run(ConnectionHealthCheckerImpl.java:164) at java.lang.Thread.run(Thread.java:613)

“TCComm Main Selector Thread” daemon prio=5 tid=0x0100f780 nid=0x8b9000 runnable [0xb0c09000..0xb0c09d90] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:135) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x05deeca0> (a sun.nio.ch.Util$1) - locked <0x05deecb0> (a java.util.Collections$UnmodifiableSet) - locked <0x05deec58> (a sun.nio.ch.KQueueSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84) at com.tc.net.core.CoreNIOServices.selectLoop(CoreNIOServices.java:338) at com.tc.net.core.CoreNIOServices.run(CoreNIOServices.java:61)

“SampledCounterManager Timer” daemon prio=5 tid=0x0100f180 nid=0x8b8200 in Object.wait() [0xb0b88000..0xb0b88d90] at java.lang.Object.wait(Native Method) - waiting on <0x05de8a78> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x05de8a78> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462)

“startlevel job thread” daemon prio=5 tid=0x0100dbc0 nid=0x9b4600 in Object.wait() [0xb0a86000..0xb0a86d90] at java.lang.Object.wait(Native Method) - waiting on <0x05d03118> (a org.knopflerfish.framework.Queue) at org.knopflerfish.framework.Queue.removeWait(Queue.java:108) - locked <0x05d03118> (a org.knopflerfish.framework.Queue) at org.knopflerfish.framework.StartLevelImpl.run(StartLevelImpl.java:170) at java.lang.Thread.run(Thread.java:613)

“Finalizer” daemon prio=8 tid=0x01007810 nid=0x81a000 in Object.wait() [0xb0a05000..0xb0a05d90] at java.lang.Object.wait(Native Method) - waiting on <0x05ab3048> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) - locked <0x05ab3048> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

“Reference Handler” daemon prio=10 tid=0x01007410 nid=0x819200 in Object.wait() [0xb0984000..0xb0984d90] at java.lang.Object.wait(Native Method) - waiting on <0x05ab30c8> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:474) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) - locked <0x05ab30c8> (a java.lang.ref.Reference$Lock)

“main” prio=5 tid=0x01001450 nid=0xb0801000 runnable [0xb07ff000..0xb0800148] at java.lang.Throwable.fillInStackTrace(Native Method) at java.lang.Throwable.(Throwable.java:181) at java.lang.Exception.(Exception.java:29) at java.lang.RuntimeException.(RuntimeException.java:32) at java.lang.IllegalMonitorStateException.(IllegalMonitorStateException.java:31) at Main.load(Main.java:46) at Main.run(Main.java:70) at Main.main(Main.java:78)

“VM Thread” prio=9 tid=0x01006b60 nid=0x809800 runnable

“VM Periodic Task Thread” prio=9 tid=0x01019870 nid=0x9e2200 waiting on condition

“Exception Catcher Thread” prio=10 tid=0x01001670 nid=0x80a800 runnable

Hung Huynh 2014-02-12

DSO is discontinued