• Bug
  • Status: Closed
  • 1 Critical
  • Resolution: Fixed
  • asingh
  • Reporter: bobrob
  • February 11, 2008
  • 0
  • Watchers: 1
  • April 10, 2009
  • March 10, 2009

Attachments

Description

[Posted to JIRA by request from the Forums] If one Http request to a Spring controller is active and running, other requests (such as AJAX requests) are blocked until the first request completes.

The example that started it: We have a Spring app that allows the upload of a file in a FORM POST with input type=file. The files might be large, in the 200-250MB range. We want to have a progress meter that shows the percent complete of the upload that is updated via AJAX (using DWR).

Running standalone under Tomcat, this application works as expected. The progress meter moves smoothly between 0 and 100% complete. Under Terracotta, the percent does not update until the file is completely uploaded. The suspicion is that the Spring dispatch servlet is locking the session somehow, and isn’t allowing the DWR request to complete until the first request is done.

Flow:

  1. FORM POST to a file upload controller in Spring
  2. AJAX progress % is shown every second in the browser until #1 is 100%
  3. FORM POST (file upload) completes.

Terracotta fails at #2, while standalone Tomcat does not.

I’ve attached a simplified test application that shows the problem. Running standalone, the clock in the page updates consistently. Running under TC (both 2.5.0 and 2.5.1), the clock stops after you hit the “Upload File” button, and resumes when the file is uploaded (upload is simulated with Thread.sleep). Log files belong are from the test application, showing the 10 second gap between AJAX updates, instead of the expected 1 sec gap. Test app can be referenced with [server:port]/tctester/index.html

[Here is where the post starts] 2008-02-10 13:03:39,671 [http-8080-Processor3] INFO com.terracottatech.dso.runtime - Named lock [54138000CE95493621DF] acquired with level WRITE (2) at com.terracotta.session.util.Lock.getWriteLock(Lock.java:36) at com.terracotta.session.util.DefaultSessionId.getWriteLock(DefaultSessionId.java:5 at com.terracotta.session.SessionDataStore.find(SessionDataStore.java:83) at com.terracotta.session.TerracottaSessionManager.getSessionIfExists(TerracottaSessionManager.java:346) at com.terracotta.session.SessionRequest.getTerracottaSession(SessionRequest.java:10 at com.terracotta.session.SessionRequest.getSession(SessionRequest.java:72) at org.springframework.web.context.request.ServletRequestAttributes.(ServletRequestAttributes.java:81) at org.springframework.web.context.request.RequestContextListener.requestInitialized(RequestContextListener.java:63) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:167) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at com.tc.tomcat55.session.SessionValve55.tcInvoke(SessionValve55.java:63) at com.tc.tomcat55.session.SessionValve55.invoke(SessionValve55.java:50) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:14 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:595)

[Here is the next AJAX call lock, 10 secs later instead of the 1 sec expected] 2008-02-10 13:03:49,703 [http-8080-Processor4] INFO com.terracottatech.dso.runtime - Named lock [54138000CE95493621DF] acquired with level WRITE (2) at com.terracotta.session.util.Lock.getWriteLock(Lock.java:36) at com.terracotta.session.util.DefaultSessionId.getWriteLock(DefaultSessionId.java:5 at com.terracotta.session.SessionDataStore.find(SessionDataStore.java:83) at com.terracotta.session.TerracottaSessionManager.getSessionIfExists(TerracottaSessionManager.java:346) at com.terracotta.session.SessionRequest.getTerracottaSession(SessionRequest.java:10 at com.terracotta.session.SessionRequest.getSession(SessionRequest.java:72) at org.springframework.web.context.request.ServletRequestAttributes.(ServletRequestAttributes.java:81) at org.springframework.web.context.request.RequestContextListener.requestInitialized(RequestContextListener.java:63) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:167) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at com.tc.tomcat55.session.SessionValve55.tcInvoke(SessionValve55.java:63) at com.tc.tomcat55.session.SessionValve55.invoke(SessionValve55.java:50) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:14 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:595)

Comments

Fiona OShea 2008-02-11

http://forums.terracotta.org/forums/posts/list/789.page

Fiona OShea 2008-02-19

Validate that assumptions about Spring and/OR DWR upload are correct. That it would work in a session replicated non-sticky load balanced environment.

Fiona OShea 2008-02-19

Potential solution

  1. Advanced Session Mode: Remove the lock that is held for the duration of the request. The only implicit locking that will be done will be around retrieving the session from the store, and on things like getAttribute()/setAttribute(). This means that mutation made to share objects will now need to be locked (previously the lock held around the request would have defined the txn/lock scope for such mutations)

It’s also worth mentioning that the lock that is held now is used to determine whether a session can be invalidated. If the lock is held, that means there is still an active request using that session even though the “lastAccessedTime” might indicate that a session has exceeded it’s max idle timeout. Removing the request wide lock will interfere with this and will be compensated for. SessionInvalidatorTest tests this scenario of a long running request

open question: should we allow this behavior to be scoped to some set of matching URL patterns? So if the request doesn’t match some configurable pattern(s), then the existing locking applies. Not sure if this is too confusing or has any other technical flaws

Abhishek Singh 2009-01-20

Done. The changes are in trunk. Need to resolve DOC issues.

A working demo showing session-locking is present at https://svn.terracotta.org/repo/forge/projects/session-locking-demo/

nadeem ghani 2009-02-26

session-locking=true or false doesn’t change behavior

to reproduce, use the sample app at forge/session-locking-demo and run mvn -Ptomcat6xWithSL tc:run no deadlock seen edit pom.xml to point to TC 2.7.3 deadlock seen

Alex Miller 2009-02-27

Assigning to you since Abhishek is off having fun.

Abhishek Singh 2009-03-09

Session-locking=true/false is not supported in 2.7 branch.

Do we need support in 2.7 ?

nadeem ghani 2009-03-10

Not a problem with the feature, but with the demo app.

Fiona OShea 2009-03-20

This should have been closed as “Will not Fix” as it is not a bug