• Bug
  • Status: Resolved
  • 2 Major
  • Resolution: Fixed
  • ehcache-core
  • cdennis
  • Reporter: sb017701
  • March 01, 2013
  • 0
  • Watchers: 4
  • September 12, 2014
  • March 05, 2013

Attachments

Description

We have noticed a periodic issues in our production environment with multiple caches where it appears that when calling getAllWithLoader() the CacheLoader implementation is not being invoked for the key (either because we know the key exists or the cache loader implementation was written to always return a value).

After further investigation we believe the issue is related to attempting to retrieve an expired, but un-purged key from the cache.

In net.sf.ehcache.Cache.getAllWithLoader() there is the following block of code:

    for (Object key1 : keys) {
        key = key1;
        Element element = get(key);

        if (element != null) {
            map.put(key, element.getObjectValue());
        } else if (isKeyInCache(key)) {
            map.put(key, null);
        } else {
            missingKeys.add(key);
        }
    }

We believe the issue is the call to get(key) returns null, since it is expired, however the doc for isKeyInCache(key) states: {quote} This method is not synchronized. It is possible that an element may exist in the cache and be removed before the check gets to it, or vice versa. *Since no assertions are made about the state of the Element it is possible that the Element is expired, but this method still returns true.* {quote}

This results in a null entry being added to the results map for the key and the cache loader not being invoked.

This is behaviorally different from getWithLoader() which does not evaluate isKeyInCache() and instead correctly calls the cache loader for an expired key, so it appears to only be an issue when attempting to load multiple keys.

I have attached a sample jUnit that demonstrates the behavior we are seeing.

Sample Test Output: java.lang.AssertionError: Encountered 20 failures; start time: 2013-03-01T14:34:16.241-06:00; failure messages: Failed to find value:4 Failed to find value:5 Failed to find value:4 Failed to find value:0 Failed to find value:1 Failed to find value:1 Failed to find value:5 Failed to find value:8 Failed to find value:8 Failed to find value:2 Failed to find value:2 Failed to find value:3 Failed to find value:3 Failed to find value:3 Failed to find value:3 Failed to find value:1 Failed to find value:1 Failed to find value:1 Failed to find value:2 Failed to find value:2 at org.junit.Assert.fail(Assert.java:88)

Comments

Chris Dennis 2013-03-05

The original fix for EHC-809 did not account for concurrent reads being able to prevent inline removal of expired entries. There is no reason why we should ever put an explicit null value in to the results map - so I’ve removed that code, if the Element is now null we invoke the reader, if it isn’t we add it’s value to the results map.

Daniel Barker 2014-09-12

I’m still experiencing the same failures with the provided test. I have run the test using versions 2.6.9, 2.7.1, 2.7.6, and 2.8.3. I get intermittent passes and failures with the test on all versions. I believe there remains a concurrency issue here and this has only fixed the adding an explicit null value issue that caused more errors than I am seeing now.