• Patch
  • Status: Open
  • 1 Critical
  • Resolution:
  • ehcache-core
  • ssubbiah
  • Reporter:
  • September 21, 2009
  • 2
  • Watchers: 4
  • November 02, 2011

Attachments

Description

It would be good if ehcache supported invalidation groups, like oscache and jcs. e.g. when caching an object you can specify the set of groups it belongs to, so that you can then invalidate a group of objects rather than just specific objects via their id.

You can emulate this functionality by creating a set to represent each group, and when you cach an element you fetch from the cache the appropriate sets, add the element’s key to them too, then cache the sets again. It could be implemented more efficiently if it was built-in functionality to ehcache though.

Sourceforge Ticket ID: 1734460 - Opened By: nobody - 10 Jun 2007 16:19 UTC

Comments

gluck 2009-09-25

This has been implemented by Andrew Liles in a branch which is used in production. It will be released soon-ish.

Fiona OShea 2010-07-21

This might already be done based on Greg’s comments

Fiona OShea 2010-09-01

Moving all unresolved “Fix Revision 2.2.1” to fix revision “unknown” as we are releasing Magnum first which is 2.3. Currently not sure which fix version these will actually be in, but they are targeted for Fremantle release

gluck 2010-10-19

Greg,

I am running short on free time, so I though I’d at least send you Andrew’s patch ported to the current Ehcache trunk. The only modification I did was to replace group keys from Object to String, to make sure a cache element remains Serializable.

Some additional cleanup could probably be done via a cache event listener to remove obsolete group keys. This shouldn’t incur too much cost IMHO.

Do you have documentation on the new decorator feature? Some bootstrapping would allow me to make the most efficient use of my free time! ;-)

Also, how about storing all of the groups meta-data in a single cache element? We would avoid clashing with regular elements that way, at the cost of managing our own (internal) structure.

Cédrik

gluck 2010-10-19

Latest Groups feature patch against current trunk

Lasse Voss 2010-10-20

I have looked at the patch and compared it to the home-made solution I cooked up (using a map to track the key-group mapping that is external to the cache).

This missing feature of EhCache is effectively keeping us from migrating from OsCache at the moment. I think it is an essential feature of the cache.

Good to hear this has been in production, but before merging it, I think the points raised by Cédrik should be addressed:

  • Use a cache event listener to remove mappings of expired keys etc. (this is what I did as well).
  • Storing a separate cache element for each group pollutes the cache with “special entries” and increase the risk of key clashes with “normal” elements.

Any chances these issues might be addressed soon?

Cedrik LIME 2010-10-21

Lasse,

I _may_ have some more time next week for a go at storing all of the groups data in a single cache element (instead of several). (This would have the added side effect that the single element will very frequently accessed, so should mean the chance of the group meta data being evicted is minimized.)

Would you mind contributing your cache event listener that remove mappings of expired keys?

Lasse Voss 2010-10-25

Adapted portions of our EhCacheFacade code to illustrate usage of a CacheEventListener for cache group cleanup.

Lasse Voss 2010-10-25

I realize that the code I just submitted is quiet particular to our usage of EhCache.

Actually, I had to adapt the code manually, so this is not exactly what runs here, because we have a fixed set of cache group keys, which allows for optimizations for storing the mapping (just a fixed list of cache element sets, not a map of sets). Our cache group mapping is external to the cache.

You will also see, that since we did not patch the Element class so that it contains the associated cache group(s), the event listener is only used for removing elements (also takes care of inline expiration). With the patched Element class, it should be possible to move all code handling cache groups into the listener and thus maybe implement the whole cache group logic through a listener? This way maybe the cache group feature could be optional through a config setting, and if switched off, the listener would not be installed and thus cache performance not impacted.

We (try to) use JGroups replication with our caches, replicating only removals and updates without copy, so effectively only cache removals. So there’s no issue there, but for replicating puts and updatesViaCopy, the JGroupEventMessage would need to be extended with the cache groups of the element (same for JMS replication etc.).

Cedrik LIME 2010-11-12

Updated Groups feature patch against current trunk. This one is storing all of the groups data in a single cache element (instead of several). I haven’t had time to have a look at the cache event listener to remove mappings of expired keys yet.

Cedrik LIME 2010-11-15

Committed the groups feature in trunk. Still missing is the cache event listener to remove mappings of expired keys.

Lasse Voss 2010-11-23

I have reviewed Cedrik’s patch and have a suggestion to make things more simple:

Why don’t we make an Element’s groups final, i.e. they have to be assigned on instance creation and are immutable afterwards? This would do away with a lot of code in the Element class since there is no need to care about synchronization issues.

Lasse Voss 2010-11-30

Another issue that needs to be addressed before the groups feature released:

At the moment, the new cache operation Cache.removeByGroup(…) just removes the elements belonging to the group.

While this is what needs to be done locally, it is not efficient for replication because it may result in a great number of remove message to be send and is likely to miss elements that belong to the same group in another cache instance.

What we need is a new message for this case, alogn the lines of EventMessage.REMOVE_GROUP, which could simply contain the grouo name in the EventMessage.serializedKey property and upon being received by another instance would trigger a local Cache.removeByGroup(…) for that group.

Lasse Voss 2011-02-14

Any chance this feature will make it into the Freemantle release?

We cannot move to EhCache unless this feature working reliably.

Alas, our own add-on solution (using EhCache 2.3) does not perform under the kind of load we have in out cluster, although there might still be some tuning possible.

Fiona OShea 2011-02-18

Email Thread regarding this issue - From Steve They are asking to be able to group a bunch of objects and have them invalidate together. I believe there is an ehcache patch associated with it. I’ll try to dig up more information. It’s been a long standing feature request because OSCache has it but I can’t remember why it’s useful :-)

Cheers,

Steven Harris [email protected]

On Feb 14, 2011, at 11:19 PM, Saravanan Subbiah wrote:

Not sure whats the feature they are requesting. Also seems like they are talking about ehcache standalone. Anyways, unless someone explains to me the problem they are having and the fix they are proposing I dont have anything to comment about.

cheers, Saravanan

On 2/14/11 1:34 PM, Fiona O’Shea wrote:

Saro can you add a private “terracotta_internal” comment on this? I’ll send it over to PM then to decide when they want to try to add this. Greg also said move it out

—– Original Message —– From: “Fiona O’Shea”[email protected] To: “Steve Harris”[email protected], “Saravanan Subbiah”[email protected] Sent: Monday, February 14, 2011 11:26:55 AM Subject: Fwd: EHC-240 - support for Invalidation Groups

Guys Greg asked me if you two could look at this. He says

We can do this standalone but I am not sure if we can do it distributed efficiently. Can you ask Steve to get someone from the server team to do an evaluation of this feature?

Fiona OShea 2011-02-18

Investigate the distributed aspect of this for Ulloa. See email thread in previous comment.

Lasse Voss 2011-02-22

We have recently upgraded to EhCache 2.4 and implemented this feature using the new search functionality and this seems to work okay. This required us to patch several classes, especially the Element itself. The beauty of this implementation is that there is no need for storing group-key mappings within or outside of the cache, avoiding the penalty of additional synchronization.

Since we only allow a single group per element (actually an enum) this is pretty fast in our case with memory-only storage.

Maybe that is a pointer for how to implement this properly in EhCache.

Mindaugas Zaksauskas 2011-11-02

+1 for this feature.

We share a similar story (migration from OSCache, heavily using groups, etc.) so we had to implement storing the groups in the element value itself. We can only bear this hack because we only flush the entries rarely (never mind proper synchronization), but this might not be good enough for anyone else.