CDV ❯ Support Lucene Search Index
-
New Feature
-
Status: Closed
-
2 Major
-
Resolution: Fixed
-
-
-
juris
-
Reporter: jhartley
-
January 07, 2007
-
0
-
Watchers: 0
-
July 27, 2012
-
March 20, 2007
Attachments
Description
Motivation: Target Lucene (http://lucene.apache.org/) support to increase Terracotta’s transparency and add to the portfolio of Terracotta clustered frameworks.
Requirements: Successfully cluster a Lucene search index such that an index is generated on one node and utilized on a second node. Test with an existing Lucene powered Java application (http://en.wikipedia.org/wiki/Lucene) or develop a sample application if needed.
Comments
Nathaniel Harward 2007-01-31
orion 2007-01-31
I’ve put up some sample code that does clustering lucene. We also have a request from a community member (officialleader AT yahoo.com) to participate in the Lucene clustering effort.
The sample code that I made is up on the Forge Labs project: http://wiki.terracotta.org/confluence/display/labs/Lucene+Clustering
Eric Ellis 2007-02-19
The first phase of support is complete. We have a Lucene project with a simple test which uses the Lucene demo classes to parse an HTML version of the Bible and then search for lamb and Buddha. The test clusters two nodes, searches for Buddha (which is not found), then adds the keyword and searches again. The same is done with lamb (which is found in 14 books) where the phrase “Mary had a little lamb” is added and again searched on both nodes.
A second test will be written which will query, under load, all methods of the Directory interface to produce lock contention.
Gary Keim 2007-02-28
There’s apparently more to this than meets the eye.
Eric Ellis 2007-03-01
Various research files such as bookmarks, a thread dump (of lock contention using hashtable locking), lock attributes.
Nathaniel Harward 2007-03-01
Eric got this working, but apparently the performance is not what it should be. It will require a deeper investigation to tune this to be satisfactory – not sure if this can happen for Moraga or if it will have to wait.
Steve Harris 2007-03-02
Maybe tomorrow or monday I’ll go over it with him. Is it slow on add or query? Used to be pretty fast on query but a bit slow on adds.
Fiona OShea 2007-03-08
I’ve spoken with Taylor and we agree that this has been finished to requirements outlined in PRD. Any other issues with it, outside of the requirements should be logged as bugs. Is there anything else to complete on this feature?
Nathaniel Harward 2007-03-08
This is not quite ready to go, it needs to be converted to a configuration module and apparently requires some tuning before it will be usable in large indices.
Juris Galang 2007-03-19
We now have a Clustered Lucene module. We only have 1 test - it basically checks that the RAMDirectory is clustered. We’ll need to add some performance and load tests.
Fiona OShea 2007-03-20
Resolving as this is done for 2.3, except for adding more tests.
This should be done in /trunk for now – as the “config bundles” infrastructure plays out we can migrate over.