[pylucene-dev] pylucene and pylons

[Pete]

Hi John-

You might be interested in my project GrassyKnoll. It's somewhat  
similar to Solr, in that it's a standalone app and fills a similar  
role.  From your perspective, it differs in a few key (and hopefully  
helpful) ways:

  * standalone server.  GrassyKnoll includes it's own webserver - no  
Apache/Tomcat required
  * pure python. Besides extension modules, such as PyLucene, all GK's  
code and it's minimal dependencies are pure python modules.
  * written in Python.  If you need to customize the server, you can  
do so in Python instead of Java
  * extensive documentation.  GrassyKnoll is documented on a wiki, not  
the mailing list.
  * concurrency issues resolved.  As a standalone server, GK handles  
PyLucene concurrency issues so you don't have to.

We should have a production-ready 0.4 release in early May.

More info (incl. mailing lists & IRC) at http://grassyknoll.googlecode.com

--Pete


On Apr 11, 2008, at 12:14 PM, John Beaver wrote:
> Hi TJ. Sounds like a Solr-like solution. I was avoiding Solr, since  
> I'm already going to have a hard time getting IT to install a Pylons  
> app on the webserver; Tomcat would take more time.
>
> What software/libraries do you use for this? Code samples?
>
>
>
> TJ Ninneman wrote:
>> I'm sure someone can shed some light on this for you but I'm a  
>> strong believer in separating the front end web app and any backend  
>> search/indexing services (just like we separate the front end  
>> webapp from the backend database (mysql, pgsql, etc).
>>
>> Our Pylons app simply queries a backend XML-RPC instance which can  
>> be threaded, forking, single threaded, twisted or whatever.  Let me  
>> just say it's a lot easier debugging concurrency issues in a a  
>> simple XML-RPC server than in Apache, Paste Http, CherryPy, etc.
>>
>> TJ
>>
>> On Apr 11, 2008, at 10:47 AM, John Beaver wrote:
>>
>>> Hi, I'm trying to integrate PyLucene with a Pylons app, but I'm  
>>> having serious stability problems. Does anyone have experience  
>>> with this?
>>>
>>> Specifically, I have a single action which does the following:
>>>
>>> lucene.initVM(lucene.CLASSPATH)
>>> directory = FSDirectory.getDirectory(Globals.LUCENE_INDEX_PATH);
>>> searcher = IndexSearcher(directory)
>>> analyzer = StandardAnalyzer()
>>> query = QueryParser('terms', analyzer).parse(term)
>>> hits = searcher.search(query)
>>> ... process hits
>>>
>>> It works perfectly the first time I view the corresponding  
>>> webpage, but it reproducibly crashes the webserver (paster serve)  
>>> the second time I try, always while in the initVM() function.
>>>
>>> I've tried putting the initVM() call in middleware.py, and I've  
>>> tried using a global flag to make sure that initVM() is only  
>>> called once in the controller. Both seem to fail when a different  
>>> thread tries to access PyLucene functions.
>>>
>>> I'm assuming this is a threading issue, so I'm going to write some  
>>> code to ensure that everything happens within a single thread. But  
>>> does anyone have a simpler solution for getting this working  
>>> within Pylons?
>>> _______________________________________________
>>> pylucene-dev mailing list
>>> pylucene-dev at osafoundation.org
>>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>>
>> _______________________________________________
>> pylucene-dev mailing list
>> pylucene-dev at osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>>
> _______________________________________________
> pylucene-dev mailing list
> pylucene-dev at osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev

-- 
Pete
pfein at pobox.com