[pylucene-dev] pylucene and pylons

[John Beaver]

Sounds interesting. It looks like I've got what I need for now, but I'll 
watch your project for when I need something a bit more robust.

Pete wrote:
> Hi John-
>
> You might be interested in my project GrassyKnoll. It's somewhat 
> similar to Solr, in that it's a standalone app and fills a similar 
> role.  From your perspective, it differs in a few key (and hopefully 
> helpful) ways:
>
>  * standalone server.  GrassyKnoll includes it's own webserver - no 
> Apache/Tomcat required
>  * pure python. Besides extension modules, such as PyLucene, all GK's 
> code and it's minimal dependencies are pure python modules.
>  * written in Python.  If you need to customize the server, you can do 
> so in Python instead of Java
>  * extensive documentation.  GrassyKnoll is documented on a wiki, not 
> the mailing list.
>  * concurrency issues resolved.  As a standalone server, GK handles 
> PyLucene concurrency issues so you don't have to.
>
> We should have a production-ready 0.4 release in early May.
>
> More info (incl. mailing lists & IRC) at 
> http://grassyknoll.googlecode.com
>
> --Pete
>
>
> On Apr 11, 2008, at 12:14 PM, John Beaver wrote:
>> Hi TJ. Sounds like a Solr-like solution. I was avoiding Solr, since 
>> I'm already going to have a hard time getting IT to install a Pylons 
>> app on the webserver; Tomcat would take more time.
>>
>> What software/libraries do you use for this? Code samples?
>>
>>
>>
>> TJ Ninneman wrote:
>>> I'm sure someone can shed some light on this for you but I'm a 
>>> strong believer in separating the front end web app and any backend 
>>> search/indexing services (just like we separate the front end webapp 
>>> from the backend database (mysql, pgsql, etc).
>>>
>>> Our Pylons app simply queries a backend XML-RPC instance which can 
>>> be threaded, forking, single threaded, twisted or whatever.  Let me 
>>> just say it's a lot easier debugging concurrency issues in a a 
>>> simple XML-RPC server than in Apache, Paste Http, CherryPy, etc.
>>>
>>> TJ
>>>
>>> On Apr 11, 2008, at 10:47 AM, John Beaver wrote:
>>>
>>>> Hi, I'm trying to integrate PyLucene with a Pylons app, but I'm 
>>>> having serious stability problems. Does anyone have experience with 
>>>> this?
>>>>
>>>> Specifically, I have a single action which does the following:
>>>>
>>>> lucene.initVM(lucene.CLASSPATH)
>>>> directory = FSDirectory.getDirectory(Globals.LUCENE_INDEX_PATH);
>>>> searcher = IndexSearcher(directory)
>>>> analyzer = StandardAnalyzer()
>>>> query = QueryParser('terms', analyzer).parse(term)
>>>> hits = searcher.search(query)
>>>> ... process hits
>>>>
>>>> It works perfectly the first time I view the corresponding webpage, 
>>>> but it reproducibly crashes the webserver (paster serve) the second 
>>>> time I try, always while in the initVM() function.
>>>>
>>>> I've tried putting the initVM() call in middleware.py, and I've 
>>>> tried using a global flag to make sure that initVM() is only called 
>>>> once in the controller. Both seem to fail when a different thread 
>>>> tries to access PyLucene functions.
>>>>
>>>> I'm assuming this is a threading issue, so I'm going to write some 
>>>> code to ensure that everything happens within a single thread. But 
>>>> does anyone have a simpler solution for getting this working within 
>>>> Pylons?
>>>> _______________________________________________
>>>> pylucene-dev mailing list
>>>> pylucene-dev at osafoundation.org
>>>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>>>
>>> _______________________________________________
>>> pylucene-dev mailing list
>>> pylucene-dev at osafoundation.org
>>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>>>
>> _______________________________________________
>> pylucene-dev mailing list
>> pylucene-dev at osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>