[pylucene-dev] pylucene and recommendations for RAM

[Andi Vajda]

On Thu, 5 Apr 2007, David Pratt wrote:

> Hi Andi. I was thinking of using twisted's pb to retreive the objects from 
> remote servers over the wire - sorry for not being clear. I should be able to 
> load balance pb servers as to which server handles aggregation easily enough 
> and hopefully merge results. I thought hits objects might be able to be 
> merged.
>
> To get better performance, each remote server could use index in RAM as 
> opposed to filesystem. As far as writing to the distributed index, I could 
> keep track of which index an object is writing to in the same way the load 
> balancing is done for searching but a db is needed to keep track. A 
> replication strategy could use the db to rebuild index in case a machine goes 
> down (or you would temporarily loose results from all servers until the index 
> was rebuilt).
>
> Does this type of approach have any merit? What sort of strategy do you 
> envision? Many thanks.

It's getting the objects over the wire that is going to be tricky. Once you 
have that working the rest is the usual web serving scaling replicating 
yadayada...

Andi..