[Cosmo-dev] Re: [service-dev] Deployment of Cosmo persistence load
bcm at osafoundation.org
Tue Jul 18 08:08:34 PDT 2006
On 7/17/06, Jared Rhine <jared at wordzoo.com> w
> So ok, let's say we've got a service locater in operation; we can look up a
> username and get back what, a server id?
> Is that server id going to be the MySQL database hostname to use? Or the
> Cosmo app server to use?
the database hostname:port, most likely.
> In terms of deployment, I'd really like to have a stateless Cosmo. Though
> we probably need to partition RDBMS data, wouldn't it be great to have an
> army of identical Cosmo servers ready to serve requests for any username at all?
> This of course is greatly helped by a stateless Cosmo, and I admit a chill
> recently when I read you say that part of planned changes (the merge I
> believe?) may require a stateful Cosmo. I didn't fully catch why, though.
i think i might have been a little pessimistic. the issue is one of
concurrent writes to a resource or collection, and how we manage that
situation. the more i think about it, the less i see need for state in
cosmo to deal with it. we probably just need to implement locking and
encourage our clients to use it.
> With a stateful Cosmo, can I have even 2 Cosmo servers serving parallel
> requests for a given set of users (ie, a particular database slice, users
> aa-ag)? Or were you thinking a 1:1 correspondence between database servers
> and app servers?
i'm thinking n cosmo instances per database server. we're going to
have to think a lot more about data partitioning, though, because
eventually cosmo will need to query across the entire user base to
resolve acls, to search shared calendars, etc.
> For Scooby/web-ui sessions, I can likely provide for session stickiness, so
> that subsequent web sessions end up back at the same server. I'm trying to
> think of how session stickiness issues relate with non-Scooby HTTP requests,
> like test/admin scripts, CalDAV, etc. Do I need to stick a layer in the
> front of Cosmo that auto-detects the username incoming and then proxies to
> the correct app server?
no, i think we should be able to get away without host affinity for
non-web requests. and for web requests we still have options. we can
cluster tomcat so that web session state is shared among all cosmo
instances, which guarantees an uninterrupted user experience should a
tomcat instance fail, or we could do host affinity and force the user
to log in again to a new tomcat if his original tomcat fails.
> Best practice I feel is to avoid doing cheesy things like having a
> www14.osaf.us and redirect to the right server if connections end up on the
> wrong server. (I realized recently that redirection works fine for
> browsers, but maybe not so well for CalDAV PUTs etc; haven't tried it). I'd
> like to have an exposed IP address (per data center/cluster), and whatever
> proxying/redirection magic is necessary, happen behind the scenes.
i don't see that being an issue.
> I admit that what I really want in our architecture is separate Scooby
> layers with session persistence, stateless layers of Cosmo servers, and
> service-locater lookups of user data to specific databases. In this model,
> each layer can be sized independently and all layers are fully
> load-balanced. I'm definitely not saying that other models can't be
> attractive, but I do want to ask these types of questions when we start
> talking about user partitioning.
merged scooby/cosmo will work just as well - the cosmo layer can use
clustered web sessions via tomcat and clustered database caches via
ehcache, oscache etc.
> Well, I thought recently that "distributed + transactional = EJB" so a JEE
> solution might be one route. I admit I don't understand fully how JEE would
> implement *distributed* serialization to databases, though the words
> "persistence" are plastered all over the EJB specs. I have no "full" EJB
> deployments in my background; Bobby can say more, probably others.
ejb doesn't mean anything special with regard to architecture anymore.
it's just another persistence api at this point. ejb 3 took its cues
from hibernate and the other popular orm packages and added some sauce
from java 5 - annotations and so forth.
> There might be some Java implementations of some kind of distributed
> writable cache that using non-EJB APIs and is nice and lightweight. If that
> cache layer knew how to distribute its storage, then all Cosmo's could use
> the same distributed write API.
there are several, with different feature sets. you can have caches
local to each app server, caches clustered across all your app
servers, transactional caches, non-transactional caches, etc. all of
which are independent from ejb.
More information about the cosmo-dev