[Dev] RDF tax
uche.ogbuji at fourthought.com
Sun Nov 17 09:29:04 PST 2002
> One lesson I learned while working on Mozilla is that it can be tricky
> to make a scalable data source using RDF, even if the underlying format
> (beneath the RDF representation) is scalable. As an example, consider
> a large mailbox with 50,000 messages. If you force the UI to display
> information through communication with your RDF layer, then you can end
> up with a real challenge on your hands as far as avoiding walking too
> much of the RDF data (especially when hierarchies are involved). This
> problem is compounded by aggregation, where even simple things like
> counting how many items are in a subfolder become challenging (since
> all data sources have to get involved). Trying to do projection in a
> message display becomes very difficult, especially with aggregation.
I want to point out that we did run into some problems on this in large-scale
RDF projects, and we came up with some solutions, even in the case of scaling
to millions of triples.
One is 4Suite scopes. You can narrow a query to a scope, which usually uses
the underlying driver to narrow down the search space accordingly.
ANother is specific optimizations we made to the implementation of the Versa
RDF query language in 4Suite. The 4Suite Versa optimizer does a lot to take
advantage of the back-end facilities for scalable query.
All that having been said, these solutions do not cover all potential RDF
scalability problems, even if we were considering use of 4Suite for the RDF
engine. I think one would have to look at the specifc Chandler needs during
design and put in some analysis. I have some experience in this area, and I'm
happy to lend it.
Uche Ogbuji Fourthought, Inc.
http://uche.ogbuji.net http://4Suite.org http://fourthought.com
Python&XML column: 2. Introducing PyXML - http://www.xml.com/pub/a/2002/09/25/p
The Past, Present and Future of Web Services 1 - http://www.webservices.org/ind
The Past, Present and Future of Web Services 2 - 'http://www.webservices.org/in
Serenity through markup - http://adtmag.com/article.asp?id=6807
Tip: Using generators for XML processing - http://www-106.ibm.com/developerwork
More information about the Dev