[Design] Visualizing data, Grokking data

Seth Johnson seth.johnson at RealMeasures.dyndns.org
Wed Feb 15 23:15:25 PST 2006

Mimi Yin wrote:
> Hi Seth,
> I have a feeling what you're describing is highly relevant to this
> issue, but I'm having a little trouble understanding your proposal.
> Could I ask you to define some terminology you use and put them in
> the context of specific usage examples?
> On Jan 31, 2006, at 2:21 PM, Seth Johnson wrote:
> > Yes, I did go through and try to come up with something to say,
> > but didn't have anything in particular.  I don't have a head for
> > visual design so much as for data architecture -- and these days
> > I "see everything as a nail," if you know what I mean.  I do have
> > some more notions about how attributes and categories that have
> > "scopes of relevance" translate into useful ways to categorize
> > data and retrieve it.  I'm pretty sure these notions would help
> > for understanding how to get that high level view that you're
> > looking for, but I don't know exactly how that works out in
> > interface terms.
> What do you mean by Scopes of relevance wrt attributes?

See below, I think you start to get it . . .

> > Assume all "entities" and "relations" are contained in one core
> > data structure with use types, link types, uses and links that
> > are definable by users.  You indicate that Chandler's data model
> > is like this.
> What constitutes an entity versus a relation? Use types, link types,
> etc.

You can think, for the sake of modeling a real world
circumstance, in terms of entities in the form of flat files,
related in the traditional way into a whole slew of entities and
their relations.

On the other hand, you can take the basic notion of a relation
and expand it to a more generic notion of a relation, which I
call a context, and put all entities into that physical data
structure.  This more generic version of any relation among
entities refers to the parent entity in terms of uses and use
types, and the child entity in terms of links and link types. 
Then both the parent and child "entities" -- actually, the use
types, link types, uses and links -- are virtualized such that
they are not actually records in their own flat files holding
attribute values, but pointers (which is why the generic term for
the child "records" is "link").

So an entity is either a pair of labels assigned by a user to
describe a use type and use, or a pair of labels assigned to
describe a link type and link(s).

A relation in the sense of a context is defined generically as a
use type and a link type brought together.  A particular instance
of the parent entity -- a "use" of the "use type" -- is the
equivalent of a "record" in a flat file entity table.  Likewise a
"link" of the "link type" is equivalent to a "record" of the
child entity.

> > Let's suppose that you can also assign certain scopes of
> > relevance to attributes ("named labels" that can hold values
> > associated with a particular "record" -- like "fields").  So you
> > can have attributes for "child records" (links) that are relevant
> > (available for capturing appropriate values) to:
> >
> > any context with x use type
> > any context with y link type
> > any context with x use type and y link type
> > any context with z use of x use type
> > any context with z use of x use type with y link type
> >    (that is, the attribute would therefore only be relevant
> >     to all links in this particular instance of a generic
> >     context)
> > v link when used in any context (with any use type, link type or
> > use)
> > v link when used in any context with y link type
> > Etc.
> >
> > I also apply the same scopes of relevance to categories (I think
> > these are basically like Chandler tags).  Basically, named labels
> > you declare that can be marked yes or no to indicate whether the
> > label applies to a "record."  Then suppose you could declare that
> > every time you work on a link (a "child record") in a context
> > that fits a certain cateogory's scope of relevance, you will be
> > prompted to indicate whether that category applies to that link
> > ("record").
> >
> Is this a proposal for determining whether a particular attribute
> applies to an item based on the Kind of that item (ie. All emails
> have a From field).

Pretty much.  But more generically.  You can say a "record" that
starts as an email (Link type = email; link = message ID) has
attributes that are relevant:

- Wherever it is used, so long as it's still treated as an email
- Wherever it is used, even if it is no longer treated as an
- when it is used as an email with an "Email Folder" use type 
- when it is used as an email with the "OSAF" folder (particular
use) of the "Email Folder" use type
- when it is used as an email with the "OSAF" use, regardless of
use type (Not just in an "Email Folder")
- when it is used not as an email, in the above contexts.

These are "scopes of relevance" for attributes and categories,
defined by making use of the generic notions of use types, uses,
link types and links.

> > My impression is that a process of designating categories and
> > attributes that have scopes of relevance like this:
> >
> >   1) will foster people increasingly using common attributes --
> > and attributes that aren't really specific to flat file
> > "records," but to any links in any contexts that fall within
> > their scopes of relevance; and then
> I think you've lost me at 'aren't really specific to flat file
> "records"'.

You can say "here is a flat file for storing emails" and that
might be represented as a flat file visually in some interface
where you design an application with an elaborate
entity-relationship diagram.  Another "flat file" might be
represented holding "records" representing email folders.

On top of that, a designer will create an interface, likely to
look like an outline of folders.  And that's the way a user might
want to think about it while working with the information in that

But when backing out and looking at how data is used across
applications, you don't speak in terms of those two entities, but
rather in terms of how they're used.  The generalization I use to
speak generically across all applications, is that of contexts
defined by use types, link types and their uses and links.  Those
are the general notions to have in mind.  In interface terms, I
think I'm saying, give people access to working with data in
those terms when their purposes go wider than particular
applications.  They understand all applications as at heart,
particular contexts, defined as a use type and a link type
brought together.

For example, each branch of an outline could have its own use
type and link type designatable by the user.

What I'm saying in the message you responded to is that when
people see applications as comprised of these generic notions
(and then, going deeper, with attributes that have scopes of
relevance), they will begin to use common attributes.  Some
elements they choose to use in new contexts will come with their
own attributes, that are relevant or not based on how they're
used.  Some of these elements will have the right attributes, and
some won't.  So they will begin to pick the right ones, and add
attributes as needed.

And then these generic notions, plus this encouragement of reuse
of common attributes, will facilitate designing tools that let
you survey across all applications.


> Thx! Mimi
> >
> >   2) will foster the development of tools that let you survey
> > what contexts are using what attributes and categories.
> >
> > I think that 2) may lead the way to the "10,000 foot, eagle-eye,
> > contextualized view of our [heterogeneous] data" that you're
> > looking for.
> >
> >
> > Seth
> >
> >
> >
> >> On Jan 30, 2006, at 5:45 PM, Seth Johnson wrote:
> >>
> >>>
> >>> Mimi Yin wrote:
> >>>>
> >>>> Yes, what you're describing is Chandler's data model which is then
> >>>> enriched with a content model that understands about specific kinds
> >>>> of user data and specific types of relationships between items.
> >>>>
> >>>> The proposed topic of this thread was to explore ways of
> >>>> visualizing
> >>>> this data/content model in a way that is easily comprehensible for
> >>>> both personal use as well as collaboration.
> >>>>
> >>>>> You can import information of all sorts into the highly generic
> >>>>> framework of relations described as use types, linktypes, uses,
> >>>>> links and their attributes.  Then all of them are intrinsically
> >>>>> interoperable.  And my point is that these generic notions can
> >>>>> provide the conceptual backbone for the interface, a framework
> >>>>> that is easily understandable, which you make accessible to the
> >>>>> user within any interface you build.
> >>>>
> >>>> It feels like there are a couple of threads running through your
> >>>> comments below, let me see if I can address them individually
> >>>> (please
> >>>> correct me if I misrepresent what you said):
> >>>> + I agree that our approach is to focus on end-user
> >>>> comprehension of
> >>>> data visualizations.
> >>>> + I agree that this approach is not at odds with ideas about mind-
> >>>> mapping. Instead it is intended to improve on mind-mapping UIs.
> >>>> + The generic framework you describe exists today in Chandler's
> >>>> data
> >>>> model (see comments above). I think I was a little confused by your
> >>>> comments because that wasn't the proposed topic of this thread.
> >>>> + In your last point, are you saying that mind-mapping IS easily
> >>>> comprehensible without further improvements? This would be
> >>>> interesting to explore further in relation to the design principles
> >>>> laid down in the wiki write-ups.
> >>>
> >>>
> >>> A better response than my previous reply:
> >>>
> >>> When I say:
> >>>
> >>>>> I mean the way you're thinking is in terms of trying to find some
> >>>>> functionality that the app will provide, that will do the
> >>>>> simplifying for grokking that you seek.  I suggest that thinking
> >>>>> this way foreshortens:
> >>> < SNIP >
> >>>>>   3) for mindmapping again, seeing that mindmaps are great for
> >>>>> grokking on the basis of seeing nodes' grouping and arranging and
> >>>>> interpreting that as a human, rather than the app understanding
> >>>>> the grouping and arrangement.
> >>>
> >>>
> >>> . . . I mean yes, it IS easily comprehensible.  I also suggest
> >>> some further improvements, specifically to add the semantics you
> >>> point out are missing.  Since the page in question is about
> >>> chunking over time, I'll beg off the visual interface design
> >>> question for now and continue to point out some things about data
> >>> architecture with respect to genericity, grokkability and
> >>> mindmapping.  I just added the following comments to the page
> >>> (http://wiki.osafoundation.org/bin/view/Journal/ChunkingOverTime):
> >>>
> >>> <Pasted from wiki>
> >>>
> >>> What I think I'm saying is that you can have a generic data
> >>> structure and also have semantics. Mindmapping software does not
> >>> currently have this, but there's no reason why it can't be added.
> >>> The "IsRelated" generic relation can be expanded to terms that
> >>> allow users to apply specific semantics to specific relations.
> >>> Instead of:
> >>>
> >>> "this node is related to these children"
> >>>
> >>> we have:
> >>>
> >>> "this particular use (Make a birdhouse) of this particular use
> >>> type (Woodworking project) has these particular links (get some
> >>> wood, measure it, cut it, hammer it together) of this particular
> >>> link type (Project steps)"
> >>>
> >>> When people begin sharing contexts defined by these abstraction
> >>> they will begin using common use types and link types, thereby
> >>> allowing chunking over time that's interoperable. One user will
> >>> be able to chunk over time any way she pleases.
> >>>
> >>> Mindmapping in which each branch can be designated with specific
> >>> use types and link types, making them each unique relations,
> >>> would let you play with information and also begin to make
> >>> information usable across particular contexts.
> >>>
> >>> Add to that the ability to visually represent specific such
> >>> combinations of use types and link types in their own assigned
> >>> default ways. Then make the visual interface (whether mindmap,
> >>> email folder, Gantt chart or any other visual interface) provide
> >>> access to the use types and link types of the relations.
> >>>
> >>> This doesn't answer the design question of how to visually
> >>> represent such disparate types of relations, whether all at once
> >>> in one interface or in separate ones, but I hope that it does
> >>> give an idea how you can have completely generic relations and
> >>> semantics at the same time.
> >>>
> >>> Now, regarding mindmapping -- this interface gives users the
> >>> ability to arrange things visually to see relationships by
> >>> grouping and to work with and access information that's in flux
> >>> as far as how it will ultimately be organized, even if to the
> >>> application there are no special semantics to the relations being
> >>> created. Now, to add semantics as I describe above and below,
> >>> would only augment the traditional mindmapping concept and
> >>> interface, as well as all other interfaces. Plus provide the data
> >>> structure for chunking.
> >>>
> >>> One note: taking chunking as a function of our human ability to
> >>> comprehend visual information, we can observe that mindmapping
> >>> naturally leads to that, though without semantics in the
> >>> traditional form.
> >>>
> >>> </Pasted from wiki>
> >>>
> >>>
> >>> Seth
> >>>
> >>>
> >>>
> >>>
> >>>>> I mean the way you're thinking is in terms of trying to find some
> >>>>> functionality that the app will provide, that will do the
> >>>>> simplifying for grokking that you seek.  I suggest that thinking
> >>>>> this way foreshortens:
> >>>>>   1) for mindmapping, seeing that it's the particular kind of
> >>>>> functionality it provides that makes it valuable, moreso than the
> >>>>> criterion of how well some functionality intended to simplify
> >>>>> grokking, meshes with mindmapping functionality;
> >>>>>   2) thinking about how more genericity in conceptualizing
> >>>>> information, not some function, might give you a better framework
> >>>>> for getting to better grokkability; and
> >>>>>   3) for mindmapping again, seeing that mindmaps are great for
> >>>>> grokking on the basis of seeing nodes' grouping and arranging and
> >>>>> interpreting that as a human, rather than the app understanding
> >>>>> the grouping and arrangement.


RIAA is the RISK!  Our NET is P2P!

DRM is Theft!  We are the Stakeholders!

New Yorkers for Fair Use

[CC] Counter-copyright: http://realmeasures.dyndns.org/cc

I reserve no rights restricting copying, modification or
distribution of this incidentally recorded communication. 
Original authorship should be attributed reasonably, but only so
far as such an expectation might hold for usual practice in
ordinary social discourse to which one holds no claim of
exclusive rights.

More information about the Design mailing list