[Chandler-dev] [Sum] July 17 - 23

Katie Capps Parlante capps at osafoundation.org
Mon Jul 31 14:33:35 PDT 2006

Release, Build and QA
*Alpha3 readme*
Sheila posted a draft of the Alpha3 Readme and asked for comments. Ted
noted the licence change to Apache 2.0. Dan noted the startup management

*Alpha3 schedule*
We had 5 serious bugs early in the week, and were looking at a release
candidate late in the week. Philippe is taking over release manager role
through the rest of the release. By the end of the week, we still had 2
tricky, time consuming bugs. We decided to wait for these bugs, reducing
the number of release candidates created and tested.

*Alpha/Beta Release Planning*
Sheila gave some context about how Alpha4 fits into the roadmap for
Chandler. We have two significant milestones: Alpha (proof of concept
for 1.0, dogfoodable inside of OSAF) and Beta (usable outside of OSAF,
dogfoodable for 1.0). We're adjusting our 0.7 release to become a series
of alpha milestones to hit the first "beta" release. The working plan is
to be able to reach "beta" in March 2007, with 2 month alpha milestones:
alpha4 (Sept), alpha5 (Nov), alpha6 (Jan). With the alpha releases,
we're building up to the point where we can be internally dogfooding
some key features (stamping and dashboard scenarios). The primary tenets
for Alpha4: (1) first iteration of the dashboard, triage and stamping
workflows (2) enhance usability of the calendar with visual and
performance improvements (3) support collaboration workflows by tackling
the sharing format design and making it easier for users to "share their
world" instead of having to manage sharing collection by collection.
Sheila gave links to plans and specs on the wiki.

*Functional Tests*
Travis found the ChandlerAutomatedTestSystem wiki page, and asked for
more information about functional tests. Dan explained that we are
moving to a slightly different framework (CATS 0.2), and gave the
current links to the docs. Dan mentioned that looking at existing tests
as examples is the easiest way to learn how to write them; he gave
pointers to the tests in svn.

*Debugging Chandler*
John checked in pre-built Wing projects (debug and release). He's
documented minor snags and included instructions on how to debug
functional tests. Bear updated the manifests, so they should show up in
the continuous builds.

Miscellaneous Projects
*Startup Management Dialog*
Reid wondered if the key held down to get the startup dialog should be
"option" on the mac instead of "control". Bryan noted that he chose the
control key on purpose to be the same across all platforms -- that way
we don't have to figure out what platform someone is on to give them
instructions. He'd like to keep it consistent unless ctrl-at-startup has
some conflicting use/meaning.

Referring back to a thread about reminders while Chandler is not
running, Jeffrey liked the option of having two separate processes that
both communicate with the repository. Using a flat file may be the
cheapest way to get a reminder widget up and running, so that would be a
reasonable alternative place to start. A small background process could
offer other features: (1) hotkey that pops up a preview area, (2)
quick-entry widget, (3) run appropriate server to accept quick entries
from XML-RPC. Given that we've invested in handling merges well, we
could take advantage of it. Grant is also inclined to take this approach
-- he thinks he could write the periodic repository refresh code faster
than the flat-file approach.

*Thunderbird Extension Proposal*
Chris Haumesser proposed extending the stamping functionality to some
well known, cross-platform, open source mail client (aka Thunderbird).
We could write a simple Thunderbird extension to add a toolbar button to
"stamp" a message as a Chandler calendar event or to-do list item. The
extension UI could be very simple, perhaps a username/password dialog, a
URL field, a couple of toolbar buttons. Foxmarks might be a good
starting point. When a user "stamps" a message, the extension could add
a few custom headers to identify the content as a Chandler calendar
item/to do item/etc. The extension could send the modified message to
Chandler/Cosmo through some well-established protocol (SMTP or IMAP).
The item would then appear in an "incoming" section of the dashboard. In
addition to supporting our end-user triage workflow, this approach could
have community benefits by opening up a bridge to the Mozilla community.
The fast growth of the Foxmarks project is a good example of the
potential of Mozilla extensions. A plausible Thunderbird extension could
also inspire developers to extend other mail clients (Evolution, Apple
Mail, etc.).

Chris went on to answer some of Mimi's questions from the design list.
An extension is "deeper" than just adding a toolbar, as it extends the
functionality of the client. We'd have to write code to add the custom
mail headers, etc. Questions/issues: Cosmo to understand email?
Receiving end to recognize "stamped" email? Standard set of custom
headers? Chandler/Cosmo to parse headers? Send data to Cosmo or
Chandler? IMAP or SMTP?

Travis explained that Chandler's IMAP server runs with standard
settings. A "quick setup" button could point an IMAP account to
Chandler. When a mail item is "stamped" in Thunderbird, it could (a) add
the stamping information in custom email headers to the underlying
representation used by Thunderbird -- tricky with IMAP and possible with
POP (b) add headers and send mail to Chandler in one step -- making it
tricky to stamp something more than once. When a message is sent to an
IMAP server (drag-n-drop to the server's folder in Tbird or "send to
chandler" button), this information needs to be included as header
information. This might be a hard problem in Tbird, depending on what
kind of access to internals/message representation is available to the
extension framework. When messages with special headers are received in
Chandler, Chandler should know what to do with them; Chandler's mail
handling mechanisms need to handle them. Travis suggested this might
make a great side project for someone with a bit of Thunderbird experience.

Andi added a new type of ref value: dict of ref collections. More info
about this coming soon in a chandlerdb blog post. Andi refactored
watchers using this new type; they are now better at cleaning themselves
up when the watcher item or watched item disappear. Andi added support
for "free values", which may end up being used in the ad-hoc attribute
implementation. Any item may now refer to any other item without extra
schema definition via new APIs: setFreeValue(), addFreeValue(),
getFreeValue(), removeFreeValue().

In another round of repository changes, the queued collection
notifications now use watchers. notificationQueueSubscribe() was renamed
watchCollectionQueue(). You can also name the method that gets invoked
for these notifications. Andi removed support for persistent queued
collection notification subscriptions, simplifying the code. (They were
unused except in a unit test).

Andi added support for undoing the latest commits in a repository, as an
experimental feature. It can be triggered by the --undo <toVersion>
option at startup time. Perhaps Bryan could add another option to the
Ctrol startup dialog? Undo can be used to get rid of versions that are
corrupt due to bugs (indexes getting out of whack, for example). One
approach could be to undo until check() passes. This might be a better
alternative to throwing out an entire repository.

*Committing during refresh*
Travis ran into a StaleItemError, triggered by a repository commit
during a notification callback. (Travis ran into this problem when
stamping an Item as a MailMessage from the UI -- the Item needs to be
placed in a 'directory' Item to be noticed by the IMAP server). Travis
created a simple test to reproduce the problem. Grant tracked the
problem down, noting that Travis' commit() during the refresh() seems to
get CPIA in a weird state, exposing some dependency on the order of
notification processing.

Andi noted that doing a commit() inside a refresh() might be the
problem. He explained that at the end of a refresh(), notifications are
sent out, including asynchronous notifications from the collection watch
queue (used by CPIA). Andi asked if he should add support for delaying
calls to commit() inside refresh() until refresh() is done. After
talking more with Bryan Stearns, Travis figured out how to implement his
feature without calling commit() from refresh(). Bryan explained that he
tries to keep notification handlers as simple as possible to avoid
notification cascades. Andi used Travis' reproduction case to fix the
commit-during-refresh problem -- the repository now defers committing
until the refresh completes.

*When to Commit?*
Bryan and Travis brought up a related issue; Travis wanted to add a
commit() after stamping. Bryan noted that he had removed it for
performance reasons. Even without the commit, the stamping test is 200%
of the target time. Grant asked when the UI commits changes in general.
He noted that committing after every user operation has a reliability
advantage: less chance of data loss if the app or machine crashes, and
data is reasonably up to date in background sync operations. In Travis'
case he needs the commit for a stamped MailMessage to show up in the
main thread.

*Help with broken 0.6.1 repository*
Xun restarted Chandler before he had finished shutting down another
instance, leading to a situation where he can't start up Chandler with
the existing repository. He has been collecting slashdot articles for
his project (tagging them, etc.), and would appreciate help in
recovering them. Bear tried running Chandler against Xun's repository,
noting that it stalled at "rendering main view". He then tried running
headless.py, where he was able to open the repository successfully. He
ran view.check() without error, indicating that the repository is OK.
This likely means that something is wrong with the UI, preventing the
app from starting up. Bear included a fixed-up RunPython.bat, so that
Xun can run in headless mode to walk the data and perhaps export it.
Morgen may be of further help.

Domain Model
*Stamping as annotation*
Grant has a pass at stamping-as-annotation in his sandbox. Issue 1: How
to do a ref collection of all items with a particular stamp? Currently
one defines a str (__collection_name__) in the annotation subclass
defining where to put the ListCollection. Grant overrides
_create_schema_item() to install the collection. PJE suggests
subclassing schema.AnnotationItem and putting the collection there (he
gave sample code). Issue 2: The implementation of Stamp.delete() goes
through and del's the Annotation's attributes. PJE thought removing a
stamp was supposed to keep the attributes intact. If not, it could
override remove() and delete them before a super call to the original
remove(). As written, it's impossible for a stamp to retain values after
removal. Grant explained that he was trying to avoid leaving
dangling/unreachable items in the repository. The various alternative
behaviors (leaving attributes around for restamping, deleting
attributes, or having stamps delete selected attributes) should all be
implementable. Issue 3: the recurrence implementation declares
attributes that refer back to CalendarEventMixin. Can EventStamp refer
back to itself (or to other Stamps) using the schema API, even if
Annotation/Stamp classes aren't persisted directly by the repository?
PJE thinks it should, if not let him know. Grant will need to go through
the recurrence code that makes use of occurrences and add the
EventStamp(), otherwise the code will be operating on the base items and
not the stamp wrappers.

*displayName changes*
Brian Kirsch had a patch ready for Bug 1745, and asked for review. This
opened up another discussion about displayName/title. Morgen checked in
code to keep Chandler backwards-compatible w/sharing for the displayName
--> title change for collections.

Phillip Eby summarized the displayName discussion (from irc and a
meeting), listing agreements and next actions. Brian is concerned about
asking developers to localize some displayNames and not others, as most
are never actually displayed (one reason for the new "title" attribute).
In practice, we don't use displayName as a centralized, universal
attribute for an item's user-visible name (the original intent). As we
don't really have a need for displayName at all, we agreed to remove it
from the schema API and the repository and to use application specific
attributes for UI display (e.g. "ContentItem.title"). Plan: (1) John
will go through uses of displayName in UI and replace them with
ContentItem.title or some other appropriate attribute. "title" should be
indexed. (2) Brian K will review the patch for i18n issues. (3) PJE will
remove displayName from the schema API. (4) Andi will remove
displayName/displayAttribute from the repository code and core schema

Intern and SoC projects
*Date/time parsing*
Darshana's work on date/time parsing in the detail view is being tracked
in Bug 6249. She has some code working, not yet ready to be checked in.
Code and Jeffrey's review are posted in the bug.

Bear asked for help naming his date parsing library, thread starts here:

*Log Calendar Project?*
Markku explained the "Morgen trick", where a calendar collection is used
as a logger to monitor background sync. Chandler can be used to browse
the data from the calendar or table view, with details of each sync in
the detail view. Markku wondered if this approach could be used in other
places, such as for feeds. He asked if we could have one global
"logging" calendar to record in/out transactions. He noted that we could
make queries to the log calendar, as the data is stored in the
repository. For example, Markku could ask for all feedsparcel log
entries in a given datetime range. Questions arise: (1) we don't want to
bloat the repository (2) we wouldn't be trying to replace the existing
loggers (3) is this feasible (4) anyone else interested in this as a
service? For (1) and (2) it would make sense to be selective about what
was being logged; Markku suggested only information that is useful for
the end users but not beyond that.

*IMAP server*
In working on his project, Travis is thinking about adjustments and
improvements to the mail schema. He's keeping track of his thoughts on a
wiki page. In particular, he has a proposal to change MIME handling from
a first-class Item, making use of annotations instead. Travis asked for
Brian Kirsch's input. In general, Brian is in agreement that the logic
for converting a specific item type to MIME should move to the Item
class. Brian raised the concern of multiple representations of the same
data bloating the repository. He also noted reply and forward as
requirements that we need to support; we need to maintain the original
'sent' message (i.e. the rfc2882 blob currently used). He noted that
we're getting a more clear picture of the design requirements, which
should help.

Ernesto has a preliminary version of contacts with primitive vCard
support working. He's committing his progress, and has instructions for
testing on the project wiki page. You can try importing .vcf files.
Ernesto listed some known limitations, which he is working on.

Meetings and announcements
Philippe posted his and Brian's i18n slides for EuroPython.

Apps team meeting:

More information about the chandler-dev mailing list