[Chandler-dev] Sharing pain points

[Brian Kirsch]

See comments in-line.

-Brian

On Jan 23, 2008, at 5:04 PM, Jeffrey Harris wrote:

> Hi Folks,
>
> Last week Mimi and I sat down and talked about pain points  
> associated with sharing once collections are set up and working.   
> We came up with a list of issues and possible remedies.
>
> The executive summary is that there are still several identified  
> issues causing sharing pain, broadly grouped into:
>
> 1. Failure to sync
> 2. Inaccurate pop-to-now
> 3. Pop-to-now with the byline showing the wrong editor of the item
> 4. Items have conflicts, but the conflicts are in error or not  
> obviously associated with a user-visible attribute
>
> We came up with possible remedies to these problems, based on my  
> understanding of the underlying causes.  It's possible that fixing  
> just the first few problems may get the frequency of pain low  
> enough that we don't need to address all issues.
>
> Problems
> ========
>
> The first problem blocks usage, the rest mainly affect people  
> heavily using shared collections and the dashboard view.  These  
> latter problems aren't show-stoppers by themselves; the occasional  
> confusing item popping to now doesn't make Chandler useless.   
> Unfortunately, the frequency of inaccurate pop-to-nows is high  
> enough to make the NOW section's signal-to-noise ratio much lower  
> than it could be.
>
> 1. Collection fails to sync [1]
> -------------------------------
>
> Esther's been seeing the main example of this [2].  What's  
> happening in her particular case is the desktop and the server have  
> different pictures of what an item's state is, so when the desktop  
> sends a diff, the server rejects it for not containing enough  
> information.  It'd be great to figure out why this is happening,  
> unfortunately we haven't had much luck diagnosing the problem so  
> far, but even if we can't completely stomp out this problem, we  
> should fail more gracefully.
>
> A fix to this could be to have the server's failure include a  
> machine-readable error (right now there's text that contains the  
> failing UUID, if we put that UUID in an invisible XML element, the  
> desktop could try resetting that item and sending the whole item,  
> not a small diff). There'd be a little bit of work on the server  
> side, plus a fair amount of desktop sharing work.
>
> [1] http://chandlerproject.org/Planning/ 
> OneDotZeroBugReset#CriticalBugs
> [2] http://lists.osafoundation.org/pipermail/chandler-users/2008- 
> January/001127.html
>
> 2. Mysterious pop-to-now [3]
> ----------------------------
>
> There are a wide variety of reasons this is happening, it may be OK  
> if we fix 90% of them rather than 99% of them.  With that in mind,  
> my educated guess is that the lion's share of these could be fixed  
> by adding a whitelist to the logic for deciding whether a sharing  
> change should pop an item to now.  Right now, changes to most  
> attributes (even if their values aren't visible to the user in the  
> detail view) will trigger a pop-to-now.
>
> Whitelisting attributes isn't 100%, occasionally it's ambiguous  
> whether a change to an EIM field will cause a user-visible change,  
> but there are many fields that clearly don't directly effect what a  
> user sees in the detail view, these shouldn't be allowed to cause a  
> pop-to-now.
>
> A first cut at this would be fairly simple to implement on the  
> desktop.
>
> There are also issues with Chandler's understanding of email  
> addresses being case-sensitive; it's not clear to me if these  
> issues would be solved by white-listing, there may still be work  
> for Brian Kirsch there.

When the initial sharing work was done, the Email Address logic was  
modeled so that only one EmailAddress Object would exist per email  
address. This made things easy especially for conflict comparison.  
However, it was not ideal for the detail view which needed to  
preserve the various Full Name atrributes associated with an Email  
Address. So the one physical email address to one Email Address  
object metaphor was no longer used. Since then this has created a  
number of headaches as the majority of Email Address comparisons in  
Chandler are done on the FullName and Email Address ie. Brian Kirsch  
<bkirsch at osafoundation.org> and Brian <bkirsch at osafoundation.org> are  
not treated as equal.

To prevent False positive Pop to Nows in the Mail sharing layer I  
added the check:
  if isMeAddress(peer):
    #ignore

Where isMeAddress just looks at the EmailAddress.emailAddress values  
case insensitively for compare with all 'me' email addresses.

This same logic needs to be employed through out the sharing layer.  
All email conflict checks should be determined solely on the  
EmailAddress.emailAddress attributes.

I am happy to help with this. Although I am not sure which modules I  
need to edit to prevent the conflict.


>
> [3] http://chandlerproject.org/Planning/ 
> OneDotZeroBugReset#PopToNowBugs
>
> 3. Inaccurate byline
> --------------------
>
> There are two main sources of inaccurate bylines I'm aware of: The  
> first happens when two users have non-conflicting changes, but they  
> sync out-of-order.  The byline is chosen based on the latest  
> change, but the change time is based on when the user actually  
> changed the item in Chandler, not when they synced.  So out-of- 
> order syncs will cause the byline to stay the same, even though a  
> new change has been merged in, which can be disconcerting for the  
> person who synced first.
>
> A solution to this would be to add additional fields to the  
> ModifiedBy record, sending two users after a merge, and rendering  
> an appropriate message in this case.  This is quite  
> straightforward, but it might require a bit of server work.
>
> The second main cause of in accurate byline happens when an item is  
> deleted.  The server stores a "tombstone", recognizing that the  
> item's been deleted, but it doesn't know *who* deleted it.  This  
> isn't a huge problem for normal items (although it's a little  
> annoying).  For recurring events, though, deletion actually means a  
> triage status change.  In these cases, it's pretty confusing to see  
> an occurrence pop to now with nothing but a triage status change  
> and an inaccurate byline.
>
> One solution to this would be to change morse code to include  
> information about who's deleting an item, and having the server  
> store that alongside the existing tombstone.  That might be a lot  
> of work, though.
>
> The recurrence problem could be fixed by changing the way triage- 
> only modifications are sent to the server.  Unfortunately, changing  
> this would either break compatibility with existing data, or  
> require a huge amount of effort to maintain backward compatibility,  
> so I think that's out.  This may just be a problem we live with.
>
> 4. Spurious conflicts
> ---------------------
>
> Conflict resolution aims at preventing users from losing data.  
> Unfortunately, there are so many false positives that I literally  
> had never seen a *real* conflict until collaborating on writing  
> this document!  All the other hundreds of conflicts I've seen have  
> been spurious, adding noise to my Chandler use and forcing me to  
> examine and click through too many conflict resolution dialogs.
>
> Many spurious conflicts appear to come from having the same item  
> shared in multiple collections.  When a user changes a multiply  
> shared item locally and syncs, the first associated collection  
> receives the item change.  The second collection then sends the  
> same change back.  This ought to result in a no-op, but apparently  
> it doesn't always (that's a bug in and of itself).
>
> While ideally we'd fix the underlying bugs that are causing  
> spurious conflicts, it's not clear to me how much effort it would  
> take to solve the problem.
>
> A radical, alternate approach that might reduce this pain, while  
> still providing data preservation would be to
>
> A) Store item state before every sync, and after every sync (we  
> mostly do this now)
> B) Provide an HTML picture of the history of an item, similar to  
> the view currently available by pressing F4 but focused on  
> reproducing only the attributes visible in the detail view
> C) Auto-resolve *all* conflicts, with the more recent change  
> winning, then use the banner button currently associated with  
> conflicts to alert users if there was a conflict on a user-visible  
> field
>
> Implementing this strategy would be a moderate to a large amount of  
> work, so it's not clear if it's worth doing, even if we agree it's  
> desirable.  On the plus side, a clean-UI for the history of an item  
> is an attractive feature in it's own right, so such an approach  
> could be implemented incrementally.
>
> Conclusion
> ==========
>
> I prioritize these roughly in the order described.  Recovering from  
> sync failures and whitelisting user-facing attributes seem  
> relatively straightforward with a high value.  Expanding the  
> ModifiedBy record to work better with merges seems lower priority  
> but simple and valuable. Morse Code changes to deleted items,  
> viewing item history and auto-resolving conflicts seem potentially  
> valuable but possibly expensive.
>
> Sincerely,
> Jeffrey
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev