[Chandler-dev] [Sum] April 16 - 22

Tue Jun 19 15:03:04 PDT 2007

Build, Release and QA
---------------------
*Checkpoint*
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008108.html

*PPD triage*
Sheila and Mimi went through all of the alpha5 and 0.7release desktop 
bugs, prioritizing them for Preview. They put them in buckets:
P1: Bugs that prevent users from trying out the app or particular features
P2: Bugs that block dogfooders
P3: Bugs that are actively confusing people
P4: Bug fixes that would make it more likely for people to stick with 
Chandler (nice to have)
http://wiki.osafoundation.org/Journal/ChandlerBugTriage20070328
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008128.html

*Intermittent test failure*
Morgen observed that PerfLargeDataScrollTable fails or succeeds based on 
the time of day (as events cycle in and out of various triage sections).
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008133.html

Performance
-----------
*Notifications and redrawing*
John was exploring improvements to notification handling to eliminate
unnecessary redrawing. (1) Don't draw if a change doesn't affect the
widget. (2) Make a small change to a widget instead of recreating the
widget. Example: if an item is removed from the table, delete the single
row instead of recreating the table. To pursue the example, John needs
the deleted item's index. He asked Andi if that information could be
included in the notification.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008095.html

Andi explained that he doesn't have the old index location readily
available. He noted that indexes are versioned, and that one could ask
the index location given a UUID (but performance of such an operation
should be considered). John thought keeping a widget up to date should
be similar to keeping a selection up to date.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008098.html

Andi also mentioned keeping information about the "page" being displayed
at the UI level, tracking the position of the items having changed there.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008097.html

*Performance regressions*
Heikki kicked off a thread questioning the value of tracking performance
regressions. He was working under the assumption that tracking down
performance regressions was valuable, a first path to pursue that is
easier than regular performance work. He noted that other developers
were questioning the perf regression bugs he logs, and wondered if he
should continue to do so.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008099.html

Andi felt that logging performance regression bugs when (expected) 
slowdowns occur do to feature additions or bug fixes was unhelpful. He 
wanted to see regression bugs if a test became slower as an (unexpected) 
side effect of some other change.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008100.html

Heikki found this criteria confusing: all checkins are bug fixes or 
feature additions. Heikki won't always know what slowdowns are expected, 
he logs bugs as a task/reminder for someone to go check it out. He 
checks the graphs once a day, and doesn't log bugs for temporary spikes.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008102.html

Bryan felt that logging performance regression bugs as new features land 
was not especially productive and somewhat demoralizing. Performance 
analysis works better after the feature chain is in (not commit by 
commit); backing out the new feature isn't the productive path forward. 
Bryan also noted that the measurements vary widely, so the graphs cover 
too short a period to reliably see the result of a commit. The graphs 
indicate areas where investigation is necessary, but Bryan focuses on 
his profiles and doesn't rely too much on differences in the graphs per 
commit. Bryan also asked that we make green the "acceptable" color 
instead of orange.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008101.html

Heikki explained that he worked on another project where the expectation 
was that no new feature is committed if it jeopardized performance -- 
some projects do operate that way. He's not proposing that for Chandler 
at this stage. He acknowledged some of the measurement problems that 
Bryan brought up; improving them hasn't been the top priority but we 
might want to tackle them later.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008104.html

Philippe wrapped up the thread with some important points:
- Performance is everyone's problem. We added a 3 weeks period to the 
schedule to allow everyone to focus on performance, after feature complete.
- "Baseline performance" and "performance regression" are two different 
things. Before we can worry about regressions, we need to hit an 
acceptable baseline.
- We're shooting for "acceptable" right now. Orange is good, orange is 
the current goal.
- We're worried about any case that is not "acceptable" -- we care more 
about whether or not we're in that goal range than whether or not it is 
faster or slower than previous runs.
- Once we've hit the baseline or acceptable values, we'll need to be 
tracking regressions. Regression tracking is important, if we don't 
monitor performance and pay attention to it, performance will fall behind.
- Bugzilla is our tool to track issues. A marked performance drop is a 
real "incident" that needs tracking; Bugzilla is an appropriate tool to 
use for this kind of tracking.
Philippe asked that we stop automatically logging performance 
regressions until we hit baseline numbers.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008118.html

*Quick item entry test*
Heikki wrote a performance test for quick item entry. The operation 
ended up being significantly faster than creating an event by double 
clicking -- we don't select the event and display it in the detail view. 
This case should get faster as other similar cases get faster. Heikki 
decided not to add the test to the Tinderbox suite, as it doesn't yield 
new/different information.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008110.html

*Performance tools poll*
Heikki sent out a poll asking questions about performance tracking 
graphs and tools. A couple people questioned the statistical 
value/correctness of the standard deviation calculations. Read the 
thread for more detailed info on how people use rt:
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008112.html

*Performance test variability*
Heikki looked at variability of performance test results, and asked at 
what threshold the results became meaningless.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008122.html

Heikki turned off real time virus protection on the performance 
Tinderboxes to reduce variability.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008129.html

*Minor perf tip*
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008127.html

Dev
---
*Reload*
Heikki wrapped up an earlier thread about reload:
- File a bug to have File-->Reload restart Chandler with --reload
- File an enhancement request for a feature that dumps/reloads a 
particular collection (like import/export)
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008113.html

Morgen planned on adding a menu to Tools allowing a reload without 
blowing away current data, as he finds it useful for testing.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008116.html

Meetings, Announcements
-----------------------
Desktop team meeting (both apps and platform groups):
http://wiki.osafoundation.org/Journal/AppsMeeting20070419

Desktop release meeting:
http://wiki.osafoundation.org/Journal/EngineeringMeetingNotes20070419

Aparna explained that the QA team will schedule additional informal 
testing sessions on Fridays at 1pm PDT. They'll be conducted in the 
#osaf-qa channel.
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008114.html

Migration testing QA session
http://wiki.osafoundation.org/Journal/CosmoZeroSixOneMigrationTestPlan
http://lists.osafoundation.org/pipermail/chandler-dev/2007-April/008124.html