[Chandler-dev] Chandler Mail Service Update

[Pieter Hartsook]

Well I just downloaded Chandler_iosx_0.7.dev-r14394-20070524014347.app
and turned it loose on my Inbox with 12,552 messages.

I must say, I like the frequent status message updates (searching in
350-message blocks) BTW it took about 3 min. to complete the Inbox
search, and the downloading was very fast and the status messages
there were very informative too.

Great improvement! Thanks Brian.

Pieter

On 5/24/07, Philippe Bossut <pbossut at osafoundation.org> wrote:
> Whohoo!! That's fantastic news! Haven't tried yet though... :)
> Cheers,
> - Philippe
>
> Brian Kirsch wrote:
> > Hello,
> > As of revision 14388, the Chandler Mail Service has under gone a
> > significant architectural change
> > to support downloading of large Inboxes (12,000 plus messages). To
> > accomplish this the
> > Incoming Mail logic has been refactored in to two asynchronous threads
> > the Twisted Mail Client thread and the Mail Worker thread. The Twisted
> > Mail Client thread handles all downloading and pre-processing of mail
> > (building the body text, finding eimml / ics attachments etc). The
> > Mail Worker thread is responsible for taking the pre-processed mail
> > off of a queue and
> > assigning the values to a MailStamped Item. There are now only two
> > views in play.
> > The Twisted Mail Client view shared by all downloading clients and the
> > Mail Worker view used to convert pre-processed mail from all Clients.
> > With this
> > check in performance has greatly increased, timeouts have been
> > minimized, and
> > a pesky bus error that occurs when too many Twisted deferreds are on
> > the Python
> > stack has been resolved.
> >
> >
> > Taken from the r14388 commit message:
> > ======================================
> > Performance Performance Performance! Complete refactoring of Incoming
> > Mail Logic
> > to a Mail Worker thread with one view and all clients and Mail Service
> > sharing a second view.
> > The Mail Service can now download 12,000 messages with No timeouts and
> > Bus Errors.
> > There are still performance issues in the UI and Repo layer to tackle
> > related to merging,
> > refresh, and observer calls. But much improvement has been already
> > made. Fixed the Bus Error related to too many Twisted deferreds over
> > flowing the stack as well as added dynamic
> > commit calculation and throttling. Improved POP performance 2x when
> > the POP3 Server does not support the TOP command (gmail).Added IMAP
> > Body.PEEK commands to prevent mail on the IMAP server from being
> > marked as seen on download. Added Blocking vs. Non-Blocking logic.
> > The Incoming Mail Clients can either wait till the Mail Worker has
> > processed the last commit request or continue downloading mail while
> > the Mail Worker is converting Email Objects to Items.
> > Added Preview specific Message Parsing and Conversion to Item methods.
> > These are intended to maximize RFC2882 text to MailStamped Items
> > performance. Improved status bar messages when searching for Chandler
> > Headers and when downloading and committing mail. In addition fixed a
> > number of open Preview mail bugs.
> >
> >
> > Please not the Chandler Schema has changed as of rev 14388.
> >
> > Enjoy,
> > Brian
> >
> >
> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> >
> > Open Source Applications Foundation "chandler-dev" mailing list
> > http://lists.osafoundation.org/mailman/listinfo/chandler-dev
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev
>