[Chandler-dev] Chandler Mail Service Update

[Philippe Bossut]

Whohoo!! That's fantastic news! Haven't tried yet though... :)
Cheers,
- Philippe

Brian Kirsch wrote:
> Hello,
> As of revision 14388, the Chandler Mail Service has under gone a 
> significant architectural change
> to support downloading of large Inboxes (12,000 plus messages). To 
> accomplish this the
> Incoming Mail logic has been refactored in to two asynchronous threads 
> the Twisted Mail Client thread and the Mail Worker thread. The Twisted 
> Mail Client thread handles all downloading and pre-processing of mail 
> (building the body text, finding eimml / ics attachments etc). The
> Mail Worker thread is responsible for taking the pre-processed mail 
> off of a queue and
> assigning the values to a MailStamped Item. There are now only two 
> views in play.
> The Twisted Mail Client view shared by all downloading clients and the
> Mail Worker view used to convert pre-processed mail from all Clients. 
> With this
> check in performance has greatly increased, timeouts have been 
> minimized, and
> a pesky bus error that occurs when too many Twisted deferreds are on 
> the Python
> stack has been resolved.
>
>
> Taken from the r14388 commit message:
> ======================================
> Performance Performance Performance! Complete refactoring of Incoming 
> Mail Logic
> to a Mail Worker thread with one view and all clients and Mail Service 
> sharing a second view.
> The Mail Service can now download 12,000 messages with No timeouts and 
> Bus Errors.
> There are still performance issues in the UI and Repo layer to tackle 
> related to merging,
> refresh, and observer calls. But much improvement has been already 
> made. Fixed the Bus Error related to too many Twisted deferreds over 
> flowing the stack as well as added dynamic
> commit calculation and throttling. Improved POP performance 2x when 
> the POP3 Server does not support the TOP command (gmail).Added IMAP 
> Body.PEEK commands to prevent mail on the IMAP server from being 
> marked as seen on download. Added Blocking vs. Non-Blocking logic.
> The Incoming Mail Clients can either wait till the Mail Worker has 
> processed the last commit request or continue downloading mail while 
> the Mail Worker is converting Email Objects to Items.
> Added Preview specific Message Parsing and Conversion to Item methods. 
> These are intended to maximize RFC2882 text to MailStamped Items 
> performance. Improved status bar messages when searching for Chandler 
> Headers and when downloading and committing mail. In addition fixed a 
> number of open Preview mail bugs.
>
>
> Please not the Chandler Schema has changed as of rev 14388.
>
> Enjoy,
> Brian
>
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev