[Dev] Automatic Secure Email Update

[Andy Hertzfeld]

Last month, we posted our thinking about automatic secure email here, 
and learned a lot about various issues in the ensuing discussion, both 
on the list and privately.  This message will summarize what we learned, 
and describe our current thinking about the best approach.

I'd like to thank Aaron Swartz for pointing out Brad Templeton's work to 
me.  Brad's been thinking about what he calls "opportunistic encryption" 
for a while, and I learned a lot from discussing issues and trade-offs 
with him an off-list email exchange, and will discuss several of his 
ideas below.  You can read directly about Brad's approach at 
http://www.templetons.com/brad/crypt.html

There are two different approaches under consideration for automatically 
getting the public keys required to send an encrypted message. Brad 
Templeton's approach is to include the key in a special header as part 
of every outgoing message.  The approach suggested in my earlier 
document was to obtain them by sending a participating client a profile 
request, via a special header, which responds with a description of 
itself, including the public keys.

The "key in header" approach is simpler than the "profile request" 
approach, since it doesn't require sending or receive the profile 
response message.  If someone is using a participating client, you know 
all the information required to send them a secure message if you've 
ever received a message from them.

The biggest issue with the "keys in header" approach is that the entire 
public key is kind of large to send with every message, potentially 
increasing the message size by more than a hundred bytes.  Newer 
crytographic algorithms (ECC) can reduce key size to under fifty bytes, 
but some people think that patent issues will keep us from being able to 
use them. Also, some clients might choose to support more than one 
encryption algorithm, and therefore would have send more than one key; 
at some point the keys in the header become unwieldy.

On the other hand, the "profile request" approach is both more 
complicated and more flexible.  It has weaknesses as well, since profile 
requests could be used by spammers to verify the presence of valid email 
addresses, and the special email messages it requires might confuse 
users if read by clients that aren't aware of the framework.

Our new proposed scheme is a combination of the two schemes, equivalent 
to Brad's approach in the simplest case, but offering more flexibility 
if a client feels that it's necessary.  As in Brad's approach,  a mail 
client would add the following RFC822-style header to each outgoing 
email, something like "OppCrypt: <scheme_name>; key=<key_value>".  The 
default scheme will choose an optimized algorithm and format, and 
transmit the public key directly in the header.  However, clients can 
also support alternate schemes (this flexibility is required, since any 
scheme may be cracked someday), including a "profile request" scheme 
where the key value serves as a ticket for a future profile request, 
which overcomes the spammer vulnerability.

Another issue to consider when implementing automatic encryption is the 
extra overhead required by the security framework.  We can't get away 
with sending mail securely by default if it burdens the infrastructure 
by increasing the size of every email message significantly.  
Unfortunately, the most commonly used secure email format, S/MIME, is 
very bulky, adding more than 4K bytes to every message.  A more 
optimized format could get that down to around one or two hundred bytes, 
so there's reason not to go with the most commonly used standard.  We 
need to do more research to determine how much overhead is tolerable.

Even if our implementation of automatic secure email works great in 
Chandler, it will still be ineffective as most people won't be using 
Chandler, at least not in the beginning.  Ideally, we'd like to create 
an easy way to enable existing email clients to participate, without 
modification.  Brad Templeton suggests that we can accomplish this by 
implementing proxies for POP3, IMAP and SMTP on the client machine which 
will do all the work of adding the key in a header line and encrypting 
or decrypting as necessary, transparently to the client.  However, if we 
do this, it is relatively independent from Chandler, so it probably 
should be pursued as a separate, related project.  Or, we could 
implement it directly in Chandler, since that's less complicated and 
error prone for our users, but use a common library with the proxy, 
which would be an independent project.

-- Andy