[Chandler-dev] Unit and Functional Tests updated to handle Unicode character display

[Brian Kirsch]

Hello,
As part of our Internationalization improvements, the Functional test 
suite and the Chandler unit tests
have been updated to use strings which contain non-ascii characters.

To accomplish this, a new method "uw" was added to the i18n.tests package.

uw stands for unicode wrapper,  and that is exactly what  it does wraps 
an ascii string with unicode characters for testing.

An Example:
===============
from i18n.tests import uw

event.displayName = uw("test")
self.assert_(event.displayName == uw("test"), "display not set 
properly,  display is %s"
         % event.displayName.encode("utf8"))                          


** Going forward all strings in unit and functional tests should 
leverage the uw method. ***

Updating the tests revealed many where developers were not taking in to 
account that Chandler textual data is Unicode.

The most common mistake was not converting a Python unicode object to 
bytes when logging.

The Python logging package does not accept unicode objects.

But if the unicode object only contains ascii characters Python will 
internally convert it to bytes.
The unit tests were passing since the data in the unicode object was 
ascii. Soon as a character greater than 127 was
introduced the tests fail.

It is up to the developer to encode unicode objects as utf8 and pass it 
to the logger.

Another common mistake was file system API calls with Python unicode 
objects.
Since the unicode objects contained only ascii the tests were passing. 
Soon as a non-ascii character was
introduced the tests failed.

So always encode your unicode objects with the file system encoding when 
making  OS calls.

An Example:
=================
   import sys, os
   from i18n.tests import uw

   u = uw("test")
   path = os.path.join(os.path.dirname(__file__),
                                 u.encode(sys.getfilesystemencoding()))
  
   os.makedirs(path)  


The worst offender was in the Functional Test API. There were string 
casts of unicode objects
i.e. str(someUnicodeData).

*** Never ever cast a unicode value to str in Chandler. ***


In the .7 release, there will be more i18n improvements to the unit and 
functional tests including locale awareness of
date, time, number, and display strings.

So please take advantage of the new uw method in your tests.


Thanks,
Brian

-- 
Brian Kirsch -  Cosmo Developer / Chandler Internationalization Engineer
Open Source Applications Foundation
543 Howard St. 5th Floor
San Francisco, CA 94105
http://www.osafoundation.org