[Dev] Performance Update: Why is Chandler slow on Mac?
towns at osafoundation.org
Wed Mar 8 15:50:22 PST 2006
I actually had access to an Intel iMac today so I ran your python
test on the machine.
Intel-based iMac (running 1.83 GHz Intel Dual Core) with 1GB of RAM.
The results: 31250 pystones/sec.
By comparison, G4 based PowerBook (1.67GHz PowerPC G4) with 1GB of RAM:
The results: 22831 pystones/sec.
Just thought you might be interested.
On Mar 8, 2006, at 3:09 PM, John Anderson wrote:
> During the sprints at PyCon I worked on Performance, focusing on
> trying to understand why Chandler is slower on Mac than other
> platforms. Preliminary results seem to indicate that OSX Macs are
> just overall slower than Windows and Linux PCs. Read on for the
> nitty-gritty details.
> Launching Chandler with a preexisting repository -- the normal user
> experience -- on the 1.2Ghz OSX iBook G4 I had at PyCon took 14
> seconds compared to about 4 seconds on my 3Ghz P3 Shuttle running
> XP or Fedora Core3. Alec's XP Pentium M laptop was not far behind
> the 4 second mark.
> The 3.5 times slowdown in launch translated into a significant loss
> of productivity for me. When developing Chandler I usually run in
> WingIDE. On my 3Ghz P4 shuttle, launching Chandler in Wing with an
> existing repository takes 12 seconds and launching while creating a
> new repository takes 36 seconds. Multiplying those times by 3.5
> made the iBook too slow for development. This probably explains why
> Macintosh developers avoid using Wing or even running Chandler
> whenever possible.
> Next I decided to run some benchmarks to see how C and Python
> performance compared on the different machines. Anyone who has run
> benchmarks on modern multitasking operating systems knows that
> running the same benchmark can sometimes vary as much as a factor
> of two on the same machine for a variety of reasons. Sometimes it's
> because some other task is using the CPU. Sometimes the machine
> saves power by slowing down the processor. I ran all the tests
> several times to try to get representative numbers.
> The first test I ran was Python's pystone, which I ran as follows:
> from test import pystone
> pystone.pystones (loops=pystone.LOOPS)
> The iBook did 18,939 pytsones/sec. 3Ghz P4 XP did 39,463. 3Ghz P4
> Fedora Core 4 did 27,932. XP Pentium M was comparable to the 3Ghz
> P4 XP result. So on first glance, Python performance seems twice as
> slow on the iBook compared to the XP machines.
> Running the Python "make test" on the iBook vs. XP Pentium M
> produced a similar XP speed advantage of roughly a factor of 2.
> Since Fedora and OSX use the same gcc compiler, yet XP is 1.4 times
> faster than Fedora on the same hardware, it looks like a big part
> of the difference in performance is explained by the Microsoft
> compiler. Not only does the Microsoft compiler produce much better
> code, it runs significantly faster -- Bear mentioned that builds on
> the iBook are about 5 times slower than Windows.
> I have a 10,000 line C program I wrote years ago that I use for
> performance measures on different processors. It's an interpreted
> language, not unlike Python, which I suspect compares to the kind
> of C code that makes up Chandler. The iBook did 141K instructions/
> sec. 3Ghz P4 XP did 1226K/sec. 3Ghz P4 Fedora did 800K/sec.
> Pentium M XP did about 1000K/sec. A 1Ghz PowerBook did about 300K/
> sec. The first NeXT machine did about 30K/sec. So in this test the
> iBook was almost 9 times slower than Intel/XP. The Microsoft
> compiler advantage, 1.5 times speed up, was comparable to the
> pystone tests -- even using -O3, the highest level of optimization
> for the gcc builds.
> Alec pointed out a C benchmark, GeekBench (http://www.geekpatrol.ca/
> geekbench/) which has lots of tests that measure the speed of
> different tasks on a variety of processors. Although their tests
> didn't compare the same processors I tested, the most varied CPU
> test gave the Microsoft/Intel vs OSX/G4 a 2 to 4 speed advantage.
> The exciting news, however, is that the iMac Core Duo 2.0 was
> roughly equal to the best Microsoft/Intel machines.
> Here are my conclusions: Although these results don't explain the
> whole story, the XP/Intel Chandler launch advantage of 3.5 seems
> like it could be entirely be explained by the difference in C
> performance. Furthermore, a significant part of this advantage is
> due to the Microsoft compiler. There are more tests that would be
> interesting to do: profile the C code in Chandler to see where time
> is spent during launch and other time consuming tasks. Run tests on
> the Intel Macs. Measure how much IO is involved running Chandler.
> Construct some Berkeley DB benchmarks and measure them on the
> different platforms. Finally, Given Moore's law, the current state
> of Chandler, and its probable lifetime (assuming it's successful),
> performance probably won't be a significant obstacle to adoption.
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> Open Source Applications Foundation "Dev" mailing list
More information about the chandler-dev