[maemo-developers] [maemo-developers] Re: Request: Python and Flash benchmarks

[From: Osvaldo Santana Neto]

Hi Ken,

Ken Rimey wrote:
> Rudá Moura wrote:
> 
>> This is what pystone shows me in my Nokia 770.
>>
>> su-18-21:~# python2.4 pystone.py
>> Pystone(1.1) time for 50000 passes = 184.19
>> This machine benchmarks at 271.459 pystones/second
> 
> 
> Strange.  I get 515 pystones/second with Python for Series 60
> (1.1.6, "fast" version) running on a Nokia 7610, which uses a
> 123 MHz ARM-9.  I wonder what's going on.

Very strange, indeed. After reading your email, we decided to redo the
tests and this time, our results were much better than the results we
had before.


Python 2.3 (N770 ARM9/220MHz) Avg Result: 727.4 pystone/s
-----------------------------------------------------------------
# python2.3 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 67.04
This machine benchmarks at 745.823 pystones/second
# python2.3 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 70.51
This machine benchmarks at 709.119 pystones/second

Python 2.4 (N770 ARM9/220MHz) Avg Result: 611.3 pystone/s
-----------------------------------------------------------------
# python2.4 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 80.5
This machine benchmarks at 621.118 pystones/second
# python2.4 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 83.12
This machine benchmarks at 601.54 pystones/second


We are still amazed by the results because the device is the same of the
previous benchmark.  There were no software changes/upgrades or tuning
in hardware for such different result.

We have some ideas to discuss and share about these results. We also
really appreciate any feedbacks for the subject... Let's go!

First of all, it's important to notice that both platforms (60 series
and Maemo) have huge software differences. They involve Linux vs
SymbianOS and Graphical Environment too.

Pystone is only affected by CPU power. Other related stuff like I/O,
Video throughput, memory access, etc is not relevant for Pystone (or
less relevant).

Let's enumerate some points who directly influence in benchmark's
results for Pystone.


1. Different releases of Python
===============================

PyMaemo is based upon Python 2.4.1 while PyS60 is based upon Python
2.2.2. On my desktop I started pystone with Python 2.2 and then with
Python 2.4. I have a Pentium4 class machine -- I have no time to build
and install other Python releases on many small devices ;)


Python 2.2 (desktop Intel Xeon 3.5) Avg Result: 31251.2 pystone/s
-----------------------------------------------------------------
$ python2.2 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 1.59
This machine benchmarks at 31446.5 pystones/second
$ python2.2 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 1.61
This machine benchmarks at 31055.9 pystones/second

Python 2.4 (desktop Intel Xeon 3.5) Avg Result: 44445.3 pystone/s
-----------------------------------------------------------------
$ python2.4 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 1.13
This machine benchmarks at 44247.8 pystones/second
$ python2.4 pystone-python2.4.py
Pystone(1.1) time for 50000 passes = 1.12
This machine benchmarks at 44642.9 pystones/second


This show us that different versions of the Python interpreters (2.2 and
2.4) can influence in results for about 30%

One more benchmark. This is on OMAP1611 (ARM9 @ 192MHz) who has a
similar CPU of Nokia 770.


Python 2.3 (OMAP1611 ARM9/192MHz) Avg Result: 535.2 pystone/s
-------------------------------------------------------------
$ python2.3 pystone-python2.2.py
Pystone(1.1) time for 10000 passes = 18.85
This machine benchmarks at 530.504 pystones/second
$ python2.3 pystone-python2.2.py
Pystone(1.1) time for 10000 passes = 18.52
This machine benchmarks at 539.957 pystones/second

Python 2.4 (OMAP1611 ARM9/192MHz) Avg Result: 482.0 pystone/s
-------------------------------------------------------------
$ python2.4 pystone-python2.2.py
Pystone(1.1) time for 10000 passes = 21.03
This machine benchmarks at 475.511 pystones/second
$ python2.4 pystone-python2.2.py
Pystone(1.1) time for 10000 passes = 20.47
This machine benchmarks at 488.52 pystones/second

4. In Resume
============

+------------------------------+-----------------------------+
|                              |         pystones/s          |
+------------------------------+---------+---------+---------+
| Python                       |   2.2   |   2.3   |   2.4   |
+------------------------------+---------+---------+---------+
| Intel Xeon 3.4GHz            | 31251.2 | 37880.9 | 44445.3 |
| OMAP1611b H2 ARM9/192MHz     |      -- |   535.2 |   482.0 |
| N770 ARM9/220MHz             |      -- |   727.4 |   611.3 |
| N6600 ARM9/104MHz SymbianOS7 |   130.5 |      -- |      -- |
+------------------------------+---------+---------+---------+

These tests are done with pystone script available in Python 2.4.1
sources with 50k loop iterations. From now and then we believe that we
should choose this script for our tests.

Thanks in advance,
INdT SDK Team