[maemo-developers] N800 & Video playback

From: Siarhei Siamashka siarhei.siamashka at gmail.com
Date: Sun Mar 18 19:57:36 EET 2007
Hello All,

I did some tests with the framebuffer when trying to find a way to reduce
tearing effect in MPlayer. Here are the results.

I did a mistake when I assumed that screen updates are synchronous for video
planes. They are actually asynchronous just like with Nokia 770, but just a
lot slower so that it is not possible to keep framerate in real time. So we
get blocking when trying to display the next frame and the previous screen
update is still in process.

If we look at the framebuffer API. There are two ioctl important for screen
updates and tearing synchronization if I understand them correctly now:

* ioctl(fd, OMAPFB_UPDATE_WINDOW, &update);

This ioctl initiates asynchronous screen update from a 'framebuffer' memory
(actually that's just a system memory) to a graphics chip. If a previous
screen update request is still incomplete, this call blocks and waits until it
is done. The structure 'update' can have OMAPFB_FORMAT_FLAG_TEARSYNC 
flag set which instructs the framebuffer driver to wait internally (ioctl call
is not blocked) and start data transfer on the start of the next LCD internal
screen refresh (aparently refresh rate is something ~60Hz, but the numbers
are below).

* ioctl(fd, OMAPFB_SYNC_GFX);

This ioctl call ensures that current screen update is done (blocking
until data transfer is complete).

Perforrming both these ioctls consequently we can benchmark screen update
performance. The results are the following.

On N800 (OS2007 2.2006.51-6), every YUV screen update (OMAPFB_COLOR_YUY422)
takes about 41ms without tearsync enabled and 41-58ms with tearsync. It does
not matter what video resolution we try to watch, the result is the same. So
the maximal screen update rate is about ~24fps without tearsync and ~20fps on
average with tearsync. That's not enough to watch 30 fps video and achieving
24 fps is theoretically possible, but very tricky and unrealistic. If tearsync
comes into action, watching full framerate videos is impossible now. Analyzing
the difference in screen update times with vsync, looks like full cycle of LCD
internal refresh takes ~17ms (that's ~60Hz, but as the precision is not good,
that may be something else, 50Hz for example). Nevertheless screen update on
N800 can't be completed for these 17ms and tearsync does not work perfectly
(most likely it can fill the screen up to the bottom horizontal line observed
on playing video).

If we try RGB screen updates, we can see that the time needed for screen
update gets lower for updating smaller screen regions). The numbers are 
the following (without tearsync enabled):
640x480: 33.7ms
400x230: 10.2ms
320x240: 8.5ms

Of course RGB screen updates are not very suitable for video as we would lose 
much more time doing YUV->RGB conversion.

If we benchmark the screen update performance on Nokia 770, the numbers are:
640x480: 11.1ms
320x240: 2.9ms (that's fullscreen playback with pixel doubling)

If we estimate bus performance on Nokia 770, it is ~55MB/s and is more than
enough to display 800x480 sized video frames with 30 fps. Adding a tearsync
would be a nice addition, as 11.1ms for 640x480 screen update time is lower
than 17ms LCD refresh cycle. And in the worst case of video sync when we get
11.1ms+17ms=28.1ms for a single frame, it will be still capable of displaying
35 fps at the very least. So any resolution video can be played with perfect
quality given enough cpu performance for video decoding (that's a real
bottleneck on Nokia 770).

Looks like graphics bus on N800 is 3x slower than on Nokia 770. It might 
be caused by inefficient framebuffer driver implementation in its initial
revision. But if it is a hardware issue, getting normal video playback at
native framerate may be troublesome. Performing software downscaling of 
video before sending data to the graphics chip may be a solution, but it
sacrifices image quality. Switching to 12bit YUV format from 16bit will save
~33% of bus bandwidth, but it can't compensate 3x performance regression 
and may be not enough for 30 fps fullscreen video playback.

Right now, I can workaround tearing somewhat, but some frames will have to 
be skipped, resulting in somewhat jerky playback (even for transcoded video
unless framerate is halved).

Apparently the same issue applies to emulators, games and other software.

As Daniel explained, the next firmware will bring a big improvement in this
area. I'm not sure whether it is worth to release the next version of MPlayer
before that, since it will still be far from perfect on N800.

A preview of the next kernel for beta testing might reduce time needed to get
MPlayer fully working on N800, but I'm not demanding or expecting anything. It
is just a matter of time anyway and I'm not so impatient :)

I would be grateful for any comments and corrections. Some things are not 
yet clear to me, figuring them out myself is just a waste of time that could
be spent on something more useful. Even a small hint may save a huge 
amount of time.

PS. The last 'inefficient' period of time was when I was struggling with
gstreamer API (with no prior experience with it) to get MP3 playback in
MPlayer working on DSP for a few months. Looks like the history repeats. 
Once again, I'm not demanding anything, it is just a matter of 'optimizing'
development and spending scarce amounts of spare time more efficiently.
I know that Nokia developers are too busy with their primary work, and
really appreciate what they are doing. So consider this as a polite request 
for a favour (not necessary to fulfil right now or fulfil at all).

More information about the maemo-developers mailing list