[maemo-developers] N800 & Video playback

[From: Siarhei Siamashka]

On Monday 19 March 2007 22:34, you wrote:

<snip>

> Again, if there are any particular questions I can answer, don't be
> subtle: ask me straight up.  If I can answer them (some things I can't
> necessarily say, some things I don't necessarily know), I will.

Thanks, here we go and sorry for a long delay with this answer.

First thanks for Xv update which makes it really usable now, MPlayer now uses
Xv video output on N800 by default. But there are still some problems. Using
unmodified upstream MPlayer code for Xv (N800 with 3.2007.10-7
firmware at the moment) does not work good. It has two at least problems:

1. Lockups which look like cycling two sequential frames, very similar or the 
same problem as https://maemo.org/bugzilla/show_bug.cgi?id=991 
Also keypresses are not very responsive. A fix (or workaround) required
changing XFlush to XSync in screen update code, now it looks a lot better.

2. Switching windowed/fullscreen mode generally makes mplayer terminate
with the following error messages:
"X11 error: BadValue (integer parameter out of range for operation)"
"Xlib: unexpected async reply (sequence 0x5db)!"
A workaround to make this problem less frequent was a code addition which
prevents screen updates until we get Expose even notification.

All these Xv patches for MPlayer code can be viewed here:
https://garage.maemo.org/plugins/scmsvn/viewcvs.php?root=mplayer&diff_format=h&view=rev&rev=166

I really don't know much about X11 programming and only started to learning
it, so your help with some advice may be very useful. Looks like MPlayer code
X11/Xv output code is a big mess with many tricks and workarounds added to
work on different systems over time. Maybe it contains some bugs which get
triggered on N800 only, but apparently this code is used for other systems
without any problems. Can you try experimenting a bit with MPlayer (upstream
release) yourself to check how it works with N800 xserver? Maybe it can reveal
some xserver bugs which need to be fixed? Also if MPlayer has some apparently 
bad X11 code, preparing a clean patch and submitting it upstream maybe a 
good idea.

One more strange thing with Xv on N800 can be reproduced by trying to watch
standard N800 demo video in MPlayer. It has an old familiar tearing line in
the bottom part of the screen and the performance is very poor. The same file
plays fine in the standard video player. The only difference is that mplayer
respects video aspect ratio (this video is not precisely 15:9 but slightly
off) and shows some small black bands above and below picture and 
default video player scales it to fit the whole screen. Disabling aspect ratio
in mplayer with -noaspect option also 'fixes' this problem.

Using benchmark option we get the following numbers:

# mplayer -benchmark -quiet Nokia_N800.avi
[...]
BENCHMARKs: VC:  33,271s VO:  66,768s A:   0,490s Sys:   5,703s =  106,232s
BENCHMARK%: VC: 31,3189% VO: 62,8517% A:  0,4614% Sys:  5,3681% = 100,0000%
BENCHMARKn: disp: 1732 (16,30 fps)  drop: 778 (30%)  total: 2510 (23,63 fps)

# mplayer -benchmark -quiet -noaspect Nokia_N800.avi
[...]
BENCHMARKs: VC:  32,226s VO:  14,350s A:   0,456s Sys:  55,699s =  102,731s
BENCHMARK%: VC: 31,3694% VO: 13,9687% A:  0,4439% Sys: 54,2180% = 100,0000%
BENCHMARKn: disp: 2501 (24,35 fps)  drop: 0 (0%)  total: 2501 (24,35 fps)

So when showing video with proper aspect ratio, we get tearing back and more
than 4x slowdown in video output code (66,768s vs. 14,350s). This all results
in 30% of frames dropped.

These were the 'usability' problems with Xv. Now we get to performance
related issues. As YV12 is not natively supported by hardware, some 
color format conversion and bytes shuffling in video output code is
unavoidable. It is a good idea to optimize this code if we need a good
performance for high resolution video playback. Color format conversion 
can be optimized using assembly, for example maemo port of mplayer
has a patch for assembly optimized yv12-> yuy2 (yuv420p -> yuyv422) 
nonscaled conversion which provides a very noticeable ~50% improvement
on Nokia 770:
https://garage.maemo.org/plugins/scmsvn/viewcvs.php?root=mplayer&rev=129&view=rev

Also here is a JIT accelerated scaler for yv12-> yuy2 (yuv420p -> yuyv422)
conversion, it is very fast and supports pixels interpolation (good for image
quality) :
https://garage.maemo.org/plugins/scmsvn/viewcvs.php/trunk/libswscale_nokia770/?root=mplayer

I have seen your code in xserver which does the same job for downscaling, but
in nonoptimized C and with much higher impact on quality. Using JIT scaler
there can improve both image quality and performance a lot. The only my
concern is about instruction cache coherency. As ARM requires explicit
instructions cache flush for self modyfying or dynamically generated code, I
wonder if  using just mmap is safe (does it flush cache for allocated region
of  memory?). Maybe maemo kernel hackers/developers can help with this
information?

It should be noted, that all this assembly optimized code was developed for
Nokia 770. N800 has a much faster memory (up to 190MB/s memory copy
performance vs. 110MB/s on Nokia 770) but requires a bit different
optimizations (seems to need explicit prefetch with PLD instruction for
reading data). I'm going to try making N800 optimized color format conversion
functions a bit later.

But here is one more problem. As color format conversion is done in xserver, 
it will take a really long time before any such optimizations can be delivered
to end users. Nokia seems to have unpredictable (to outsiders) and slow
releases schedule.

So for any performance optimizations experiments which result in immediate
video performance improvement, either direct framebuffer access should be 
used again or it would be very nice if xserver could provide direct access to
framebuffer (video planes) in yuy2 and that custom yuv420 format in one of the
next firmware updates. The xserver itself should not do any excess memory copy
operations as they degrade performance (and it does such copy for yuy2 at
least).

Also I'm curious about that yuv420 format. From the comments in your code, it
looks like it is different from what is described in Epson docs. That seems a
bit weird.

Thanks for doing a great job supporting maemo community, your comments have
been always very informative and helpful here.