[maemo-developers] N800 & Video playback

[From: Siarhei Siamashka]

On Tuesday 01 May 2007 17:49, Kalle Vahlman wrote:
> > OK, here is this untested a patch for xserver to add ARMv6 optimized
> > YUV420 color format conversion. Theoretically it should compile
> > (I did not try to build xserver myself though) and work. If it refuses to
> > compile, fixing the patch should be not too difficult.
>
> Applied and build without problems for me.

Thanks a lot for building the package and putting it for download, everything
seems to be fine, but more details will follow below.

> For testing, I fabricated some video with gstreamer:
>
> which resulted in 640x480 at 30fps and 800x480 at 30fps videos. For some
> reason 320x240 and 352x288 refused to play with:
>
> X11 error: BadValue (integer parameter out of range for operation)
> MPlayer interrupted by signal 6 in module: flip_page while gstreamer did
> play them just fine. Also the Nokia_N800.avi and  NokiaN93.avi died in the
> same way. 

This X11 error on video playback start and also sometimes on switching
fullscreen/windowed mode is a known problem [1] reported in this mailing list.

If MPlayer dies on start, usually trying to start it again succeeds. So these
320x240 and 352x288 videos could be played as well if you were a bit more
persistent :)

As Daniel replied in one of the followup messages, it is most likely some race
condition. The question is which code is a suspect. Is it MPlayer Xv video
output code that has been around for ages and worked fine on different systems
or relatively new Xv extension code from N800 xserver? In addition, a previous
revision of N800 firmware had a serious bug [2] related to video playback. It
should be noted, that MPlayer needed only about 1 minute to freeze on the
initial N800 firmware. So the problem could be identified much more easily
if MPlayer was included in the standard set of tests done by Nokia QA staff
before each new IT OS release. Surely, Nokia is only interested in a
properly working xvimagesink for the software included in IT OS by default.
But testing with more client applications can improve overall xserver quality.

With all that said, I don't know if MPlayer Xv code is bugfree, it wasn't me
who developed it.

> My mplayer is compiled from the svn
> trunk of the garage project, with some additional cflags I use (so
> maybe those were the problem...).

Do you have a set of cflags settings which work better than the default set?
Can you share this information?

> There's something fishy in the decoding or something as the color bars
> in the test video were broken (yellow and cyan to be precise), but
> that seemed to be the case in a "vanilla" image too so nothing to do
> with this patch. I could not see any other glitches in the output.
>
> But on to the results:
>
> VIDEO:  [DX50]  640x480  24bpp  30.000 fps  1597.6 kbps (195.0 kbyte/s)
[snip]
> VIDEO:  [DX50]  800x480  24bpp  30.000 fps  1976.5 kbps (241.3 kbyte/s)
[snip]
> There is a clear drop in amount of time used to output the videos for
> 800x480 (the numbers were stable trough multiple runs).
>
> So I gather from the >10s benchmark time that we didn't get to real
> time yet, but close to it? And of course this is just video, audio
> decoding should be considered for real video playback performance
> measurement.

These videos are way too heavy for N800 to decode and play in realtime. We
may expect playback for videos up to 640x480 resolution with <1000kbps 
bitrate and 24fps. This is probably current realistic limit which can be
achieved. Some minor variations to these parameters are possible (for example
we can get 30fps, but should also reduce resolution or bitrate, etc.).

If you want a guaranteed video playback with divx/xvid/mpeg4 codecs, you
should restrict to 512x384 resolution or lower and keep bitrate reasonable.

The results for these 'insane' videos you have posted are somewhat weird, a
complete statistics would  require also a number of frames dropped, otherwise
we don't know how much work was done by the player. Probably missing audio
track resulted in MPlayer not being able to provide a proper report. Don't
know. Also it is strange that you did not see any improvement at all for
640x480 video, are you sure you really tested it with the patched xserver?

Anyway, the new xserver package works really good. If we do some tests with
the standard Nokia_N800.avi video clip, we get the following results with the
patched xserver:

#  mplayer -benchmark -quiet -noaspect Nokia_N800.avi
BENCHMARKs: VC:  29,764s VO:   7,666s A:   0,468s Sys:  64,635s =  102,534s
BENCHMARK%: VC: 29,0287% VO:  7,4767% A:  0,4565% Sys: 63,0381% = 100,0000%
BENCHMARKn: disp: 2504 (24,42 fps)  drop: 0 (0%)  total: 2504 (24,42 fps)

#  mplayer -benchmark -quiet -noaspect -dr -nomenu Nokia_N800.avi
BENCHMARKs: VC:  30,266s VO:   5,490s A:   0,467s Sys:  66,286s =  102,509s
BENCHMARK%: VC: 29,5255% VO:  5,3554% A:  0,4560% Sys: 64,6631% = 100,0000%
BENCHMARKn: disp: 2501 (24,40 fps)  drop: 0 (0%)  total: 2501 (24,40 fps)

Results with unpatched xserver and some more explanations can be found in [3].
Yes, now N800 is faster than Nokia 770 for video output performance at last :)

Video output overhead on N800 is really at least halved. Of course, video
output takes only some fraction of time in video player. So overall
performance improvement for Nokia_N800.avi playback is approximately 20% 
but not 250%-300% which can be observed for  'omapCopyPlanarDataYUV420'
function alone.

In my opinion, that's a good result for less than a week of work (a few
evenings for initial research, full weekend of intensive coding,  some
more time for additional testing, final tweaks and communication on the
mailing list) :)

I have also submitted this patch to maemo bugzilla, hopefully it (or its
modification) can get included into the next version of N800 firmware:
https://maemo.org/bugzilla/show_bug.cgi?id=1278


1. http://maemo.org/pipermail/maemo-developers/2007-April/009864.html
2. https://maemo.org/bugzilla/show_bug.cgi?id=991
3. http://maemo.org/pipermail/maemo-developers/2007-April/009925.html