[maemo-developers] [maemo-developers] Xvideo support for Nokia 770?

[From: Charles 'Buck' Krasic]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Siarhei Siamashka wrote:

> On Tuesday 09 January 2007 20:59, Charles 'Buck' Krasic wrote:
>
>> Any chance the Xvideo support in the Bora 3.0 will turn up in a
>> 770 OS?
>
>
> I asked the same question on #maemo irc channel and daniels
> explained that video scaling is done by gpu on N800, so probably
> the same code can't be reused on 770:
> https://mg.pov.lt/maemo-irclog/%23maemo.2007-01-08.log.html
>
> Actually I have been thinking about trying to implement Xvideo
> support on 770 for some time already. Now as N800 has Xvideo
> support, it would be nice to have it on 770 as well for better
> consistency and software compatibility.

As you may recall, I was considering this back in August/September.  
I tried a few things, and reported some of my findings to this list.  
The code for all that is still available here:  
http://qstream.org/~krasic/770/dsp/

>
> I see the following possible options:
>
> 1. Implement it just using ARM core and optimize it as much as
> possible (using dynamically generated code for scaling to get the
> best performance). Is quite a straightforward solution and only
> needs time to implement it.

It is my impression that this might be the most attractive option.   
I noticed that TCPMP which seems to be the most performant player for
the ARM uses this approach, and it is available under GPL, so it may
be possible to adapt some of its code.

In the long run, I would hope that integrating TCPMP scaling code into
libswscale of the ffmpeg project might be the most elegant approach,
since that seems to be the most performant/featureful/widel adopted
open-source scaling code (but not yet on ARM).   For mplayer, it works
out of the box, since libswcale actually originated from mplayer, and
only recently migrated to ffmpeg.

>
> 2. Try using dsp tasks that already exist on the device and are
> used for dspfbsink. But the sources of gst plugins contain code
> that limits video resolution for dspfbsink. I wonder if this check
> was introduced artificially or it is the limitation of DSP scaler
> and it can't handle anything larger than that. Also I wonder if
> existing video scaler DSP task can support direct rendering [2].

I tried direct rendering in the above mentioned experimentation.  I
never got it to work exactly correctly, i.e. I could get images
fragments on the screen, but they were not the whole image, and never
in exactly the correct screen position.   I suspected this was tied to
the baroque memory addressing constraints of the DSP (e.g. 16bit data
item limitations).   I tried very hard to work around them but was not
successful.

I think the benefits of direct rendering may be a false temptation on
the DSP anyway.    My impression was that the DSP access to
framebuffer memory slowed down the scaling algorithm tremendously, so
it was actually faster to scale into DSP local memory, and then do a
fast bulk copy to the FB, or to SDRAM on the ARM side.    Plus you
have all the AV synchronization headaches.

I think these gains pale compared to the gain from just using the fb
in YUV mode, and doing all the video stuff on the ARM side.
Hence, option 1 seems to sound very attractive.

> It would need to support arbitrary number of memory mapped buffers
> for video output in order to avoid unnecessary memcpy, otherwise
> performance will suffer.
>
> Maybe we can ask Nokia developers to provide some information about
> the internals of these plugins. The most important questions are: *
> What are the real capabilities of DSP based scaler, can it be used
> for resolutions let's say up to 800x480?

I doubt 800x480.   The added quality benefit over 400x240 with pixel
doubling in the fb is probably way to marginal to justify the
effort.   The DSP hardware doesn't seem to have any meaningful support
for general scaling (beyond doubling).

> * Where is the screen update performed after dsp has finished
> scaling/converting video from mapped buffer to framebuffer? Is it
> done on ARM side, or probably screen update can be also triggered
> from DSP directly?

I seem to have the rough impression from inspecting X code that ARM
side does the final update (copy) to fb memory.  I'm not 100% sure on
that right now though.

> * Is it possible to get direct rendering [2] support with existing
> dsp tasks on 770? If not, would it be too hard to implement this
> feature? * How are timestamps handled in dsp? Is it possible to
> just send a one shot signal to dsp task for rendering video frame
> from a mapped buffer as fast as possible?
>
> A brief dsp interface description would be welcome. Maybe some
> questions may be trivial, but unfortunately I did not have much
> time for a detailed walk through the sources in order to figure out
> how this all works. If any Nokia developer finds time for some
> short answers, it would really help a lot.

Agreed.

>
> 3. Try implementing a new DSP based scaler from scratch. The most
> important thing to know is how to access framebuffer directly from
> DSP and move data to it from mapped buffer without any overhead.
> The first test implementation can just perform nonscaled planar
> YV12 -> packed YUV422 conversion, if it proves to be fast and
> useful, it could be extended to also support scaling.
>
This is what I did in August.   I did YUV -> YUV scaling plus RGB
conversion on the DSP.   I think I did YUV->YUV scaling later. The
results (performance) were abysmal.   Maybe I committed some mortal
DSP programming sins that dragged the performance down, but it was soo
slow I gave up even hoping.   I think my DSP code was maxed out on the
DSP at like 20 fps, where the ARM was able to do 24fps with about
10-20% cpu.

Anyway, my code is still there which may be a start if you want to
attempt it.   However, I think your first option is probably the most
fruitful option.    My little project made me very cynical of the
value of the DSP.  ;-)

> PS. This is unrelated to Xvideo support development, but also it
> would be nice to have more or less detailed description of dsp
> based gstreamer elements and their properties. While the sources of
> these plugins are available (with a hidden dsp part), some docs are
> needed to know how they are supposed to work in order to use them
> efficiently and probably improve.

I second that request. :)

- -- Buck

>
> [1]
> http://repository.maemo.org/pool/scirocco/free/source/g/gst-plugins-dsp0.10/gst-plugins-dsp0.10_0.32.1-1.tar.gz
> [2] http://www.mplayerhq.hu/DOCS/tech/dr-methods.txt
> _______________________________________________ maemo-developers
> mailing list maemo-developers at maemo.org
> https://maemo.org/mailman/listinfo/maemo-developers


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFFpCp7PrrWIMa4SMsRAp4tAJ4iIl+ZSjVAom/qC9w0oQyCQFJfhQCeKcVO
ciS0VosqJlC+dNZHU6GOqqY=
=8bCt
-----END PGP SIGNATURE-----