[maemo-developers] DVD content playback possible or not? Re: Wishlist (was:Re: N800 and USB host mode)

[From: Siarhei Siamashka]

On Saturday 10 March 2007 01:57, Daniel Stone wrote:

> On Fri, Mar 09, 2007 at 10:34:52PM +0200, ext Siarhei Siamashka wrote:
> > On Friday 09 March 2007 12:20, Daniel Stone wrote:
> > > Not really.  The next firmware release has gone to great lengths to
> > > improve video performance by doing scaling on the LCD controller, as
> > > well as the colourspace conversion.  I think you'll be pleasantly
> > > surprised. ;)
> >
> > Thanks, that's a very good news. We all are looking forward for this
> > firmware update. By the way, is it possible to get an early access to the
> > updated kernels in the future for the purpose of testing and ensuring
> > compatibility?
>
> It's a kernel and large X server update.  Unfortunately I'm not in a
> position to be able to release them to the public.

That's why I asked this question in the mailing list. I hope that somebody in
a position to make such decision is reading it.  Nokia did some beta releases
of OS2006 before, so maybe it could be possible to continue this tradition?

> > N800 is a bit different with a more complicated framebuffer driver with a
> > support for more hardware features (such as a very high quality hardware
> > scaler),  but its graphics chip does not seem to support planar YUV color
> > formats, so something else (ARM core?) should do the conversion wasting
> > the same ~20% of resources. By the way, did you consider trying to use
> > DSP at least for unscaled planar->packed color format conversion? It
> > should provide some improvement at least theoretically.
>
> The LCD controller takes in a planar format, so we indeed avoid that
> conversion.  The bottleneck, though, isn't CPU or memory load, but the
> bus between the display controller and the LCD controller.  So it
> doesn't matter where we do the conversion, we just have to minimise the
> load.  Sending 12bpp (i.e., pre-scaled) video over instead of 16bpp
> post-scaled is obviously a pretty huge win.

I'm not quite sure what do you mean by 'LCD controller' and 'display
controller'. Which one of them is the Epson chip? Is there anything done
by OMAP chip in this scheme?

OK, looks like I'll have to take a look at the sources when they get released. 
The matter where we do the conversion is actually very important, if it is
done by ARM core, we'll have less cpu resources available for decoding video.
Sending 12bpp will be surely a win, I just preferred 16bpp packed YUV 
format on Nokia 770 because it used the same layout as RGB565 and 
could be placed into the same buffer. As N800 supports different planes
for video, this is not an issue anymore and using 12bpp format should be 
fine.

> The X server does all this for you -- the semantics are, uhm,
> 'nightmarish'; the LCD controller can't do colourkeyed video, only a
> single cliprect.  The Xv support already has this worked out, including
> automatic migration of your videos when a menu gets popped out or
> whatever.  And it quite rightfully expects that it's the only thing
> managing the framebuffer, so your planes may well get stomped.  You
> really want to use it.
>
> (Is there any special reason why you want to do it directly?  If so, let
> me know, and I'll see if I can introduce support for what you're trying
> to do in the X server.)

Yes, I really want to use X server and Xv. But if using it sacrifices
performance a lot, we just have to look at some other options. I'll 
have a look at Xv in the coming firmware update and if it is good 
enough, I'll be happy to drop a hack using direct framebuffer access 
in MPlayer for N800.

It is too early (or too late) to discuss it, but maybe some kind of Xsp
extension for video support to precisely match hardware capabilities 
could be developed? If LCD controller has problems with colourkeyed 
video, that's ok, not everyone needs it. If we need to make a choice 
whether to sacrifice compatibility or performance, I myself would prefer 
to keep good performance. That's why I'm still experimenting with 
framebuffer.

> > CPU performance for video
> > decoding is still another bottleneck. It is even worse bottleneck than
> > video output as you can skip displaying of some frames, but you can't
> > skip decoding.
>
> We aren't able to hit a situation where the CPU is an absolute
> bottleneck, except maybe with some absurdly complicated codec.  I
> haven't seen this arise yet.

Hanno Zulla already raised this issue, an example of such absurdly 
complicated codec is MPEG2. One more example is DIVX, I would like
to ensure that all the video samples from the following page can be 
played smoothly:
http://www.divx.com/movies/browse.php?categoryID=1

So looks like we have somewhat different goals and our vision of 
current bottlenecks differs a bit :)

> > Is IVA really unusable on N800? What kind of cpu does it have inside? If
> > it is done by TI, we can probably suppose that it is TMS320C64x (at least
> > I have seen information that IVA2 is a lower clock and more power
> > efficient version of DaVinci which uses TMS320C64x).
>
> I'm not sure, as I haven't played with the IVA.  But believe me when I
> say that right now the bottleneck is the bus between the display
> controller and the LCD controller.  You can do the maths on the maximum
> transfer rate if you don't believe me ...

While I agree that the bus transfer is slow, ARM core should not stall because
of this and can do a job of decoding the next frame already. In this sense it
is not strictly a bottleneck as this data transfer operation is performed in
parallel. If this bus transfer is fast enough for the video resolution and 
framerate we are using, that's perfect. If it is not fast enough, we can drop
some frames and still play the video. But in any case, displaying video
should not have any noticeable impact on decoding performance. I already
provided the numbers, MPlayer uses no more than 20% of ARM core
resources for displaying high resolution video right now, the results are not
very pretty (tearing is most likely caused by unability to display all the
frames properly in time because of slow bus, overlapping some of the frames,
etc.), but it is not the performance bottleneck.