[maemo-developers] Qt and Multimedia: the big mess

[From: Kai.Vehmanen at nokia.com]

Hi,

I've been meaning to reply to this a few times, but now I
cannot resist anymore... comments inline.
 
On 16 July 2010, Benno Senoner wrote:
>- using pulseaudio in Maemo and in Meego is very bad since pulseaudio
>was not designed with real time and low latency robustness in mind.
>Instead of Nokia tapping into the linux audio talent and it's members
>http://www.linuxaudio.org and using and extending the JACK audio
>server ( http://www.jackaudio.org ), they did chose

Now this is not a really a fair assessment.

Like Benno, I have a linuxaudio.org background (since way back in 1999). 
I have been working with JACK project since its start in 2001. 
And I also work for Nokia/Maemo/MeeGo (since long before Maemo). We also 
have a few other developers on board who have linuxaudio.org 
community background, and have also committed code to upstream JACK. And we 
do have been discussing (and at times promoting) JACK during the years. 
I'm not sure how much this changes anything from your point of view, but 
at least I can say JACK hasn't been ignored without consideration.
 
Now while I'm not working in the team that selected Pulseaudio
back in Maemo/Fremantle times, I've been following the work since
the beginning. And while I still very much like - and promote - JACK, 
for many uses, it's not obvious it would a good fit the Maemo/Meego
use-case.

Just a few things to note. PA has...
  - existing infra to handle non-realtime-safe clients,
  - multi device support (e.g. on N900 we make soft handovers from one 
    ALSA PCM to another -> PA has this out-of-the-box),
  - existing infra to handle streams of differing sample-rates,
  - volume management infrastructure (still way to go, but it's
    already fairly advanced)
  - various optimizations to lower power consumption
  - good infra to implement the fairly complicated policy rules
    common on mobiles
  - ... and others I most probably have missed

Of course JACK could be extended to cover this (and many things
are doable with existing code), but PA is just more ready to handle
this type of non-pro-audio use now. While this does not prove anything,
PA _was_ picked up independently by Moblin, Maemo and Palm Pre.

And one very important factor is also that the project is aimed to solve
problems for desktop-ish usage. Paul (the main JACK author) has 
on many times said explicitly that JACK is aimed at pro-audio
use. Many would argue that this strict design focus has been one
key factor why JACK is so good in pro-audio use-cases. Lack of full 
alignment with upstream is not a showstopper, but for a distro like 
Maemo, it makes sense to pick projects whose lead devs share the same 
goals. In recent years, there has been many discussions 
about merging JACK and pulseaudio (by extending either project 
sufficiently to cover enough use-cases), but it's for sure not
a trivial task, nor still whether desktop and pro-audio reqs 
could me met simultaneously without compromising the other.
 
Plus then a few comments to what you Benno say about PA:

> to go with an amateurish project like pulseaudio and now want 
> to use it in Meego too.

I think this is just not true. I think PA has a bit bad reputation
among other audio developers mostly due to bad interactions with rest of 
the audio communities (the recent policykit flamewar on linux-audio-dev,
Lennart's why-I-hate-ALSA mail to alsa-devel, and various other
episodes). Looking at the actual codebase, it's not bad at all. Sure some 
design choises irk those of us with pro-audio background, but there aren't
too many of them. And as for the bad interactions, the subsequent story 
hasn't been that bad in all cases. E.g. PA devs have been pushing a lot 
of good patches to ALSA (which is to be applauded, as ALSA has been 
historically a project that is widely used, yet everybody complains about 
it all the time, but still very few people jump in to contribute).

>That is simply insane ! The wheel was already invented (jack is
>multiprocessing capable and provides single digit latency) but Nokia
>did chose to adapt a broken wheel )to their needs.

Latency is not the problem with PA, trust me. Pulseaudio actually performs
really well latency-wise on the N900 (also the N900 kernel
is really solid in this regards). The problematic area is how
to offer low-latency access to the audio subsystem to higher
level APIs, and how the intermediate APIs are configured w.r.t.
buffering (a single push-based API in the middle of the stack of
course spoils the latency immediately). Allowing access to SCHED_FIFO/RR is 
also problematic, as faulty apps (not necessarily malicious but simply
bugs in otherwise legit apps) might cause severe system-wide issues (e.g. not
being able to make calls, and various other side-effects).
This is no excuse for the bad latency you've experienced with
QAudio, but this is not because of PA. What we effectively need is
managed access to real-time scheduling, and exposing that through
e.g. QAUdio. But this problem is no way specific to PA (nor solved 
by JACK alone).

>all processes and callbacks involved into low latency audio processing
>needing to be scheduled with high priority
>(SCHED_FIFO under Linux) or like jack does very elegantly, where jack
>takes care of the scheduling priorities while the programmer only

N900 uses good ol' linux-audio-dev design patterns extensively 
for its audio cases, and PA has worked just fine in implementing
these. It provides lock-free internal helper APIs, does not do blocking 
actions on real-time paths, and is able to run with SCHED_FIFO/RR without 
issues (specific to the scheduling policy). I've struggled and chased 
down various real-time offenders during the N900 project, and PA was 
never the one giving me headaches.

PS With all that written, I do appreciate your feedback and I hope
   relevant stakeholders in Maemo/MeeGo take note (and I will do
   my best to spread the word). Also, I think your general point that 
   support for low-latency cases should be  improved, is valid and I do 
   agree with it.
 
Br,
-- 
first.surname at nokia.com (Kai Vehmanen)