[maemo-developers] Low Latency Audio Capture on the N900, QAudioInput BUG: has 5000msec latency, Meego audio future

[From: Benno Senoner]

Thanks to all for your responses, yes I agree with what you said.
I will try to track the evolution of the  audio subsystem on meego and
provide my feedback and findings.
I'll discuss the issue with the JACK authors too what they think about
it, about adding power management to jack etc.
(the main author of Jack2, Stephane Letz is one of the coauthors to
LinuxSampler too).

It's understandable that Nokia will probably not change the audio
subsystem for the current Meego version, so at least
they should try to provide a stable working system, giving developers
a way to achieve <50msec mic to speaker latency.
I'm not going to invest time to improve pulseaudio as I, like any
audio developer I know, consider it a wastly inferiour audio server
with regard to low latency and stability. But I will provide
benchmarking apps and feedback in order to point out problems if they
arise.
And as said, if Nokia and Intel manages to provide stable low latency
operation I'll happily use the API they provide and if pulse audio
is capable to do it then QAudioInput/QAudioOutput can certainly be
fixed to provide the same kind of performance since it's simply a
wrapper.

But if we discover that the performance of the current Meego for user
space apps is not easily fixable then my advice is to swap out
pulseaudio for JACK2, provide a compatiblity layer that exposes the
pulse API (if needed) so that no app needs to be rewritten etc.

It's still early to say how the whole thing turns out but I can
certainly give my share of contribution to improve the audio
subsystem.

LinuxSampler in interesting for benchmarking an OS, because it is a
very very demanding app, it really puts big stress on the OS since it
streams large, multi GByte samples
from disk, pre-caches the sample heads (first part) into memory and
allows low latency playback of the samples (with single digit msec
milatency).
It runs one thread (or a callback) with real time priority that
renders the audio, another that listens to MIDI data and a lower
priority disk I/O thread which
writes to lock-free zero-copy buffers.
And if you load a lot of samples and play complex musical pieces with
hundreds of voices you end up using 95% of your CPU, 95% of your RAM,
95% of your disk I/O bandwidth, while still providing only a few msec
audio latency without dropout.
Linux kernels with scheduling preemption enabled and JACK on ALSA have
proved to be capable to satisfy all the above conditions.
I'm not saying that Meego on a phone should be able to provide such
excellent numbers as on a desktop, but Meego engineers should work
hard to give it
response times comparable to the leading mobile operating systems and
sorry again for mentioning the Iphone :)
I fully disagree with Apple's closed systems but apart of the brand
and that's fashionable to buy their products, they get some stuff
working right especially the
audio/video parts. I don't know to which extent the
impossibility/difficulty to achieve low audio latency "worsens" the
appeal of a smartphone platform,

but it is certainly depressing when you spend several hundred $/E on a
phone with excellent specs and you cannot have apps
like virtual musical instruments (eg piano,flute, etc),  vocoders
(voice changers), apps that performs by analyzing the audio in real
time etc, while your Iphone colleagues
have all sort of apps for that. Of course the usefulness of those apps
is debatable but the fact is that the bigger the app market the more
the platform becomes,
and we all know that Ovi Store still pales compared to Apple's app
store in terms of number of daily downloads and numbers of apps
available, despite the fact that Nokia
has a much bigger market share and has deals about sharing Ovi apps
with big mobile operators like China Mobile ecc.
Apple is yet another proof that getting the basics right your product
sells without advertising.
I know nowadays it's fashionable to bash Nokia at any occasion,
especially in the US where journalists write as Nokia and Ovi Store
did not exist and the whole company will fail soon,
which is very unfair, since the US != World.
So Nokia just needs to demonstrate that there is a reason why they are
#1 and that they can beat the competitors with better hardware AND
software.
Nokia has been very open source friendly lately and developers highly
esteem this, but as said they need to get the details right and
working like a swiss clockwork, including audio :)
Nokia needs to get their things right and release products and
software much faster than in the past.
Android is advancing very fast too (100k phones / day shipped ATM) and
it's open source too so it's a big competitor for Symbian/Meego.
OTOH it is good that Nokia has competitors so they are forced to
evolve and raise the bar otherwise users will switch platforms and
perhaps if Nokia was the only player on the market
the world would probbly still use an old symbian version on keypad-style phones.

Again sorry for the long digression, and as many here, I wish Meego
and Symbian becoming relevant players on upcoming smartphones, being
able to compete and outperform rivals
on both raw performance and price/performance ratio.

regards,
Benno
The LinuxSampler Project
http://www.linuxsampler.org




2010/6/18 Sivan Greenberg <sivan at omniqueue.com>:
> On Fri, Jun 18, 2010 at 2:19 PM, Benno Senoner
> <benno.senoner at googlemail.com> wrote:
>> Sadly, this week Nokia stock again dropped like a rock, while Apple
>> gets preorders for 600k Iphone 4 in a day and have trouble
>> to process all the orders. Does that not ring  alarm bell at Nokia ?
>
> Right, this is certainly not the appropriate medium for this kind of
> discussion, but still I had the give my argument to stand up for those
> who support and follow open source with concept operating systems and
> platforms before it reaches the polish of the competitors, in sack of
> open development process.
>
> What Nokia does and has done with Maemo and MeeGo is yet unprecedented
> from a commercial market point of view. Let me assure you that if you
> communicate your issues you will get a serious response and if you're
> right with your complaints action will be taken.
>
> Sivan
>