[maemo-developers] Playbin volume bug for OGG and WMA playback

[From: Nick Nobody]

This has been bugging me for quite some time and I think I've finally
gotten to the bottom of it. In this message I present my (limited)
understanding of how gstreamer works, what this annoying bug is,
potentially why this bug occurs and perhaps how to fix it.

Lets get on with it.

There are several ways that audio gets played through gstreamer, the
audio can either be decoded on the device's processor or it can be
offloaded to one of a few different hardware decoders (DSPs).
Gstreamer's playbin provides a very easy way for the developer to not
have to worry about writing pipelines for every possible type of audio
that can be played back by their application. For MP3 and AAC audio,
gstreamer uses hardware decoding provided by dspmp3sink and dspaacsink
respectively. OGG and WMA audio must first be decoded to PCM (decoderbin
takes care of this nicely) and then get passed to dsppcmsink.

On maemo, the audio sinks (dspmp3sink, dspaacsink, and dsppcmsink) all
have their own "volume" property (an integer between 0 and 2^16) and
another property named "fvolume" which seems to simply be volume/2^16 (a
float between 0 and 1). Either of these can be set, they accomplish the
same goal; controlling the volume level.

Gstreamer also provides a nice volume control element that you can add
in a pipeline to presumably control the volume before it gets to the
audio sink in the event that the audio sink doesn't have a volume
property. The documentation says that the volume control element can
take a value from 0 to 10 but during my testing it seems that anything
over 1 distorts. So in reality you'll only be using it from 0 to 1.

The Problem.

If you create a pipeline using playbin to playback an ogg or wma file,
you end up with a volume control element stuck right before dsppcmsink
(see image #1).  In this case playbin's volume property is connected to
both the volume control element and the dsppcmsink "fvolume" property.
This is were it gets weird, when you set the volume property of the
playbin, the volume control element gets set to that exact number BUT
the dsppcmsink's "fvolume" property gets set to one tenth of that
number.

Big deal, right?

Wrong. This means that you can only set the output volume to about 10%
before it begins to distort. For example, if I set the playbin's volume
property to 3 (referring again to image #1), the volume control element
gets set to 3 (we have distortion at this level) and the dsppcmsink's
"fvolume" property is set to 0.3. The maximum value we can set the
playbin's volume property is 1 (instead of 10, like when playing mp3s),
anything else sounds bad.

The Solution.

There are two ways I can think of to solve this problem. Make the volume
control element and the dsppcmsink share the same volume property or
remove the volume control element entirely for playbins. I just don't
really know how to go about fixing it...

I suspect this issue has something to do with the way dsp sinks work on
maemo's gstreamer. When using a dspmp3sink or a dspaacsink via a
playbin, the volume property is a number between 0 and 10 which gets
divided by 10 and used as the sink's "fvolume" value (see image #2).
Could it be that someone simply forgot to divide by 10 in the volume
control element code?


Here are some gstreamer pipelines to help you experiment with this
problem yourself (use these on your NIT).

The following two pipelines sound exactly alike (very distorted), the
first using playbin and the second using decodebin, a volume control
element and dsppcmsink. The second is essentially the same as what
playbin creates, which is wrong.

gst-launch-0.10 playbin uri=http://tinyurl.com/524rwv volume=6

gst-launch-0.10 gnomevfssrc location=http://tinyurl.com/524rwv \
  ! decodebin ! volume volume=6 ! dsppcmsink fvolume=0.6


Now if we change the volume element's volume setting from 6 to 0.6,
there's no distortion:

gst-launch-0.10 gnomevfssrc location=http://tinyurl.com/524rwv \
  ! decodebin ! volume volume=0.6 ! dsppcmsink fvolume=0.6


These images were generated on an n810 running Maemo 4.1 with the
ogg-support 0.9 and gstreamer version 0.10.22 (the version that comes
with Maemo isn't able to generate images of pipelines).

Image #1 (OGG file, playbin, volume=3): http://imgur.com/H1Uiy.png
Image #2 (MP3 file, playbin, volume=3): http://imgur.com/oEB4Y.png


I'm pretty sure this is a bug, but if not please let me know what I'm
doing wrong here. Or perhaps someone could explain how Nokia's media
player handles volume or pipeline creation since it doesn't seem to
experience this bug.

It would be a shame to have to force the application developer to work
around this problem by either worrying about creating different
pipelines for different media types or just limiting playbin's volume to
1. I guess my point is, we should be able to take advantage of
gstreamer's awesome playbin simplicity on Maemo.

Thank you for your time and patience,

nick