[maemo-community] Reforming Karm

[From: Andrew Flegg]

On Thu, Oct 2, 2008 at 4:50 PM, Dave Neary <dneary at maemo.org> wrote:
>
> I'm going to invoke Occam's Razor on some ideas (including some of my
> own) having looked a bit furthher into how these things might get
> implemented. I think we need to make a distinction between *really* easy
> to do and *really* hard to do stuff, and favour easy over hard.

:-)

> Andrew Flegg wrote:
>
>> I think there is a lot of community building, assistance and
>> discussion on IRC. Therefore, it should (in some small way) be
>> counted.
>
> As I said, I'm sceptical, but that's beside the point. I was asking
> "what would the IRC metric be?" - how do we measure it and award karma?
> One suggestion by Eric Warnke was one point per month when your nick
> joins the channel in the last year - but of course, that doesn't measure
> participation.

It also doesn't help for the many people who stay logged in all the
time, and keep track of scrollback etc. Let's invoke Occam's Razor:

   * Someone "participates" in IRC when they say something.

(There's a potentially better version which is when someone says
something /to them/, but that's overcomplicating).

Obviously, saying something is as easy as posting to a mailing list
(well, easier) and can be just as valuable, or throw-away as a mailing
list post. However, the people who say things regularly on IRC are, I
think, contributing to the community (or a subset of it) more than
someone who doesn't.

   * Let's give a *very* low karma score for every line someone says on IRC.

The only problem here is people changing their nicks, hence the IRC
field on someone's profile probably needs to handle comma-separated
nicks (or we discourage changing nicks anyway).

> Is there any place where we can get # of comments per
> nick?

    http://mg.pov.lt/maemo-irclog/

> How would we translate that to karma?

0.0001 karma point per line, or something? Perhaps a log scale for
those who have *way* too much time.

Someone did some basic scraping and found the totals for the last year
or something. I can't remember the numbers now, but lcuk was one of
the highest with a few tens of thousands. lcuk *is* contributing to
the platform (as the excitement about liqbase showed at the summit),
but because his software isn't yet end-user ready - and, at the
moment, is mostly about experimenting with getting the fastest
performance out of the hardware - he is overlooked with the current
karma system.

> A karmabot that keeps count & updates the midgard database daily?

That would be one approach. mgedmin's already doing it, though.

>>> Some other modifications I would bring in are max and min points for
>>> things like blog entries scores and products - I think for blogs the min
>>> should be about 2 karma, the max about 10.
>>
>> Doesn't this move to weighting the karma *solely* towards
>> development(-process) tasks? Writing emails or blog posts can
>> encourage change in the community, bring disparate activities together
>> or coalesce a number of thoughts into a concrete plan.
>
> Not solely, no. It merely limits the effects of blogs - if your blog
> post gets 50 faves, you currently get 51 karma for that. If you maintain
> a popular product, you get maybe 30 karma. I know which one took more
> work...

But that's a scaling factor. Introducing a maximum's saying "we value
this contribution, but - once you've done it X times - you might as
well stop".

I'm all for weighting blog post faves much lower than product rankings
etc; absolutely. I'm also willing to accept something on a square root
or log scale to limit the effects of it at the higher end. I
*disagree* with the concept of a maximum for any aspect of the karma
system.


> Currently it doesn't even include bugzilla opening & closing bugs, I
> believe. Or wiki creates & edits. I think it should, and I think those
> activities count as doing something productive.

Absolutely. And, as Ryan says, also commenting on bugs (in fact pretty
much *any* modification to Bugzilla should get you some positive
karma); otherwise triagers and people helping diagnose problems aren't
being told their contribution is valued.

> I agree that talking has value - and I propose that we give it some
> value - but I think that doing stuff has *more* value, and I want to
> make sure Karma weights things appropriately.

Again, 100% agreed. As I say, I don't like the idea of an arbitrary
maximum, though. Weighting "harder" things more strongly than "easier"
things should absolutely be the case.

>>> For products, I would stop counting after 4 releases, which I think is a
>>> nice balance between rewarding product maintainers, historical
>>> participation and supporting older distributions on the one side, and
>>> overpowering karma by over-weighting products in the case where they are
>>> maintained for every single Maemo release.
>>
>> Would it be better to do something like some of us discussed at the
>> summit: karma elements have a half-life; if you released 3 version of
>> a product for the 770, that *is* less valuable to the community today.
>> This kind of arbitrary limit on the amount of karma you can earn seems
>> like the wrong approach to me.
>
> Actually, both of these propositions (mine & yours) would be
> inordinately difficult to manage with the midgard database as it is.

Good. The "4 releases" thing is just another aspect of arbitrary karma
limits for some activities.

> I like the half-life idea, I know that Eclipse uses something similar to
> manage inactive committers (there it's binary obviously, but if you make
> no commits in six months, or fewer than 3 over the previous year or so,
> you can lose committer status). But I worry that if there's a lot of
> work putting it into practice that it won't be a good investment.

I don't know the midgard code, so if it's a lot of work to implement
"fading" karma, we perhaps shouldn't do it. I think it's probably the
best way of ensuring that no-one can hoard karma, and that people need
to stay active in the community to maintain their karma level.

For example, when karma was introduced it was suggested that it could
be used to help determine who might get any future potential device
programme discounts. a) this should favour people who are currently
more active than were just *very* active 3 years ago; b) it should be
possible to eye the top 50-100-200 karma ratings for any tweak and say
"does this roughly look like the names I'd expect to see"?

Cheers,

Andrew

-- 
Andrew Flegg -- mailto:andrew at bleb.org  |  http://www.bleb.org/
maemo.org Community Council member