[maemo-community] Sprint task: Refine the karma system

[From: Randall Arnold]

> ----- Original message ----- 
> From: "Sebastian 'CrashandDie' Lauwers" <crashanddie at gmail.com>
> To: "List for community development" <maemo-community at maemo.org>
> Subject: Re: Sprint task: Refine the karma system
> Date: Wed, 13 Jan 2010 01:31:52 +0000
> 
> 
> "Karma opponents" don't necessarily believe there is any kind of
> abuse. This may have been a subdiscussion some time ago, however this
> isn't the case now. If you are referring to the troll-making-karma
> example I used, I'm only pointing out it is undeserved karma.
> 
> A few points about salary: most contracts will discourage or ban the
> discussion of one's salary. Here karma would be used to make
> decisions. Also salary is a means to an end: living. I don't see karma
> as being nearly as important. Unless we will be able to buy Maemo
> t-shirts using karma points? The next point is that salary
> differentiates those who have done well for themselves and those who
> haven't. My understanding up till now is that we are looking for a
> system where very active talk contributors would have roughly the same
> karma as devs (in an ideal world). Do we really want to metaphorically
> say that we support socialism (far fetched, I know; I am not
> criticising any political beliefs here, btw)?
> 
> Regarding grades, they are limited to one specific subset of people:
> students. Which is fine by me. I don't mind some devs having a
> specific "level" based on their contributions. Nor do I have an issue
> with tmo users having their quota of posts/thanks displayed loud and
> clear. The point I find doubtful is when we try to streamline such
> metrics across widely and wildely different data sets:
> 
> You can create a very smart algorithm which measures apples, pears and
> oranges to show which batch contains the best fruit, but in the end
> you're still comparing apples and oranges.
> 
> I just find it surprising that we'd want a way to "judge" people
> singlehandedly and immediately based on a number. I don't see Quim nor
> Peter post, develop or blog furiously all year round, yet they still
> are some of the most regarded and respected people around Maemo.
> 

Sorry, Sebastian, that's just some very eloquent hair-splitting.  We could spiral down that pointless trail email after email but at some point the bang for the buck evaporates.  But again, the point I was making wasn't about "streamlining" data sets but finding parity.  Yes, apples are not oranges but as you point out they ARE both fruit.  A karma system with its numerous tentacles and algorithms attempts to rationalize disparate contributions and find some way of abstracting them the same way.  Granted, we can't simply say that 1 orange = 2/3 of an apple-- but that's not what our karma system does, so a simplistic fruit analogy fails.  Rather, if one were to say "it costs 2/3 as much to grow a single orange as it does a single apple" THEN we have a sort of parity.  THAT is what we're working at.

Karma isn't so much about judging people as it is measuring their contributions.  I think you and I can find agreement on certain details, but until I see better arguments against karma in principle I'll continue to see value.

"Better arguments" includes not just tearing down the status quo with personal opinions but in offering viable alternatives.  So what are they?  How do we ensure, for example, that the right people get loaned (or even free) devices for early development and testing?  How do we pick the right people to represent the community at various industry and social events?  If not some sort of karma system, then what is to be used for measurement?

Oh, and back to real life: in a workforce, we are rated against each other even when direct comparisons are invalid.  I'm entering one such review process right now with my employer.  It's flawed, but I'm stuck with it.  At least in this case people are willing to entertain alternatives and improvements.  ; )

Randy

> 
> 
> On 13/01/2010, Randall Arnold <texrat at ovi.com> wrote:
> >
> >
> >
> >
> >> ----- Original message -----
> >> From: "Sebastian 'CrashandDie' Lauwers" <crashanddie at gmail.com>
> >> To: "List for community development" <maemo-community at maemo.org>
> >> Subject: Re: Sprint task: Refine the karma system
> >> Date: Wed, 13 Jan 2010 00:33:07 +0000
> >>
> >>
> >> Valerio,
> >>
> >> Thanks for putting this discussion up.
> >>
> >> Now, I'm not a big supporter of karma at all. I believe it is a flawed
> >> system which encourages cheating and ego-trips, and neither of those
> >> have citizenship in this community.
> >>
> >> There is no moderation on the discussion lists, which means anyone who
> >> even pollutes the lists receives karma. Same thing for tmo, a lot of
> >> redundant discussions mean a lot of redundant karma. At this point,
> >> trolling is the best, effortless way of amassing heaps of karma. Sure,
> >> off-topic doesn't count anymore, but I have yet to see (non-spam)
> >> threads be deleted instead of locked in most other fora.
> >>
> >> Rewarding app developers is a great and noble idea, however I honestly
> >> doubt karma is the most appriopriate medium for it. More than
> >> anything, I have to side with Dave and notice that this implementation
> >> is highly developer-focal.
> >>
> >> If I were crazy enough, I would post the suggestion that karma be
> >> removed as a whole. If someone is active on tmo, you can see this by
> >> his number of posts, and the thanks ratio. If someone offers well
> >> formulated and mature advice, it should ring with your own mind. If a
> >> dev writes a new app every week, I hope the website and app manager
> >> will allow to "Browse by developer", and maybe the dev's stats will
> >> indicate he has contributed positively on a number of occasions. The
> >> real important thing is that if the main argument *for* karma is that
> >> it allows newcomers to *know* who is *good*, and this is also
> >> extremely flawed.
> >>
> >> If there is a worry that false information may be spread by unsavy
> >> users, then this is a useless worry; a waste of time. People don't
> >> care about an unknown status on an unknown forum. They will care, and
> >> will listen to any person who echoes their thoughts and resonates
> >> their ideas. Whether that person is "true or false" doesn't matter,
> >> and neither does their karma rating.
> >>
> >> My main issue is that there is no such thing as karma in real life, so
> >> why have any in our community? There is no formal rating of every
> >> single individual on the planet on a website like rottentomatoes,
> >> where people can check if they should agree or listen to this
> >> politician or this sportsman; why would our community rating have
> >> something which is, by lack of better words and lack of proof
> >> impossible to create.
> >>
> >> I like politician X, but how does he match up against the NFL Top 10?
> >>
> >> Thanks for reading, and apologies about the rant.
> >>
> >
> >
> > And I continue to believe the concerns over karma abuse are significantly
> > overblown.
> >
> > Worries about cheating and abuse assume that such activities represent the
> > norm or at the very least a dangerously high percentage of all activity.
> > That defies probability and even human social behavior. I don't have proper
> > data to perform an analysis here but I'm curious: what do karma opponents
> > guess the abuse percentage to be?
> >
> > Karma serves a valuable purpose: it's a value system that strives to put
> > various contributing activities on a level playing field and make them
> > highly visible. Of course it's not perfect, and that would be an irrational
> > avenue to argue down anyway given that nothing can be.
> >
> > Comparing a microcosm like maemo.org to real life is disingenuous as well,
> > but if you want to go down that path one can make arguments that there ARE
> > real-life rating systems, job salary being one. Educational grades are
> > another, etc etc etc.
> >
> > But while I don't quibble with the concept of karma, I can understand the
> > distrust of some formulae and applications. I believe we need a karma
> > system, and I also believe we need to monitor it for success as well as
> > abuse... and continually refine it as needed.
> >
> > Randy
> >
> >>
> >> On 12/01/2010, Attila Csipa <maemo at csipa.in.rs> wrote:
> >> > On Tuesday 12 January 2010 20:46:14 Valerio Valerio wrote:
> >> >> All suggestions/improvements are welcome, but please keep in mind that
> >> >> the
> >> >> karma should be simple to calculate.
> >> >
> >> > Brainstorm seems to be missing (maybe the generic vote/comment score can
> >> > apply, but surely proposals/solutions themselves are worth something ?).
> >> >
> >> > The current scoring scheme does not take into account developers who
> >> > don't
> >> > develop end-user software (like libraries). No idea how to honor that
> >> > except
> >> > for download/spike counting.
> >> >
> >> > Also, you don't mention bugs.maemo.org, with more karma for apps it's
> >> > somewhat
> >> > better (reporting 3-5 bugs, regardless of report quality was roughly
> >> > equal
> >> > to
> >> > writing (!) an app). Anyhow, if possible, maybe it would make sense to
> >> > include
> >> > bug status (i.e. karma for developers fixing the bugs in question, or no
> >> > karma
> >> > for duplicate, invalid, etc bugs). Yes, this can be tricky with projects
> >> > handling their own bugs, but then again, it's the same now - you get
> >> > karma
> >> > only for reporting bugs on projects in b.m.o.
> >> >
> >> > Overall, I feel maybe the new proposal is noticeably tilted towards
> >> > developers, non-dev community members will very likely have difficulty
> >> > 'keeping
> >> > up' with dev karma (which may or may not be what you want).
> >> >
> >> > Regards,
> >> > Attila
> >> >
> >> > _______________________________________________
> >> > maemo-community mailing list
> >> > maemo-community at maemo.org
> >> > https://lists.maemo.org/mailman/listinfo/maemo-community
> >> >
> >>
> >> --
> >> Sent from my mobile device
> >>
> >> question = ( to ) ? be : ! be;
> >> -- Wm. Shakespeare
> >> _______________________________________________
> >> maemo-community mailing list
> >> maemo-community at maemo.org
> >> https://lists.maemo.org/mailman/listinfo/maemo-community
> >>
> >
> > --------------------------------------------------------------
> > Ovi Store: Download apps, games, videos and more
> > http://store.ovi.com/?cid=ovistore-fw-bac-na-acq-na-ovimail-g0-na-2
> >
> > _______________________________________________
> > maemo-community mailing list
> > maemo-community at maemo.org
> > https://lists.maemo.org/mailman/listinfo/maemo-community
> >
> 
> --
> Sent from my mobile device
> 
> question = ( to ) ? be : ! be;
> -- Wm. Shakespeare
> _______________________________________________
> maemo-community mailing list
> maemo-community at maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-community
>

--------------------------------------------------------------
Ovi Mail: Making email access easy
http://mail.ovi.com