[maemo-community] Application Karma

[From: Dave Neary]

Hi,

Niels Breet wrote:
> When using thumbs up/down, how would we show karma for the application.
> How to make it meaningful? Will this replace the stars?

I would say no.

In fact, I wouldn't even have thumbs up/down - stars are great. Thumbs
would be duplication.

To my mind, application karma is a measure combining various things:
 * How many people are using the application? (more people = higher karma)
 * How well maintained is the application? (more recent releases =
higher karma)
 * What is the rate of take-up of the application? (more recent
downloads = higher karma)
 * How highly do users of the application rate it? (more stars = higher
karma, more users rating = more karma)

The difficulty is getting objective measures and combining them to make
sense. The "Activity" metric on Sourceforge tries to do this, as does
the "Popularity" and "Activity" metrics on Freshmeat.

Some examples:

An application which is newly released, has a high download rate, and
very high user ratings, should have a high karma rating.

An application which has not been updated since Chinook, but whose last
release was both highly rated and popular, will maintain a high karma
for another release or so. A similar application, updated for Diablo,
but which is not as popular (downloads, rating) could probably attain a
higher karma.

A formerly popular application which has no maintainer will have a low
karma.

A new application with average download rates, and average ratings, will
have average karma.

An application that regularly brings out new versions will have a higher
karma than one that updates only once per platform release.

> Another subject is how do we implement karma decay? For news items karma
> decays to zero after a week orso and will then disappear from the news
> page. How would this work for applications on the downloads front page?
> Can people fav an application again after a certain time as suggested in
> [1].

I would degrade karma by release. The last release will have a rating of
1. N releases ago could have a multiplier effect of 1/N. An application
which was downloaded by 20,000 people for Chinook equals a release
downloaded by 10,000 for Diablo.

I would have stars be a multiplier.

I would have a mixed metric for downloads, if we can isolate downloads
of a specific platform release of the application, and if we can get
recent download stats. Download multipliers are essentially arbitrary -
and they could be a sqrt() relationship (something like
sqrt(downloads)/10 or something) - I guess we need to just pick figures
that give us a nice distribution curve of karma for apps, similar to
profile karma.

For example (if we have the data):

K(app) =
 stars(app) *
   (
     number of ratings
   +  sum (N=1..3 for last 3 platform releases)
           sqrt(downloads(app(N)))/(N * 10)
   + (number of releases in past 12 months)*4
   + (downloads last month)/10
   )

So, let's take a real life example: Canola2. And let's make imaginary
numbers, since we don't have all the numbers for downloads (we could go
to 4 releases or 5, since the divisor gets more & more significant, I
don't think it'll make a huge difference)...

stars: 3.5
Ratings: 103
Downloads(Diablo): 114539
Downloads(Chinook): 0
Downloads(Bora): 0
Releases in past 12 months: 0 (am I looking in the wrong place?)
Recent downloads: ~1000 (source: dowloads page - approximation)

Maybe I picked a bad example, since Canola 2 is only available for
OS2008, and the garage project doesn't appear to have all their
releases, but anyway...

Karma: 3.5 * (103 + sqrt(114539)/10 + (1000/10) + 0)
 = 3.5 * (103 + 33 + 100)
 = 3.5*236 = 826

Some remarks:

To ensure that one measure doesn't swamp out the others, this needs
tuning. In this case, 360 of the 480 karma comes from the decent star
rating and 103 people having rated the application. Maybe it makes more
sense to have the downloads karma just be a sqrt()? For 10K downloads,
this would be giving the application as many karma points as 100 star
ratings. 1K downloads in the past month would give the same karma.

> Ratings are now also used to add _user_ karma. How would favs and buries
> affect user karma?

Faving & burying is a low-commitment community act, and should be
weighted appropriately. Same as fav & bury of blogs. sqrt().

Cheers,
Dave.

-- 
maemo.org docsmaster
Email: dneary at maemo.org
Jabber: bolsh at jabber.org