[maemo-community] Application Karma

[From: Patrik Hirvinen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Simon Pickering wrote:
> I'd be tempted to let the application karma last until the next software
> release (for a given device), as this is really the time frame over
> which an application is used/useful. Then with the next release,
> assuming the dev releases a new version (good way to motivate people ;)
> they will start getting new karma for the new release.
> 
> I suppose this is not ideal, as people will have a see-saw pattern for
> their karma, so perhaps the karma for an old (as in no longer the
> current) release should decay (reasonably quickly) as soon as a new
> release is made, to smooth the saw-tooth pattern.

Application versions are stored and removing faves on version update
would make little sense as it would disincentivise releasing new
versions due to the sudden drop in visibility for an app. Perhaps
progressively decrease the weight of older faves with ignorance at x
days(30?) before the date of last release or something along those lines?

Dave Neary wrote:
> Hi,
>
> Niels Breet wrote:
>> When using thumbs up/down, how would we show karma for the application.
>> How to make it meaningful? Will this replace the stars?
>
> I would say no.
>
> In fact, I wouldn't even have thumbs up/down - stars are great. Thumbs
> would be duplication.
>
> To my mind, application karma is a measure combining various things:
>  * How many people are using the application? (more people = higher karma)
>  * How well maintained is the application? (more recent releases =
> higher karma)
>  * What is the rate of take-up of the application? (more recent
> downloads = higher karma)
>  * How highly do users of the application rate it? (more stars = higher
> karma, more users rating = more karma)


The idea presented at https://bugs.maemo.org/show_bug.cgi?id=2179#c11
was to simplify things by moving to just faves and burys and use that to
replace the stars' function to decide what's shown on the downloads
front page ( http://maemo.org/downloads/OS2008/ )in the section "Stars"
and on the stars page ( http://maemo.org/downloads/bestrated/OS2008/25/
) There are already sections for all-time popular and new apps, plus the
featured ones. This would be used to determine what's hot right now.

>
> The difficulty is getting objective measures and combining them to make
> sense. The "Activity" metric on Sourceforge tries to do this, as does
> the "Popularity" and "Activity" metrics on Freshmeat.
>
> Some examples:
>
> An application which is newly released, has a high download rate, and
> very high user ratings, should have a high karma rating.
>
> An application which has not been updated since Chinook, but whose last
> release was both highly rated and popular, will maintain a high karma
> for another release or so. A similar application, updated for Diablo,
> but which is not as popular (downloads, rating) could probably attain a
> higher karma.
>
> A formerly popular application which has no maintainer will have a low
> karma.
>
> A new application with average download rates, and average ratings, will
> have average karma.
>
> An application that regularly brings out new versions will have a higher
> karma than one that updates only once per platform release.
>
>> Another subject is how do we implement karma decay? For news items karma
>> decays to zero after a week orso and will then disappear from the news
>> page. How would this work for applications on the downloads front page?
>> Can people fav an application again after a certain time as suggested in
>> [1].
>
> I would degrade karma by release. The last release will have a rating of
> 1. N releases ago could have a multiplier effect of 1/N. An application
> which was downloaded by 20,000 people for Chinook equals a release
> downloaded by 10,000 for Diablo.
>
> I would have stars be a multiplier.
>
> I would have a mixed metric for downloads, if we can isolate downloads
> of a specific platform release of the application, and if we can get
> recent download stats. Download multipliers are essentially arbitrary -
> and they could be a sqrt() relationship (something like
> sqrt(downloads)/10 or something) - I guess we need to just pick figures
> that give us a nice distribution curve of karma for apps, similar to
> profile karma.
>
> For example (if we have the data):
>
> K(app) =
>  stars(app) *
>    (
>      number of ratings
>    +  sum (N=1..3 for last 3 platform releases)
>            sqrt(downloads(app(N)))/(N * 10)
>    + (number of releases in past 12 months)*4
>    + (downloads last month)/10
>    )
>
> So, let's take a real life example: Canola2. And let's make imaginary
> numbers, since we don't have all the numbers for downloads (we could go
> to 4 releases or 5, since the divisor gets more & more significant, I
> don't think it'll make a huge difference)...
>
> stars: 3.5
> Ratings: 103
> Downloads(Diablo): 114539
> Downloads(Chinook): 0
> Downloads(Bora): 0
> Releases in past 12 months: 0 (am I looking in the wrong place?)
> Recent downloads: ~1000 (source: dowloads page - approximation)
>
> Maybe I picked a bad example, since Canola 2 is only available for
> OS2008, and the garage project doesn't appear to have all their
> releases, but anyway...
>
> Karma: 3.5 * (103 + sqrt(114539)/10 + (1000/10) + 0)
>  = 3.5 * (103 + 33 + 100)
>  = 3.5*236 = 826
>
> Some remarks:
>
> To ensure that one measure doesn't swamp out the others, this needs
> tuning. In this case, 360 of the 480 karma comes from the decent star
> rating and 103 people having rated the application. Maybe it makes more
> sense to have the downloads karma just be a sqrt()? For 10K downloads,
> this would be giving the application as many karma points as 100 star
> ratings. 1K downloads in the past month would give the same karma.
>
>> Ratings are now also used to add _user_ karma. How would favs and buries
>> affect user karma?
>
> Faving & burying is a low-commitment community act, and should be
> weighted appropriately. Same as fav & bury of blogs. sqrt().
>
> Cheers,
> Dave.
>

We don't keep data on amount of "recent" downloads. Unless what's really
wanted is to determine an overall score for the whole lifetime of the
application instead of just how hot it's now, this seems overly
complicated. It's difficult to assess what would be a fair or sensible
formula for the first purpose without looking at what kind of scores it
would give for a large amount of typical applications.

- --
Patrik Hirvinen
patrik.hirvinen at nemein.com
+358-(0)40-7307117
http://nemein.com/fi/people/hirvinen/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREDAAYFAkmkJlsACgkQiCIKn84rIOhIigCffW1rIgBWsfAIh1MFzSISp3aG
iNEAn0zCx7XS+i/rK22G+gx7eAj1q8f2
=I+aU
-----END PGP SIGNATURE-----