automated unit testing will not find all bugs or problems in perfectly packages software.<br>manual human testing will not find all bugs or problems in perfectly packaged software.<br><br>a combination of both involving common sense and prior history should minimize the risk of problems though.<br>
<font color="#888888">
<br>gary</font><br><br><div class="gmail_quote">On Mon, May 11, 2009 at 10:42 AM, Jeremiah Foster <span dir="ltr"><<a href="mailto:jeremiah@jeremiahfoster.com">jeremiah@jeremiahfoster.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im"><br>
On May 10, 2009, at 2:02, Graham Cobb wrote:<br>
<br>
> On Tuesday 05 May 2009 16:30:41 Quim Gil wrote:<br>
>> ext Jeremiah Foster wrote:<br>
>>> On May 5, 2009, at 15:41, Quim Gil wrote:<br>
>>>> - We need to know when an application in extras-testing is ready<br>
>>>> for<br>
>>>> end<br>
>>>> users.<br>
>>><br>
>>> After going through a policy check and two weeks of power users'<br>
>>> tests?<br>
>><br>
>> 2 weeks minimum and 10 testers OK with no blockers?<br>
>><br>
>> We can fine tune how many days and how many testers are required.<br>
><br>
</div><div class="im">> I agreed with most of Quim's and Jeremiah's points until this one.<br>
><br>
> Absolutely, definitely not!! A specified time and a number of<br>
> testers is not<br>
> a reasonable or practical way to decide when the application is<br>
> ready for<br>
> promotion: a human being needs to make a judgement.<br>
<br>
</div>Unfortunately, having a human decide if an app is ready for promotion<br>
is fraught with problems, making the human promotion process nearly<br>
impossible.<br>
<div class="im">><br>
> For a fairly popular app like GPE (and particularly for very popular<br>
> apps like<br>
> Canola), 2 weeks and 10 people is not be enough. I released a Beta<br>
> version<br>
> of GPE months ago and have not promoted it to Extras because I know<br>
> of one<br>
> problem which is potentially serious and I haven't fixed it yet --<br>
> people are<br>
> better off sticking with the previous version unless they are<br>
> willing to act<br>
> as beta testers. But none of the beta testers has been able to log a<br>
> reproducible report (which is why I haven't been able to fix it --<br>
> but it<br>
> occurs enough to clearly be a real problem).<br>
<br>
</div>Don't you think you should label packages as a "beta" in the version<br>
number so that potential downloaders are aware that this is a problem?<br>
There will be a mechanism to allow packages to stay in testing and not<br>
get automatically promoted.<br>
<div class="im"><br>
> On the other hand, a small application may only have 20 users so<br>
> finding 10<br>
> beta testers may be impossible!<br>
<br>
</div>This is why it needs to be an automated process. _Every_ package needs<br>
to be tested.<br>
<div class="im"><br>
> And 2 weeks may be much too long to wait if the update is a security<br>
> update or<br>
> fixes a serious bug or provides a much-needed library that is<br>
> blocking other<br>
> packages.<br>
<br>
</div>Debian fortunately has a mechanism for promoting packages more quickly<br>
if there is a security problem. I think incorporating this ability to<br>
quickly promote packages might be a good idea.<br>
<div class="im">><br>
> The bottom line is that I do not believe an automatic promotion is<br>
> possible:<br>
<br>
</div>There is no other way. Here's why;<br>
<br>
1. Too many packages - humans don't scale well.<br>
2. The promotion policy needs to be tested by an automated process<br>
with proper regression and unit tests - this is a standard part of<br>
professional quality assurance, there is no reason that maemo<br>
shouldn't have the same professional standards.<br>
3. The process should be codified and clear - if you build a well-<br>
made package, if it passes all the requirements, you get promoted.<br>
This eliminates favoritism and any biases and rewards meritocratic<br>
adherence to the packaging and development policy.<br>
<div class="im"><br>
><br>
> there should be a set of people who have the privilege to do the<br>
> promotion<br>
> and the submitter should request the update and have to persuade one<br>
> of the<br>
> set of people to do the promotion.<br>
<br>
</div>Who will do this? How long will they stick around? What happens if<br>
they like one type of package and not another? How long will it take<br>
to decide who does this and the process of choosing them?<br>
<div class="im"><br>
> 2 weeks and 10 testers may be a guideline<br>
> but we should trust the people who are given upgrade privileges to<br>
> make the<br>
> decision.<br>
<br>
</div>Maemo does and will continue to do so, but will also run some tests on<br>
new software so Maemo can assure users (who ought to always be the<br>
focus) that the software will work as advertised.<br>
<font color="#888888"><br>
Jeremiah<br>
</font><div><div></div><div class="h5">_______________________________________________<br>
maemo-developers mailing list<br>
<a href="mailto:maemo-developers@maemo.org">maemo-developers@maemo.org</a><br>
<a href="https://lists.maemo.org/mailman/listinfo/maemo-developers" target="_blank">https://lists.maemo.org/mailman/listinfo/maemo-developers</a><br>
</div></div></blockquote></div><br>