[maemo-developers] [maemo-developers] N800/Newton comparison

[From: Daniel Elstner]

Am Freitag, den 02.02.2007, 10:43 -0500 schrieb Sean Luke:
> Daniel Elstner wrote:
> 
> > However, I think he's wrong in picking on GTK+ as the culprit for some
> > of the UI annoyances.  For instance, I don't think it's true that GTK+
> > only deals with simple input one character at a time.  I'm pretty sure
> > it's possible to pull off Newton-style handwriting as described by him
> > with a sufficiently sophisticated GTK+ input module.
> 
> I'd be glad to modify the article (and have) in response to  
> corrections, and actually HWR is rather low on my list of complaints  
> in the article.  But no, the Newton's facilities, so far as I can  
> tell, cannot be reasonably replicated with a plug-in input module,  
> except for providing better recognition perhaps.  But I'm not losing  
> sleep over it.  Though GTK is a classic least-common-denominator  
> framework, it does allow lots of apps to be ported to the N800  
> immediately, and that's a worthy tradeoff.

I'd call wxwidgets a classic common-denominator toolkit -- not GTK+.
But I realize you're probably considering this from a high level point
of view.  I don't have any experience with the Newton UI whatsoever.

>   My article tried to focus  
> on things they _could_ change without performing open heart surgery  
> on a large open-source framework.  HWR was low on the list.

OK, that was just one example.  Another is that GTK+ (with Pango) has
really sweet support for Unicode and complicated scripts.  The comment
that "GTK+ supposedly handles Unicode" makes it sound as though its
support were rather limited or not working.

> The Newton's HWR has three relevant characteristics for discussion here:
> 
> 1. Interpreted handwriting is rarely handled by the underlying  
> application through the event system, much less through a keystroke  
> event facility; and in fact an application typically doesn't know or  
> care that text is being written in one of its fields.  It just shows  
> up.  You *can* override this facility to shove text at an application  
> through key events (Newtons have keyboards) but it's not the default  
> mechanism.  An app could also receive text in whole paragraphs if it  
> liked, and that text could include embedded pictures and  
> intentionally uninterpreted handwriting in the form of stroke  
> sequences (they're called "rich strings").
> 
> 2. Handwriting happens directly in text fields and text areas, not in  
> a separate pop-up region.
> 
> 3. The recognition is sophisticated.
> 
> #3 is doable on the Nokia with a replacement module.  And of course  
> #1 and #2 be done inside an application with a writer who is  
> sufficiently talented (or demented).  But it's besides the point.   
> For #1 and #2, the other applications won't _automatically_ use that  
> app writer's handiwork, because GTK's framework [probably] doesn't  
> support it.  Each app would have to code for and link against his new  
> library.  But the whole _point_ of a GUI is to provide *all*  
> applications with a powerful and consistent framework so they don't  
> have to do tons of work writing their own (different from one  
> another) approaches to user experience.
> 
> Note: I wrote [probably] because as best I can tell GTK apps, like  
> most UI systems originally created on PCs, routes text in the form of  
> key events.  But I'm not positive about this and wouldn't mind being  
> corrected.

Alright then.  Text input in GTK+ is handled via loadable input modules,
transparent to the application:

http://developer.gnome.org/doc/API/2.0/gtk/GtkIMContext.html

It'll probably require some hackery to make the handwriting recognition
send status tokens or something by means of faked key press events, but
from then on things can be handled by the input module.  The application
will get plain text from it in any case.

If someone is really willing to have a go at this, it would of course
make sense to enhance the GTK+ API a little to make this kind of thing
possible without any ugly hackery.  This is Open Source, after all.  The
point remains that it should be possible to implement this without
changing the applications themselves.

Regards,
--Daniel