Hey there I&I multitouch folks!

So we are coming up on another week's class, and it would be great to coordinate our work and progress.  I have included in this email those people who've listed themselves under "Multi-touch" and "Blob detection/Software infrastructure" groups.

Here's a progress update, a call to action, investigation of issues we face, and a call for some pizza and hacking!

== Hardware! ==
Today, the table LED construction will be complete.  We will possibly have the opportunity to test projection on it today; otherwise, this can take place tomorrow morning in preparation for a finished tech demo.  Further hardware progress will involve experimentation with stick-on FTIR gadgets, and non-flat surfaces.  But none of this is usefull without some sweet sweet...

== Software! ==
Once the hardware is finished, the name of the game is software.  The BIGGEST THING at this point in the multitouch community, and at this point in our project (how fortuitous!) is standardization, and there are two things that need to be standardized: the software stack and gestures.

== Standardizing: Software stack ==
Touchlib should, no doubt, be the basis of our software stack.  It is performant and has some great development inertia behind it.  Additionally, there are several folks here who are familiar with its code base (with commit rights, even!).  (To be blunt, VVVV and processing are largely toys; blob tracking is computationally intensive, and it belongs in a high-performance low-level library.  Touchlib solves this; use of touch data in high-level environments should be achieved by publishing Touchlib event data upwards as we'll get into in just a bit : ) 

C++ Hackers: Focus on helping Touchlib to build cross-platform (Win32, OSX, Linux) would be immensely helpful.  See the White Noise blog [1] for more information on current progress.  Patrick, can you link the CSH guys' blog on the [[Multi-touch]] page on our wiki?  Co-ordinating development efforts is definitely the way to go :D  Can anyone speak for the current development of Touchlib on Linux and OSX?  I get the feeling this is nearly done, but can't personally confirm this.  Folks?

Flash Hackers: One current good software stack is Touchlib -> FLOSC -> Flash 9.  This is good for getting demos going now, and I think it would be great if one or two folks could investigate this stack (learn the ins and outs of FLOSC, share these findings tomorrow?  I can help if need be.) and get a concrete demo with this stack for tomorrow (either live, sourced from a camera, or canned, sourced from a video file).  A simple Pong game, a paintbrush app, or a slider/knob UI would be a nice proof-of-concept.  Check out the Touchlib author's blog [1] for information on his progress with bridging Touchlib to Flash 9 for the photo booth application and more.

See [2] for some more sweet videos.

(Danger!  Here lie ideas, partially-formed and hungry for discussion!)

However, one issue with this architecture is that the gesture recognition is done in Flash, which is an unnecessary dependency and too high-level to be generally useful.  Consider the fact that we would like to write multitouch applications in a variety of environments such as Flash, Java, Apollo, and native Win32/OSX/Linux applications and that we don't want to rewrite gesture recognition for each of these.  Additionally, we would like to have applications written in different languages coexisting in the same multitouch-enabled system, and we realize that gesture recognition should be performed outside of an application, as a service.

Getting into the nitty-gritty of such an architecture: Touchlib provides raw touch data.  Ideally, the window manager should associate each touch (both clicks and drags) with a particular application.  Specifically, it seems logical that the window in which a touch event begins is the window with which it is associated.  This is the Touch Assignment layer [4] [5].

Additionally needed is a Gesture Recognition layer.  This can come in the form of an independent server (like FLOSC) that accepts binary- or XML-encoded TUIO events representing touches, and publishes gestures in a format that is not yet standardized (see next section).  See a first stab at such an architecture: [6] - please give feedback!

== Standardization: Gestures ==
See [3] for a stab at gesture recognition.  We should work toward supporting those gestures where possible.  If we find that they are in need of improvement, then we should coordinate such a discussion with the community - sharing helps everyone!  Also needed is a software standard for publishing and describing these gestures, like TUIO/OSC does for touches.

== Call to Action! ==
So, we've touched on (pun intended) current progress, a short-term plan for a full-stack proof-of-concept (table+touchlib+flosc+flash = fun mini-application, can we have it running tomorrow?), and an investigation of the issues we're currently looking at.  If folks have been quietly working on such a demo, please let us all know :D  I am available to help in any way I can.  Please speak up if you are capable & interested!

On top of this, I think it's crucial to get all of us together for some meeting/hacking/pizza action; there is a lot of information to wade through, and this is best done in person, with whiteboards, hardware, and snacks aplenty.  Let's plan to meet up once or twice between now and the end of the weekend for a solid block of time.  Pizza in one of the labs (if that's kosher) sounds great to me.  You?

I look forward to us building some kick-ass stuff that dovetails with the larger course projects over the next few weeks!  See you all tomorrow,

Jason

[1] - http://www.whitenoiseaudio.com/blog/
[2] - Mike Wags' blog has some cool software vids - http://mikewags.blogspot.com/
[3] - Gesture Standards - http://nuigroup.com/forums/viewthread/95/
[4] - http://synchala.dreamhosters.com/wiki/index.php?title=Image:TouchAssignment1.png
[5] - http://synchala.dreamhosters.com/wiki/index.php?title=Image:TouchAssignment1.png
[6] - http://jayunit.net/files/ftir/multitouch-software-arch.pdf