LusidOSC v1.1

LusidOSC v1.1 -- STILL IN PROGRESS...

LusidOSC v1.1

LusidOSC is an open initiative to create protocol layer for unique spatial input devices using Open Sound Control. A unique spatial input device is any system that uniquely identifies objects (markers, tags, regions, fingertips, etc.) in physical space. LusidOSC extends ideas originally presented in the TUIO protocol (developed by Kaltenbrunner, Bovermann, Bencina, and Costanza and described in TUIO: A Protocol for Table-Top Tangible User Interfaces in 2005) but has been redesigned to enable scalability across a wide range of sensing platforms (senseTable, reacTable, trackMate, Nabaztag, g-speak, etc.) without the need for additional low-level OSC profile definitions. In particular: many parameters are now sent as strings, allowing them to be of variable lengths; timing information enables applications to handle time-based calculations; and all TUIO profiles have been simplified into a single profile for better library support and application compatibility. As standards emerge, there is a good possibility that additional parameters will be able to fit the format described here, easing the transition to future versions of LusidOSC.

LusidOSC allows for flexibility within a single, standardized profile rather than requiring the creation of a new profile for each type of tracking system. This design choice enables LusidOSC libraries to remain entirely separate from the tracking system while also enabling functionality by default with any LusidOSC-compliant application. For example, tracking systems and applications are not required to have the same dimensionality to function (3D positions can be used in a 2D application, and 2D positions can be manipulated as 3D data on a surface). If, however, a large number of applications emerge that are clearly segmented based on particular tracking technologies or capabilities, future LusidOSC versions could support multiple profiles (as TUIO does), but only insofar as they are necessary, thus keeping library support and application integration as strong as possible.

Information about each sensed object is broadcast to applications via OSC (an abstract layer on top of UDP) and provides eight components that define it in the physical world: its uniqueID (u); position (x, y, z); rotation (a, b, c); and time (s.m). Regardless of the sensing platform's capabilities, every object message has the same fundamental data structure (as well as space allocated for additional platform-specific data if available), thus allowing any LusidOSC-enabled application to function with any spatial sensing platform.

Changes in LusidOSC v1.1 include(??):

    - a general data structure that allows for data-typing, with all corresponding data being variable-length. This makes for an extremely flexible and scalable way to handle additional interaction information while keeping parsing simple.

    - (TBD?) tracking system geometry information. Since all position information is in real distance (millimeters), it's helpful to be able to send along information about the max/min range of values to obtain percentages used by some applications.

    - sub-millimeter precision. X,Y,Z now specified in 1/10 of millimeter. This is important for high-resolution interfaces such as drawing tablets. (maybe just use floats and keep it in mm??)

    - OSC the other way, too (So that client can tell server to send it data). --> subscriber model (also over OSC); request data for the next 60 seconds.
        - client sends something like:
            /lusid/1.1 subscribe ip/hostname(as string) port(int32)
        - clients recommended to request about every 30 sec.

Message Parameters

parameter	OSC type	notes
u	string	uniqueID; an object's ID written in hex format. e.x., "0x123456". A generated ID can be given to non-ID'd things (such as fingers).
x, y, z	int32	3D position; represented as tenths of millimeters with (0,0,0) at the center. The axis are arranged with positive X to the right, positive Y to the top, and positive Z pointed upwards.
a, b, c	float32	angle around the x, y, and z axis accordingly; range is [0, 2PI].
e	string	short human-readable name of object type
t	int32	type of data (see: Data Types)
d	string	data; any string, or null string if no data. Commas are the suggested delimiter if multiple values are being sent as string.
f	int32	frame number
s	int32	seconds since midnight Jan. 1, 1970 (UTC); a.k.a, Unix time.
m	int32	microseconds (decimal part of seconds only)

Message Formats

/lusid/1.1 fseq f s m
/lusid/1.1 set u x y z a b c e [(t d) pair for every additional piece of data]
/lusid/1.1 alive [list of every active uniqueID]

Message Order

LusidOSC messages are encapsulated in an OSC Bundle, and the order of messages within the bundle is important. The "fseq" message is transmitted first, as it contains information about how to compute time-sensitive calculations for object data. After the "fseq" message, all "set" messages are sent. Finally, the "alive" message is sent to help manage object lists by removing messages corresponding to objects that are no longer in use. Although redundant for sensing systems that send "set" messages for each uniqueID every frame, the "alive" message serves well in complex systems that sense different object types at different frame rates to maintain consistent state with their applications.

Ports

Though the default port for LusidOSC messages is 3333 (one used by virtually all TUIO applications), users are not limited to this choice. Trackmate allows for connectivity to a range of ports, supporting many-to-one, one-to-many, and many-to-many [(spatial input device)-to-(spatial application)] configurations with very little additional overhead. Since each OSC message begins with a protocol name (such as /lusid/), ambiguity is eliminated between multiple protocols communicating on the same port. To be fully compliant with the LusidOSC specification, however, applications using a non-default port should still allow the user to specify the port (3333 being one of the possible selections) manually.

Data Types

All data (such as size, shape, position, color, etc.) is relative to the object at its center. The default data column shows the initial value that objects should have if they are not specified in the message. Objects should retain their last updated data values as long as they remain active (allowing lengthy data to be sent intermittently if desired).

color key:
red elements should not be moved or changed (they are set)
yellow elements are in the process of being developed (free to move)

type [t]	example data [d]	default data	description
0	"anything,you,want"	""	Ad-hoc data of any kind useful for working with emerging interface technologies. Eventually this data should be formalized as specific data types in subsequent versions of LusidOSC.
?	"tm"	""	OpenTagDB Family Name (see: http://www.opentagdb.com ). OpenTagDB will allow for unique mappings between an object's ID and data stored online.
?	"0xFFCC99"	""	Object RGB color (specified in hex)
?	"true"	"false"	Object is a finger
?	"true"	"false"	Object ID was assigned on-the-fly/randomly
?	"20"	""	Object radius in mm from (x,y,z) center.
?	"-1,-3,17,28,12,-5"	""	Object 2D shape polygon as (x,y) point pairs relative to (x,y,z) center.
?	"523"	"-1"	Finger Pressure (out of 1000).
?	"12,7,4,13,19,2"	""	Object 2D skeleton as (x,y) point pairs.
?	"12,-7,2"	"0,0,0"	Position velocity in mm/second for (x, y, z).
?	"0.231,0.0,-0.117"	"0,0,0"	Rotation velocity in radians/second about (x, y, z) axis.
?	"0.57"	"1.0"	Certainty factor (applied to position, id, rotation, etc?)




			A LOT OF WORK LEFT TO GO HERE TO DEFINE INITIAL DATA TYPES....

Implications

With the OSC protocol outlined in this document, any application that communicates via LusidOSC will be able to deduce uniqueID, position, and rotation information sent from any LusidOSC-compliant sensing or tracking system. In addition, applications that use supplemental data specific to particular tracking systems can check the encoding string (e) and process the corresponding data (d) accordingly.

Pseudo Code for a LusidOSC Sender (Tracking System)

do once at startup:
1. initialize OSC Sender to send packets to HOST, PORT.

each time new object data is ready to send:
1. create a new OSC bundle.
2. create and add the "fseq" message to the bundle.
3. create the "alive" message, but do not add it yet.
4. for each object being tracked:
4.1. add the object's uniqueID to the "alive" message.
4.2. create and add a "set" message for the object to the bundle.
5. add the "alive" message to the bundle.
6. send the bundle

C Code for a LusidOSC Sender (Tracking System)

//
// Released under GNU GPLv2; this program is free software.
// Copyright (c) 2008, Adam Kumpf
// Derived from openframeworks.cc code by Matthias Dorfelt.
// Based on classes by Martin Kaltenbrunner for reactivision.sourceforge.net.
// Built on top of Open Sound Control (OSC).
//

OscSender sender = new OscSender(); // create a new OSC sender

void setupOnce(){
sender.setup(HOST, PORT); // setup the sender for a host and port.
}

void objectDataUpdated(){
OscBundle oscBundle = new OscBundle(); // create a new OSC bundle

OscMessage fseqMessage = new OscMessage(); // create a new OSC message
fseqMessage.setAddress("/lusid/1.0");
fseqMessage.addStringArg("fseq");
fseqMessage.addIntArg(frameNumber);   // frameNumber :: Int32
fseqMessage.addIntArg(seconds);       // seconds since 1970 :: Int32
fseqMessage.addIntArg(microseconds); // microseconds :: Int32
oscBundle.addMessage(fseqMessage);    // add "fseq" message to oscBundle

OscMessage aliveMessage = new OscMessage(); // create a new OSC message
aliveMessage.setAddress("/lusid/1.0");
aliveMessage.addStringArg("alive");

OscMessage m = new OscMessage();       // create a reusable OSC message

for each object sensed by tracking system {
    m.clear(); // start with a new message
    m.setAddress("/lusid/1.0");
    m.addStringArg("set");
m.addStringArg(uniqueIDString);      // unique ID :: String (in hex format)
m.addIntArg(objectX);                // X Pos :: Int32
m.addIntArg(objectY);                // Y Pos :: Int32
    m.addIntArg(0);                      // Z Pos :: Int32
    m.addFloatArg(0);                    // Angle around X axis :: Float32
    m.addFloatArg(0);                    // Angle around Y axis :: Float32
    m.addFloatArg(objectAngleRadians);   // Angle around Z axis :: Float32
    m.addStringArg("tempTracker");       // Name of Data Encoding :: String
    m.addStringArg("120,cold,roundObj"); // Data :: String
    oscBundle.addMessage(m);             // add "set" message to the oscBundle
    aliveMessage.addStringArg(uniqueIDString); // add the ID to the "alive" list
}

oscBundle.addMessage(aliveMessage);   // add "alive" message to oscBundle

sender.sendBundle(oscBundle);         // send the oscBundle on its way!
frameNumber++; // increase frame for next time.
}

Pseudo Code for a LusidOSC Receiver (Application Side)

do once at startup:
1. initialize OSC Receiver to listen for packets on PORT.

each time a new OSC message is available to read:
1. check to see if it has the correct profile address (/lusid/1.0)
    1.1 check to see if it is a "set" message
      1.1.1 if frame is out of order, immediately return
      1.1.2 update object parameters based on message data
      1.1.3 if this is the first time we've seen this object
        1.1.3.1 add the object to our local objectList
        1.1.3.2 notify event listeners of new object
      1.1.4 else (we have seen this object before)
        1.1.4.1 notify event listeners that object changed
    1.2 check to see if it is an "alive" message
      1.2.1 if frame is out of order, immediately return
      1.2.2 for each uniqueID in the "alive" message
        1.2.2.1 add the uniqueID to a list for later
      1.2.3 for each object in objectList (we've seen via "set" messages)
        1.2.3.1 if object was not included in the "alive" list
          1.2.3.1.1 notify listeners of object removed
          1.2.3.1.2 remove the object from the objectList
    1.3 check to see if it is a "fseq" message
      1.3.1 if frame is older than previous, immediately return
      1.2.2 update frame timing information

C Code for a LusidOSC Receiver (Application Side)

//
// Released under GNU GPLv2; this program is free software.
// Copyright (c) 2008, Adam Kumpf
// Derived from ofxTuioClient.h for openframeworks.cc by Matthias Dorfelt.
// Based on classes by Martin Kaltenbrunner for reactivision.sourceforge.net.
// Built on top of Open Sound Control (OSC).
//

int lastFrame = 0;
int lastFrameSeconds = 0;
int lastFrameMicroSec = 0;

List<LusidObj> listOfLusidObj = new List<LusidObj>();

OscReceiver receiver = new OscReceiver(); // create a new OSC receiver

void setupOnce(){
receiver.setup(PORT); // setup the receiver for a host and port.
}

void oscMessageReceived(OscMessage m){
if(m.getAddress().equals("/lusid/1.0")){

    // handle a SET message
    if(m.getArgAsString(0).equals("set")){
      if(currentFrame < lastFrame){
return; // UDP packet was old, don't update anything
}else{
        // let's update our parameters from the object.
        String uniqueID = m.getArgAsString(1);
        int x = m.getArgAsInt32(2);
        int y = m.getArgAsInt32(3);
        int z = m.getArgAsInt32(4);
        float a = m.getArgAsFloat(5);
        float b = m.getArgAsFloat(6);
        float c = m.getArgAsFloat(7);
        String enc = m.getArgAsString(8); // encoding
        String data = m.getArgAsString(9); // additional data
        LusidObj lusidObj = listOfLusidObj.getObjectByID(uniqueID);
        if(lusidObj == null){
          // this is the first time we've seen this object.
          lusidObj = new LusidObj(uniqueID, x, y, z, a, b, c, enc, data);
          listOfLusidObj.add(lusidObj); // add the new object to the list.
          LusidOSCEvents.notifyDown(this, uniqueID, x, y, z, a, b, c, enc, data);
        }else{
          // we've seen this object before, it's just being updated/moved.
          lusidObj.update(x, y, z, a, b, c, enc, data);
          LusidOSCEvents.notifyMove(this, uniqueID, x, y, z, a, b, c, enc, data);
        }
}
    }

    // handle an ALIVE message
    if(m.getArgAsString(0).equals("alive")){
      if(currentFrame < lastFrame){
return; // UDP packet was old, don't update anything
}else{
        List<String> aliveObjectIDList = new List<String>();
        for(int i=0; i<m.getNumArgs(); i++){
          String uniqueID = m.getArgAsString(1);
          aliveObjectIDList.add(uniqueID);
        }
        for(int i=listOfLusidObj.length-1; i>=0; i--){
          // start at the end of the list and walk backwards...
          // we do this since we may delete elements and we don't
          // want to skip anything or try a bad index.
          LusidObj lusidObj = listOfLusidObj.get(i);
          if(aliveObjectIDList.doesntContain(lusidObj.uniqueID)){
            // we have an object that is no longer alive.
            // remove it and send an event.
            String uniqueID = lusidObj.uniqueID;
            int x = lusidObj.x;
            int y = lusidObj.y;
            int z = lusidObj.z;
            float a = lusidObj.a;
            float b = lusidObj.b;
            float c = lusidObj.c;
            String enc = lusidObj.enc; // encoding
            String data = lusidObj.data; // additional data
            LusidOSCEvents.notifyUp(this, uniqueID, x, y, z, a, b, c, enc, data);
            listOfLusidObj.remove(i); // now remove it from our list.
          }
        }
      }
    }

    // handle a FSEQ message
    if(m.getArgAsString(0).equals("fseq")){
      // first, save last frame before we overwrite it.
      // this is helpful for derived timing calculations.
      lastFrame         = frame;
      lastFrameSeconds = frameSeconds;
      lastFrameMicroSec = frameMicroSec;

      // now update things to the new information.
      frame         = m.getArgAsInt32(1);
      frameSeconds = m.getArgAsInt32(2);
      frameMicroSec = m.getArgAsInt32(3);
    }

}
}

For libraries and sample applications visit: http://lusidosc.sourceforge.net/