CORE Message API Ideas

Tom Wambold <taw38@drexel.edu>

Abstract

Interacting with the CORE daemon from external processes is a pretty powerful capability for us.  An IPC API allows for language-agnostic extensions/visualizations/etc.  An improved message format and communications protocol could enable a wide variety of tools to interoperate with CORE.

Issues

• Impossible to have multiple simultaneous asynchronous outstanding messages awaiting responses.

• No means to track which response is to what message, except with the Execute message’s execution number TLV.

• Difficult to write a single-threaded program that uses the API

• Stuff like receiving all the nodes currently in a session, client has no idea how many messages it will receive before it can continue.
• For example, a program that wants to get the emulation state to do some sort of reporting doesn’t know when the daemon has sent it all of the information

• Configuration messages require client to keep a lot of state.

• Configuring the “basic_range” mobility model requires setting range, bandwidth, jitter, delay, and error values all at once.  Can’t just set one.
• This means client needs to know all 5 values in order to change one.
• I don’t think there is any means for querying for current configurations from the daemon.

• Can’t create new nodes without knowing the node IDs of all other nodes in the emulation.

• Need to specify ID in create message.
• So, clients need to query and parse all current nodes.
• Daemon should be able to assign new ID if not specified

• Can’t attach arbitrary data to nodes.

• Some visualizations/GUIs might want to add stuff such as roles (squad leaders, etc.)
• Not all GUIs would know how to understand every attribute (icons, etc.).

Ideas

• Use a new protocol and message format.
• Protocol: HTTP

• We’ve successfully used embedded HTTP servers in both C and Python programs in order to work with almost any other language.

• In particular -- JavaScript

• HTTP client libraries exist for almost every language
• HTTP embedded servers are super easy in Python...

• Built-in: (http://docs.python.org/library/basehttpserver.html)
• Also, CherryPy (http://cherrypy.org/) is a really interesting and easy way for hooking up Python functions to particular URLs

• Also pretty easy in C

• libmicrohttpd (http://www.gnu.org/software/libmicrohttpd/)

• Message responses can be directly sent in HTTP response.

• Message format: JSON

• Again, really easy to read/write from Python
• Good, mature libraries for tons of languages
• Less efficient than a packed binary message, but I don’t think that efficiency is all that important for this sort of message.

• I could be wrong though, we could run CORE on the tactical edge!

• Other formats could be useful too, though I know less about them: i.e. Google protocol buffers.

Examples

• /session

• GET: Returns information about active sessions

• {
          “sessions”: [
                  {
                          “id”: 1234,
                          “name”: “foobar”,
                          “state”: “configuration”,
                  },
                  ...
          ],
  }

• POST: Create a new session

• Don’t need to specify ID number, daemon would choose the next open one
• POST /session {
          “name”: “hello”,
          “user”: “tom”,
  }
• Would return the session object including the ID number

• /session/<ID NUMBER> - Get/set settings for particular session

• No need to join sessions
• New sessions can created by POSTing to a different number
• GET: Returns info about that session

• GET /session/1234 {
          “id”: 1234,
          “name”: “foobar”,
          “nodes”: { 0, 1, 2, 5, 8, 36},
          “state”: “runtime”,
  }

• POST/PATCH: Update settings on a particular session

• POST /session/1234 {
          “state”: “datacollect”,
  }
• Returns complete session object including changes.

• DELETE: Deletes (closes) a session

• /session/<ID NUMBER>/node

• GET: Return info about all nodes

• GET /session/1234/node {
          “nodes”: [
                  {
                          “id”: 0,
                          “name”: “n0”,
                          “ipv4”: “10.0.0.1”,
                          “type”: “router”,
                          “x”: 100,
                          “y”: 250,
                  },
          ],
  }

• POST: Create a new node

• Again, don’t need to specify node ID
• Response would include ID generated by daemon

• /session/<ID NUMBER>/node/<NODE_NUMBER>

• GET: Return info about a particular node
• POST/PATCH: Modify attributes of node

• POST /session/1234/node/8 {
          “name”: “wlan0”
          “basic_range”: {
                  “range”: 100,
                  “delay”: 0,
          },
  }

• DELETE: Remove/shutdown node

• /session/<ID NUMBER>/node/<NODE NUMBER>/execute

• GET: Maybe return all processes running on node?
• POST: Start a new command on a particular node

• /session/1234/node/12/execute {
          “command”: “foobar -x 1”
          “stderr”: “/dev/null”,
  }
• Not specifying a destination for a stream (such as “stdout”), causes the output to be streamed in the HTTP response (which may be a long-lived stream for a long process).

• This way, clients could monitor the output of programs.

• Returns: {
          “output”: “error!”,
          “status”: 123,
  }

• Depends on if output is long or not.  Maybe for long-lived commands, could return an execution number that can be queried for.

• /trace

• GET: All messages sent/received from/by the daemon are copied to this long-lived stream, for logging purposes

Notes

• This can enable some pretty neat things, like a CORE GUI in HTML5/Javascript

• On newer browsers, this is a pretty capable platform for interfaces.
• We have successfully used this approach to provide cross-platform interfaces that work on Linux, Mac, Windows, etc, and even smartphone platforms.
• Could run CORE on a cluster, connect to GUI from browser on my desk.
• New HTML5 API’s such as the EventSource API make it easy for the web server to push events into the browser.

• Could have a /session/1234/node event stream that pushes notifications of new nodes to the GUI (or any other listening program).

• A good decoupling of CORE emulation logic from the GUI seems like good design.

• Makes all GUI functionality available to scripts, etc.

• Long-lived HTTP connections to push data at clients might be a good way of hooking more data collection into CORE.

• Have basically everything dump messages to /trace URL.
• Potentially have some means of filtering

• GET query string:  /trace?filter=mobility