Personal Universal Controller Communications Protocol

Documentation originally by Michael Higgins, later updated by Jeffrey Nichols.

Protocol design by the Personal Universal Controller working group: Michael Higgins, Joe Hughes, Peter Lucas, Brad A. Myers, Jeffrey Nichols, and Mathilde Pignol.

Table of Contents

  1. Introduction
  2. Revision History
  3. Conceptual Basis
  4. List Data
  5. Message Enumeration
    1. Controller Generated Messages
    2. Appliance Generated Messages
  6. Service Discovery
    1. Discovery Messages
  7. Wire Format For Messages Over Plain TCP/IP
  8. Schema
  9. Future Work
  10. Example

Introduction

The personal universal controller (PUC) project's goal is to build a software framework running on top of a personal digital assistant (PDA) that is capable of providing an interface that will allow users to control any appliance within their environment. In essence, the PUC is a device-independent, appliance-independent, user-dependent remote control device.

For more information about the PUC framework, see the Personal Universal Controller home page. You may also be interested in the appliance specification language.

This document describes the high-level communications protocol used by the Personal Universal Controller and the appliances it is controlling. Some of these messages are used for rudimentary service discovery.

Revision History

Date Name Comments
03/07/2002 higgins First version.
03/11/2002 jwn Minor modifications prior to posting the document on the web and adding it to the cvs repository. Also changed references to reflect movement of the published DTDs and docs to the pebbles web site.
06/22/2004 jwn Updated this document with a Schema that represents our modifications to the protocol over the last few years. Updated the examples and accompanying text.

Conceptual Basis

Because the Personal Universal Controller has few pre-conceived notions of what sort of devices may be controlled or, indeed, what form the controller may take, the communications protocol must be quite general. To achieve this it has been kept simple, with a minimum of messages.

Our protocol assumes a connection-oriented transport underneath it, though because it is mostly asynchronous it could probably be easily adapted to connection-less environment or even a broadcast environment. We have implemented it over TCP/IP, the particulars of which are described in a later section.

All messages within the protocol have an XML component and may optionally append unformatted binary data (such as images, etc.) XML is used for the general syntax to achieve harmony with the specification language. This may also allow easy adaptation for use with SOAP and other emerging XML communication standards. Binary data is appended to the end to avoid possible incompatibilities that would have occurred if this data had been sent within the XML. Currently the only message that may be followed by binary data is the binary-state-change-notification message.

Typically, the appliance acts as a server, waiting for controllers to establish connections. The controller is not required to send any particular message upon establishing the connection, but typically it immediately performs a spec-request to establish the states available and their types. It then performs a full-state-request to get the complete state of the appliance. This allows the controller to build an interface reflecting the current state of the appliance.

The most interesting feature of the protocol is that it is mostly asynchronous. The appliance can emit a state-change-notification at any time; typically, the controller does not poll for state changes. If the controller suspects that it has gotten out of sync with the appliance state, it can issue a full-state-request.

The controller can issue a state-change-request or command-invoke-request at any time. There is no explicit acknowledgement of this by the appliance, but typically one or more state-change-notification messages will be sent shortly thereafter.

The controller should be prepared to receive unasked-for "responses" such as a device-spec message at any time as well.

The reason for this asynchronous design is that many appliances may change independently of the controller (typically as a result of some internal event, like the completion of a long-running chore, or because of some other command, like the manipulation of the physical controls on the device). Furthermore many real-world state changes take significant time to complete. The protocol should not artificially prevent the controller from issuing other requests or receiving other notifications while one transaction is occurring. It is also possible that we may wish to interleave many controller-appliance relationships over one transport connection. An asynchronous design helps facilitate all of these.

The one exception to this asynchronous model involves binary data. There are several reasons for this exception. Binary data may be very long and require significant time for transfer. It also does not make sense to send binary data to controllers will not be able to render it (e.g. a large color image should not be sent to a mobile phone with a black & white screen). When new binary data is available on the appliance, the appliance will send a binary-state-change-notification message naming the state that changed and the content-type of this data but without including the actual binary data. If the controller is interested in receiving this binary data, it will send a state-value-request message naming the same state. In response, the appliance will send another binary-state-change-notification message containing the binary data.

It should be noted that strict synchronicity is not enforced here. Other messages may be transmitted between during the exchange described above.

List Data

state-change-notification and state-change-request messages support a complex data structure for supporting list data structures. This is needed because lists may include large amounts of information, and it is not reasonable to send all of this data everytime the contents of the list changes.

Four list operations are supported:

Message Enumeration

The controller should be prepared to receive any appliance-generated message at any time, and to wait indefinitely for an appliance-generated message. In particular, the controller should not expect a strict request-response discipline to be maintained.

Likewise, the appliance must be prepared to receive any controller-generated message at any time.

Controller Generated Messages

state-change-request
This message requests the appliance to change the designated state to the value contained in the message.
command-invoke-request
This message requests the appliance to invoke a command. This may cause state changes as side effects.
spec-request
This message requests the appliance to send a copy of its specification. It will send this via the device-spec message.
full-state-request
This message requests the appliance to send state-change-notification messages for every state it has.
state-value-request
This message requests the appliance to send a binary-state-change-notification message containing binary data for a particular state.

Appliance Generated Messages

state-change-notification
This message is sent whenever the appliance changes state. The state name as well as its value is sent.
binary-state-change-notification
This message may be sent in two different contexts. It is sent when binary data has changed on the appliance and is sent in response to a state-value-request message from the controller.
device-spec
This message contains the appliance specification. It is sent after receiving a spec-request message.
alert-information
This message contains a string message that needs to be delivered to the user of each controller.

Service Discovery

We have implemented a very simplistic service discovery mechanism to make finding appliances easier than remembering IP addresses and port numbers. Our method relies on each machine serving PUC appliances to run a service on port 5149. Controllers may send a message to this port to get a list of all appliances running on the machine. There are also registration methods for appliances to notify the service discovery system of their existence.

Service Discovery Messages

server-information-request
This message is sent by controllers to get the list of appliances connected to a particular machine.
server-information
This message is sent by the service discovery manager to let a controller know which appliances are connected to this server. It is sent in response to server-information-request messages.
register-device
This message is sent by appliances to the service discovery manager to notify the manager of the appliance's existence.
unregister-device
This message is sent by appliances to the service discovery manager to notify the manager that the appliance is shutting down.

Wire Format For Messages Over Plain TCP/IP

For our TCP/IP implementation, a PUC message consists of an eight-byte header, XML content, and optional format-independent binary content.

The header is divided into two four-byte chunks. The first chunk is an integer giving the full length of the message (not including the header). The second chunk is an integer giving the length of the XML portion of the message. Both of these integers are sent using the network byte order. This header is needed because TCP/IP has no convenient mechanism for specifying message length, and there is also a need to know exactly where the format-independent binary data begins.

Immediately following the header is a well-formed XML document conforming to the following schema.

Following the XML, arbitrary binary content may be included. The format is usually defined within an element of the accompanying XML content.

Schema

<?xml version="1.0" encoding="utf-8" ?>
<xs:schema targetNamespace="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol" elementFormDefault="qualified"
	xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol" xmlns:mstns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol"
	xmlns:xs="http://www.w3.org/2001/XMLSchema">
	<!--
		The PUC protocol consists of an eight-byte header, XML content, and 
		optional format-independent binary content. 
		
		The header is divided into two four-byte chunks.  The first chunk is an
		integer giving the full length of the message (not including the header).  
		The second chunk is an integer giving the length of the XML portion of 
		the message.  Both of these integers are sent using the network byte 
		order.
		
		The XML content is described by this Schema.
		
		The binary content may be in any arbitrary format.  The format is usually
		defined within an element of the accompanying XML message.
	  -->
	<xs:element name="message">
		<xs:complexType>
			<xs:choice minOccurs="1" maxOccurs="1">
				<xs:element name="state-change-notification" type="PUCDataType" />
				<xs:element name="state-change-request" type="PUCDataType" />
				<xs:element name="binary-state-change-notification" type="BinaryStateChangeNotifyType" />
				<xs:element name="state-value-request" type="StateValueRequestType" />
				<xs:element name="command-invoke-request" type="CommandInvokeRequestType" />
				<xs:element name="spec-request" />
				<xs:element name="device-spec" type="DeviceSpecType" />
				<xs:element name="full-state-request" />
				<xs:element name="server-information-request" />
				<xs:element name="server-information" type="ServerInformationType" />
				<xs:element name="alert-information" type="xs:string" />
				<xs:element name="register-device" type="DeviceType" />
				<xs:element name="unregister-device" type="UnregisterDeviceType" />
			</xs:choice>
		</xs:complexType>
	</xs:element>
	<xs:complexType name="BinaryStateChangeNotifyType">
		<xs:sequence>
			<xs:element name="state" type="StateWithContentAttribType" minOccurs="1" maxOccurs="1" />
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="StateWithContentAttribType">
		<xs:simpleContent>
			<xs:extension base="xs:string">
				<xs:attribute name="content-type" type="xs:string" />
			</xs:extension>
		</xs:simpleContent>
	</xs:complexType>
	<xs:complexType name="StateValueRequestType">
		<xs:sequence>
			<xs:element name="state" type="xs:string" minOccurs="1" maxOccurs="1" />
			<xs:any minOccurs="0" maxOccurs="unbounded" />
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="CommandInvokeRequestType">
		<xs:sequence>
			<xs:element name="command" type="xs:string" minOccurs="1" maxOccurs="1" />
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="DeviceSpecType">
		<xs:sequence>
			<xs:element name="spec" type="xs:string" minOccurs="1" maxOccurs="1" />
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="ServerInformationType">
		<xs:sequence>
			<xs:element name="server-name" type="xs:string" minOccurs="1" maxOccurs="1" />
			<xs:element name="device" type="DeviceType" minOccurs="0" maxOccurs="unbounded" />
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="DeviceType">
		<xs:sequence>
			<xs:element name="name" type="xs:string" minOccurs="1" maxOccurs="1" />
			<xs:element name="port" type="xs:integer" minOccurs="1" maxOccurs="1" />
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="UnregisterDeviceType">
		<xs:sequence>
			<xs:element name="port" type="xs:integer" minOccurs="1" maxOccurs="1" />
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="PUCDataType">
		<xs:choice minOccurs="1" maxOccurs="1">
			<xs:sequence>
				<xs:element name="state" type="xs:string" />
				<xs:element name="value" type="OldValueType" />
			</xs:sequence>
			<xs:element name="data" type="ListDataType" />
			<xs:element name="change" type="ChangeDataType" />
			<xs:element name="insert" type="InsertType" />
			<xs:element name="delete" type="DeleteType" />
			<xs:element name="replace" type="ReplaceType" />
		</xs:choice>
	</xs:complexType>
	<xs:complexType name="OldValueType" mixed="true">
		<xs:sequence>
			<xs:element name="undefined" minOccurs="0" maxOccurs="1"/>
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="NewValueType" mixed="true">
		<xs:sequence>
			<xs:element name="undefined" minOccurs="0" maxOccurs="1"/>
		</xs:sequence>
		<xs:attribute name="state" type="xs:string"/>
	</xs:complexType>
	<xs:complexType name="AnyListContentType">
		<xs:choice>
			<xs:element name="data" type="ListDataType" />
			<xs:element name="change" type="ChangeDataType" />
			<xs:element name="insert" type="InsertType" />
			<xs:element name="delete" type="DeleteType" />
			<xs:element name="replace" type="ReplaceType" />
			<xs:sequence>
				<xs:element name="value" type="NewValueType" minOccurs="1" maxOccurs="unbounded" />
			</xs:sequence>
		</xs:choice>
	</xs:complexType>
	<xs:complexType name="AfterChangeOpListType">
		<xs:choice>
			<xs:element name="data" type="ListDataType" />
			<xs:sequence>
				<xs:element name="value" type="NewValueType" minOccurs="1" maxOccurs="unbounded" />
			</xs:sequence>
		</xs:choice>	
	</xs:complexType>
	<xs:complexType name="ListDataType">
		<xs:sequence>
			<xs:element name="el" type="AfterChangeOpListType" minOccurs="1" maxOccurs="unbounded"/>
		</xs:sequence>
		<xs:attribute name="state" type="xs:string" use="required" />
	</xs:complexType>
	<xs:complexType name="ChangeDataType">
		<xs:sequence>
			<xs:element name="el" type="AnyListContentType" minOccurs="1" maxOccurs="1"/>
		</xs:sequence>
		<xs:attribute name="state" type="xs:string" use="required" />
		<xs:attribute name="index" type="xs:integer" use="required" />
	</xs:complexType>
	<xs:complexType name="DeleteType">
		<xs:attribute name="state" type="xs:string" use="required" />
		<xs:attribute name="begin" type="xs:integer" use="required" />
		<xs:attribute name="length" type="xs:integer" use="required" />
	</xs:complexType>
	<xs:complexType name="InsertType">
		<xs:sequence>
			<xs:element name="el" type="AfterChangeOpListType" minOccurs="1" maxOccurs="unbounded" />
		</xs:sequence>
		<xs:attribute name="state" type="xs:string" use="required" />
		<xs:attribute name="after" type="xs:integer" use="required" />
	</xs:complexType>
	<xs:complexType name="ReplaceType">
		<xs:sequence>
			<xs:element name="el" type="AfterChangeOpListType" minOccurs="1" maxOccurs="unbounded" />
		</xs:sequence>
		<xs:attribute name="state" type="xs:string" use="required" />
		<xs:attribute name="begin" type="xs:integer" use="required" />
		<xs:attribute name="length" type="xs:integer" use="required" />
	</xs:complexType>
</xs:schema>
      

Future Work

There is no heartbeat present in the communications spec, so a controller cannot easily tell the difference between an appliance that has crashed and one that is merely slow. If the protocol is implemented in a connection-oriented environment the dropping of the connection could be used to indicate this, but that may not be a reliable indicator of the appliance's health. Many wireless environments have poor quality-of-service guarantees, so should be expected to suffer connection failures and congestion. The current workaround for this is to periodically have the controller perform a full-state-request. If no response is had within some reasonable time, the appliance can be assumed dead. Because the response to a full-state-request is typically large this is a somewhat unsatisfactory approach.

The service discovery mechanism still requires the user to know the IP address or name of a server that is serving appliances. It would be nice to incorporate some mechanism that allowed appliances to be asynchronously discovered. Perhaps some mechanism using the IP broadcast address could be useful for this purpose.

Example

For the sake of a simple example, we'll imagine using the PUC to control a simple media player. (The media player is the "appliance".) NOTE: Each of these examples references the Schema, but these attributes are not included in actual PUC communication because there is no guarantee of internet access for the controller to download the schema and verify each message.

Initially, the media player is stopped but has a few songs in its play list. A user walks up to it, armed with a PUC, and initiates communication. First the user must discover what appliances are available on the appliance server.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<server-information-request />
</message>
    

The server responds with a list of all appliances connected to the server. The user will choose the Simple Media Player to connect to.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<server-information>
		<server-name>Test Server</server-name>
		<device>
			<name>Simple Media Player</name>
			<port>5180</port>
		</device>
	</server-information>
</message>
    

Once the controller connects to the desired appliance, it sends a request for the specification.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<spec-request/>
</message>
    

The media player responds with a spec.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<device-spec>
		<spec>
&lt;?xml version="1.0" encoding="utf-8"?&gt;
&lt;spec name="MediaPlayer" version="PUC/2.1" xmlns="http://www.cs.cmu.edu/~pebbles/puc"&gt;
  &lt;labels&gt;
    &lt;label&gt;Media Player&lt;/label&gt;
  &lt;/labels&gt;
  &lt;groupings&gt;
    &lt;group name="Controls" is-a="media-controls"&gt;
      &lt;labels&gt;
        &lt;label&gt;Play Controls&lt;/label&gt;
        &lt;label&gt;Play Mode&lt;/label&gt;
        &lt;text-to-speech text="Play Mode" recording="playmode.au" /&gt;
      &lt;/labels&gt;
      &lt;state name="Mode"&gt;
        &lt;type&gt;
          &lt;enumerated&gt;
            &lt;item-count&gt;3&lt;/item-count&gt;
          &lt;/enumerated&gt;
          &lt;value-labels&gt;
            &lt;map index="1"&gt;
              &lt;labels&gt;
                &lt;label&gt;Stop&lt;/label&gt;
              &lt;/labels&gt;
            &lt;/map&gt;
            &lt;map index="2"&gt;
              &lt;labels&gt;
                &lt;label&gt;Play&lt;/label&gt;
              &lt;/labels&gt;
            &lt;/map&gt;
            &lt;map index="3"&gt;
              &lt;labels&gt;
                &lt;label&gt;Pause&lt;/label&gt;
              &lt;/labels&gt;
            &lt;/map&gt;
          &lt;/value-labels&gt;
        &lt;/type&gt;
        &lt;labels&gt;
          &lt;label&gt;Mode&lt;/label&gt;
        &lt;/labels&gt;
      &lt;/state&gt;
      &lt;group name="TrackControls"&gt;
        &lt;command name="PrevTrack"&gt;
          &lt;labels&gt;
            &lt;label&gt;Prev&lt;/label&gt;
          &lt;/labels&gt;
          &lt;active-if&gt;
            &lt;greaterthan state="PList.Selection"&gt;
              &lt;static value="0"/&gt; 
            &lt;/greaterthan&gt;
          &lt;/active-if&gt;
        &lt;/command&gt;
        &lt;command name="NextTrack"&gt;
          &lt;labels&gt;
            &lt;label&gt;Next&lt;/label&gt;
          &lt;/labels&gt;
          &lt;active-if&gt;
            &lt;lessthan state="PList.Selection"&gt;
              &lt;refvalue state="PList.Length" /&gt;
            &lt;/lessthan&gt;
          &lt;/active-if&gt;
        &lt;/command&gt;
      &lt;/group&gt;
    &lt;/group&gt;
    &lt;list-group name="PList"&gt;
      &lt;state name="Title"&gt;
        &lt;type&gt;
          &lt;string /&gt;
        &lt;/type&gt;
        &lt;labels&gt;
          &lt;label&gt;Title&lt;/label&gt;
        &lt;/labels&gt;
      &lt;/state&gt;
      &lt;state name="Duration" is-a="time-duration"&gt;
        &lt;type&gt;
          &lt;integer /&gt;
        &lt;/type&gt;
        &lt;labels&gt;
          &lt;label&gt;Duration&lt;/label&gt;
        &lt;/labels&gt;
      &lt;/state&gt;
    &lt;/list-group&gt;
  &lt;/groupings&gt;
&lt;/spec&gt;
		</spec>
	</device-spec>
</message>
    

This response contains the complete spec for the media player, which contains one enumerated state, two commands, and a list with three states. Note that the XML of the spec is escaped. Upon receiving the spec and generating an interface, the controller asks the media player for a listing of its complete state.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<full-state-request />
</message>
    

In response, the media player sends two messages. The first gives the value of the enumerated state and the next gives the values in the list.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-notification>
		<state>MediaPlayer.Controls.Mode</state>
		<value>1</value>
	</state-change-notification>
</message>

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-notification>
		<data state="MediaPlayer.PList">
			<el>
				<value state="Title">Sweet Home Alabama</value>
				<value state="Duration">190</value>
			</el>
			<el>
				<value state="Title">Tower</value>
				<value state="Duration">203</value>
			</el>
			<el>
				<value state="Title">Jane</value>
				<value state="Duration">243</value>
			</el>
		</data>
	</state-change-notification>
</message>
    

Now the user will set the media player to play.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-request>
		<state>MediaPlayer.Controls.Mode</state>
		<value>2</value>
	</state-change-request>
</message>
    

The media player starts player, and as a consequence of the state change, sends two messages indicating changes in its state. One of the corresponds to the state change requested by the user, and the other is caused a result of a selection being set in the play list.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-notification>
		<state>MediaPlayer.Controls.Mode</state>
		<value>1</value>
	</state-change-notification>
</message>

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-notification>
		<state>MediaPlayer.PList.Selection</state>
		<value>1</value>
	</state-change-notification>
</message>
    

The user decides that they don't like this song, so they advance to the next track of the play list.

 <?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<command-invoke-request>
		<command>MediaPlayer.Controls.TrackControls.NextTrack</command>
	</command-invoke-request>
</message>
    

The appliance changes the track, and as a result must send a message to the controller because the selection in the PList has changed.

<?xml version="1.0" encoding="utf-8" ?> 
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-notification>
		<state>MediaPlayer.PList.Selection</state>
		<value>2</value>
	</state-change-notification>
</message>
    

The user decides to go to the computer and update the media player's play list. This causes the appliance to send several notifications.

<?xml version="1.0" encoding="utf-8" ?>
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-notification>
		<replace state="MediaPlayer.PList" begin="3" length="1">
			<el>
				<value state="Title">Gravity</value>
				<value state="Duration">217</value>
			</el>
			<el>
				<value state="Title">1000000 Dollars</value>
				<value state="Duration">236</value>
			</el>
			<el>
				<value state="Title">Jude</value>
				<value state="Duration">209</value>
			</el>
		</replace>
	</state-change-notification>
</message>

<?xml version="1.0" encoding="utf-8" ?>
<message xmlns="http://www.cs.cmu.edu/~pebbles/puc/puc-protocol">
	<state-change-notification>
		<delete state="MediaPlayer.PList" begin="1" length="1" />
	</state-change-notification>
</message>
    

Jeffrey Nichols
Last modified: Tue Jun 22 15:04:36 EST 2004