Skip to content

Initial draft of RFC 0006 Device Data Types #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jboynes wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Initial draft of RFC 0006 Device Data Types #1

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
271 changes: 271 additions & 0 deletions rfc-drafts/rfc0006.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,271 @@
# Summary
SignalK provides an open model that allows for both commercial and amateur implementations. This RFC defines how each implementation can describe its data model, without needing a central repository for such definitions. It enables simple, config-free devices to be integrated into a holistic vessel-wide configuration in an automated manner.
# Motivation
To allow interoperation between independently developed systems, a typical approach is to define a common standard to which implementations must conform. Such a standard may be controlled by a single vendor, by committee, or by a formal standards body. Regardless, any new functionality must be approved by the controlling entity and integrated into existing implementations before it can be used. This limits the pace of change and discourages innovation.

This RFC describes an open, federated data model that can be used by devices to describe the data they can provide and the commands that they can support. By defining a standard for a meta model it provides a mechanism that allows any other device to integrate with the new device's data using standardized configuration entries.

Although the model is fully open, we expect common patterns of use to emerge, and that implementers will prefer to use those patterns over proprietary definitions when possible. These common patterns will initially be defined by community working groups before maturing into group patterns defined by SIgnalK.

This issue was initially identified as a [TODO item in the sensors group](https://github.com/SignalK/specification/blob/master/schemas/groups/sensors.json#L15). This RFC refers to devices in preference to sensors to allow for other types of unit such as actuators or more complex systems.
# Detailed Design
Put simply, this RFC defines patterns for describing the representation of JSON data using JSON Schema. A system need only support these patterns in order to integrate with any other system that uses them to describe its data model. Integration is performed by introspecting a JSON schema provided by a device that describes its data model, combined with schema annotations defined by this RFC that supplement the raw schema; an existing example of such an annotation is the `units` property used to specify the unit of measure for a JSON number.

## Device Data Model
In an open model, each device may potentially support an arbitrary set of data and messages used to exchange information with other systems. To support this, each device is required to provide a JSON schema that describes the data and messages; this schema may be defined by the device's creator, or may a reference to a common schema defined by other implementers or this project for use by similar devices.

The schema is responsible for defining:
* the atomic data values the device supports
* the format of messages used to interact with the device

### Atoms
An atom is a non-decomposable fragment of data used for interaction with a device. This may be a single value (such as a voltage), or may be a structure containing multiple, closely coupled properties (such as a position comprised of latitude, longitude and an optional altitude). A atom is the smallest unit of information that may be exchanged between systems, although higher level messages may contain multiple atoms.

In addition to the actual value, an atom may contain meta information describing the format, source, timestamp, validity, or accuracy of the value. An atom is therefore defined to be a JSON `object` type whose schema is decorated with the `atom` boolean property. A parent or child definition of a definition annotated with `atom` MUST NOT be annotated with `atom`.

### Atom Metadata
All property names beginning with `_` are reserved for atom metadata. Implementers SHOULD begin device property names with characters from the set `[a-zA-Z0-9]`.

#### Format
The content of an atom may be specified using a JSON schema reference i.e. a property named `$schema` containing valid JSON schema or a reference to a JSON schema definition.

Example:
```json
{"$schema":"http://devices.acme.example/signalk/bme280.json"}
```
#### Atom ID
The `_id` property should contain a unique identifier for this atom. How this identifier is created is undefined; applications may use a sequence number, structured UUID, or random token. Systems receiving this id must treat as an opaque token.

Examples:
```json
{"_id":"1234567"}
{"_id":"acme-bme280-f1df70-1234567"}
{"_id":"4556-gdgdfg-fge45-5646"}
{"_id":"dsgflwj56tdsufisdt50wtgd"}
```
#### Device ID
The "_deviceId" property may contain a unique identifier for the device that created this atom. Atoms should contain a device id when sent over a shared channel such as a UDP multicast group or message topic.

Example:
```json
{"_deviceId":"acme-bme280-f1df70"}
```

#### Timestamp
The `_timestamp` property may contain an estimation of the time at the source when the atom was created, expressed as an absolute time or a duration since the Unix epoch (1970-01-01T00:00:00Z).

Examples:
```json
{"_timestamp":"2016-11-26T21:23:28.454Z"}
{"_timestamp":"PT1480195408.454S"}
{"_timestamp":1480195408.454}
```

### Non-normative Examples
The atom for a voltage value could be defined as:
```json
{
"type": "object",
"atom": true,
"allOf": [{
"$ref": "https://signalk.github.io/specification/schemas/definitions.json#/definitions/atomMetadata"
},{
"properties": {
"voltage": {
"type": "number",
"units" : "V"
}
},
"required": ["voltage"]
}]
}
}
```
with a sample data point of
```json
{"voltage":12.25}
```
A position as
```json
{
"type": "object",
"atom": true,
"allOf": [{
"$ref": "https://signalk.github.io/specification/schemas/definitions.json#/definitions/atomMetadata"
},{
"properties": {
"latitude": {
"type": "number",
"units" : "deg"
},
"longitude": {
"type": "number",
"units" : "deg"
},
"altitude": {
"type": "number",
"units" : "m"
}
},
"required": ["latitude", "longitude"]
}]
}
}
```
with a sample data point of
```json
{
"_deviceId":"acme-GPS01-5d34e7",
"_id":"1480195408454",
"_timestamp":"2016-11-26T21:23:28.454Z",
"latitude":40.345,
"longitude":-122.765
}
```
where the "Acme GPS01" device with serial number "5d34e7" uses time as a unique id and is able to provide an absolute timestamp.
### Device Messages
Systems interact by exchanging messages containing zero or more atomic values. Each message is structured as a JSON object, described by a JSON schema supplemented with SignalK specific annotations.

Each message should include a `$schema` property at the root corresponding to the content of the message. Receiving devices may validate the message against that schema and may discard non-conforming messages without further processing.

The message types a device can send and receive are listed in its device metadata.

### Device Specific Model Space
Every SignalK device is allocated a portion of the model tree unique to it under the context path `/devices/{deviceId}` where `deviceId` is the globally unique id for that device. From the example above, this location would be under the path `/devices/acme-bme280-f1df70/...`

This property must contain an object conforming to the SignalK type "#/definitions/device" including a "_type" property that contains the device metadata, either as a JSON object or as a reference to another JSON object.

For example, the device model
```json
{
"devices": {
"acme-bme280-f1df70": {
"_type": "http://devices.acme.example/signalk/bme280.json#/metadata",
"serialNumber": "f1df70",
"lastReading": {
"_timestamp": "2016-11-26T21:23:28.454Z",
"temperature": 293.15,
"pressure": 99845,
"humidity": 45.65
}
}
}
}
```
gives the unique id, serial number and last reading from this device. The linked metadata `http://devices.acme.example/signalk/bme280.json` contains:
```json
{
"metadata": {
"manufacturerName": "Acme Devices, inc",
"productName" : "BME-280 Environmment Sensor",
"model": {
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "http://devices.acme.example/signalk/bme280.json#/metadata/model",
"allOf": [{
"$ref": "https://signalk.github.io/specification/schemas/definitions.json#/definitions/commonDeviceProperties"
}, {
"properties": {
"lastReading" : { "$ref": "#/data" }
}
}]
},
"sends": {
"hello": { "$ref": "https://signalk.github.io/specification/schemas/messages.json#/hello"},
"data": {
"type": { "$ref": "#/data"},
"interval": 10
}
},
"receives": {
}
},
"data": {
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "http://devices.acme.example/signalk/bme280.json#/data",
"type": "object",
"atom": true,
"allOf": [{
"$ref": "https://signalk.github.io/specification/schemas/definitions.json#/definitions/atomMetadata"
},{
"properties": {
"temperature": {
"type": "number",
"units" : "K"
},
"pressure": {
"type": "number",
"units" : "Pa"
},
"humidity": {
"type": "number",
"units" : "ratio"
}
}
}]
}
}
}
```
that shows the device sends a standard `hello` message, as well as its device specific `data` message every 10s.

## Device to Vessel Integration
This RFC describes application-level integration that occurs after a device has been installed, authorized to join a vessel's network, discovered, and been discovered, by other SignalK applications. It assumes that the messages defined by this RFC can be exchanged.

Examples of such integration include:
* a voltmeter may need to be associated with the circuit location that it is measuring (e.g. house vs. engine battery)
* an environment sensor may need to be associated with a living space (e.g. aft cabin vs. saloon, lower vs. upper deck, inside vs. outside)
* a relay may need to be associated with the circuit it operates (e.g. main cabin lights vs. engine start)
* a compass sensor may need to be associated with a deviation calibration set

Each device announces its capabilities by sending a standard `hello` message containing a minimal copy of its device model including the device metadata and, optionally, the current device state. The minimal model is defined by the `https://signalk.github.io/specification/schemas/messages.json#/hello` schema:
```json
{
"type": "object",
"properties": {
"devices": {
"type": "object",
"patternProperties": {
"[a-ZA-Z][a-ZA-Z0-9._-]*": {
"type": "object",
"properties": {
"serialNumber": {
"type": "string"
},
"_type": {
"type": "object string",
"description": "Device type definition, either as an inline object or the URI of resource containing the definition"
}
},
"required": ["serialNumber", "_type"]
}
}
}
},
"required": ["devices"]
}
```

The device responsible for integrating the new device into the vessel's data model creates an entry in its own model containing the new device's metadata, and performs any mapping needed to include the devices data into the vessel's model. For example, the an installer might enter location information indicating the environment sensor was located in the master's cabin thereby associating its readings with the `/vessels/2309999/environment/inside/masterCabin` path (or wherever the installer deemed most appropriate).

## Schema Distribution
The JSON schema documents associated with a device may be distributed in any way that allows them to be read by the integrating system.

A simple, albeit verbose, mechanism is to include them, by value, in messages sent by the device. This may be done during initial negotiation, for example, as part of a `hello` message.

More typically, the schema would be referenced using its unique id (from the JSON schema definition). If this id is a dereferencable URL, any integrating device with internet access would be able to retrieve it directly; such devices should cache the copy locally for offline access. Other alternatives include:
* a device with an embedded web server could support download from the device itself
* a copy of the schema could be provided on installation media shipped with the device (e.g. a flash drive)
* common schema may be pre-installed on the integrating device; for example, a manufacturer may pre-install schemas defined by this specification, or for other devices they produce

# Drawbacks
Although the device specific information is kept to a minimum, more work must be performed by the integrating device to retrieve and process the metadata objects and schemas. For example, the environmental sensor need only deal with simple messages containing its serial number (unique id) and atomic values. The rest of the information is static content that could kept on the device or sourced elsewhere. However, the integrating server needs to be able to retrieve or otherwise install that information.

A device manufacturer is responsible for producing the device metadata needed for automatic integration. However, it seems better to place the burden there rather than on end users.

# Alternatives
We could maintain a central registry of device types and metadata groups. However, that means any new device type implementations would need to be defined here first, entered in the registry, and would not be usable until a new version of the schema and keys was published.

We could require every device to allow configuration that defined how it was associated with the vessel-wide model. That would require some form of user interface for the installer to use (although that could be some form of remote client) and would require all clients to have local persistent storage (e.g. EEPROM or flash) to store the association mapping. For example, the environment sensor would need to be configured to send its readings as part of the `/vessel/{vesselId}/{inside}/{cabin}` context.

# Unresolved Questions
The schema entries for this need still need to be fully defined, including more verbose descriptions.