Agent example

With the help of Bork, the availability of agents can be monitored by reasoning from the reception of heartbeats. In the context of Bork, agents are broadly spoken IoT-Devices. These agents must send their heartbeats by publishing an entity that corresponds to the Heartbeat schema. Besides that they must publish an entity to configure how they are monitored by Bork, that corresponds to the Agent schema. With running instances of Orion Context Broker and Bork the following scenarios can be passed through.

Publishing the agent entity

An agent must publish a Agent entity that is used by Bork to identify this particular agent, recognize his Heartbeat entity and the monitoring properties. The individual attributes of the Agent entity and their meanings can be viewed in the corresponding spec file of that schema. For instance, the agent "id": "IML:SAN-123a" of type": "Agent" must publish the following entity. Because Bork supports Orion Context Broker's multi tenancy capability and hierarchical scopes and thus can be configured using one or both of this options, the entity must be published into the same service and service path as Bork has been configured. To show this in action the following example will be using the service t_01 and service path /iml.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/entities?options=keyValues'>

{
    "id": "IML:SAN-123a",
    "type": "Agent",
    "category": "SAN",
    "capabilities": ["pressure" , "temperature"],
    "refHeartbeat": {
        "id": "IML:SAN-123a-Heart",
        "type": "Heartbeat",
        "servicePath": "/iml",
        "service": "t_01"
    },
    "monitorProperty": {
        "isMonitoringCapable": true,
        "wantsToBeAvailable": true,
        "supportsNotifyIntervalChange": false,
        "currentNotifyInterval": 9.76
    }
}

Note: This entity could be extended with vendor information or other contextual data. Also the properties category and capabilities are optional and could be filled with any type of information.

Mode 1: Agent set notify interval (Heartbeat)

In Mode 1 the agent sets his own notify interval, in other words, the interval he uses to publish his heartbeats. This interval is specified, as can be seen in the section before, with the currentNotifyInterval property of the Agent entity. The agent then needs to publish the heartbeats using the Heartbeat schema, for example like the following POST request.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/entities?options=keyValues'>

{
    "id": "IML:SAN-123a-Heart",
    "type": "Heartbeat",
    "refAgent": {
        "id": "IML:SAN-123a",
        "type": "Agent",
        "servicePath": "/iml",
        "service": "t_01"
    },
    "dateSent": "2016-11-30T07:00:00.00Z",
    "sequenceNumber": 0
}

Note: The dateSent Timestamp has to be in UTC ISO 8601 format, like YYYY-MM-DDThh:mm:ss.ssZ. Other ISO 8601 formats are currently not supported.

Then ongoing update the published entity in currentNotifyInterval second steps, like following.

<Method: 'PATCH'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/entities/IML:SAN-123a-Heart/attrs?type=Heartbeat&options=keyValues'>

{
    "dateSent": "2016-11-30T07:09:76.00Z",
    "sequenceNumber": 1
}

Note: The sequenceNumber was incremented.

If a change of the notify interval is desired, the currentNotifyInterval property of the Agent entity can be adjusted.

<Method: 'PATCH'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/entities/IML:SAN-123a/attrs?type=Agent&options=keyValues'>

{
    "monitorProperty": {
        "isMonitoringCapable": true,
        "wantsToBeAvailable": true,
        "supportsNotifyIntervalChange": false,
        "currentNotifyInterval": 6.22
    }
}

Note: Updating just a part of the monitorProperty property is not possible without removing the other part of that property. Therefore the whole attribute has to be updated, like shown above.

Then use that new notify interval after the next heartbeat. That means the next heartbeat to be send should be use the old notify interval 9.76. Once that was send out the agent can use that newly configured interval 6.22.

Mode 1: Agent set notify interval (Data update)

To support agents that update a certain entity at a constant interval, the agent can use this entity as a replacement for his heartbeat. This could be something like sensor that sends updates at fixed timepoints. The advantage of this approach is that this agent does not have to have an extra procedure for sending heartbeats. This reduces complexity and, if applicable, the need for low power devices to wake up more than necessary from their sleep cycles. So as stated, the agent creates his Agent entity with a reference to that certain data updating entity as the refHeartbeat property, like following.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/entities?options=keyValues'>

{
    "id": "IML:SAN-123a",
    "type": "Agent",
    "category": "SAN",
    "capabilities": ["pressure" , "temperature"],
    "refHeartbeat": {
        "id": "IML:SAN-123a-Measurements",
        "type": "Measurement"
    },
    "monitorProperty": {
        "isMonitoringCapable": true,
        "wantsToBeAvailable": true,
        "supportsNotifyIntervalChange": false,
        "currentNotifyInterval": 9.76
    }
}

Note: To show as an example that these properties are optional, the servicePath and service of the refHeartbeat was omitted here. The type of the the refHeartbeat is freely chosen.

The further workflow is the same as shown in the above section with a normal heartbeat. When using the measurement data updates as heartbeat replacement, it should be noted that the missing sequence number may limit the calculation of the estimated arrival time at Bork and the metrics skippedHeartbeatsCount and messageLossProbability.

Mode 2: Bork set notify interval

In Mode 2 the agent uses a notify interval that was calculated by Bork. To do so the agent specifies three bounds which have to be respected by Bork. In order for Bork to be able to comply with these bounds, Bork calculates the interval that is the largest possible interval while complying with the bounds. That means that calculated interval is optimal with regard to that specifications. The individual attributes, including those specifications, of the Agent entity and their meanings can be viewed in the corresponding spec file of that schema. The calculated interval is made available to the agent through the HeartbeatConfig entity following the HeartbeatConfig schema.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/entities?options=keyValues'>

{
    "id": "IML:SAN-123a",
    "type": "Agent",
    "category": "SAN",
    "capabilities": ["pressure" , "temperature"],
    "refHeartbeat": {
        "id": "IML:SAN-123a-Heart",
        "type": "Heartbeat",
        "servicePath": "/iml",
        "service": "t_01"
    },
    "monitorProperty": {
        "isMonitoringCapable": true,
        "wantsToBeAvailable": true,
        "supportsNotifyIntervalChange": true,
        "detectionTimeUpperBound": 29.98,
        "mistakeRecurrenceTimeLowerBound": 2592000.0,
        "mistakeDurationUpperBound": 60.0,
        "currentNotifyInterval": 9.76
    }
}

As shown here the property supportsNotifyIntervalChange has switched to true indicating Bork that the Agent can change his interval on request. The next three properties specify the mentioned bounds. As noted in the Agent spec file, the detectionTimeUpperBound is the "Time that should maximal elapse from the beginning of a crash of the Agent, to the detection through Bork". That means with respect of the following two properties, namely the mistakeRecurrenceTimeLowerBound and mistakeDurationUpperBound, and properties that are calculated by Bork, namely the variance of message delay and the probability of message loss, Bork calculates a notify intervall that holds that detectionTimeUpperBound. As already mentioned this notify intervall is made available to the agent through a HeartbeatConfig entity maintained by Bork. This entity will only be created if Mode 2 is used. To be informed about changes of that entity, subscribe it, like following.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/subscriptions'>

{
    "description": "Agent HeartbeatConfig Subscription to be informed about newly calculated notify intervals",
    "subject": {
        "entities": [
            {
                "id": "IML:SAN-123a",
                "type": "HeartbeatConfig"
            }
        ]
    },
    "notification": {
        "http": {
            "url": "http://localhost:54321/notifyIntervalUpdate"
        },
        "attrs": [
            "*",
            "dateCreated",
            "dateModified"
        ],
        "attrsFormat": "keyValues"
    }
}

Note: The id of that entity is the same as the id of the corresponding Agent entity. This entity is created by Bork in the same Fiware-ServicePath and Fiware-Service as the corresponding agent entity is located. In this example that is the service t_01 and service path /iml. See the installation guide for more information.

Such a received HeartbeatConfig entity can look something like the following. With the property notifyInterval declaring the notify interval the agent should use after the next heartbeat.

{
    "id": "IML:SAN-123a",
    "type": "HeartbeatConfig",
    "refAgent": {
        "id": "IML:SAN-123a",
        "type": "Agent",
        "servicePath": "/iml",
        "service": "t_01"
    },
    "notifyInterval": 9.76,
    "dateModified": "2016-11-30T07:00:00.00Z",
    "dateCreated": "2016-11-30T07:00:00.00Z"
}

To indicate that the agent received and accepted the new notify interval the agent has to update his Agent entity accordingly, as shown in the Mode 1 section. The agent can also change the three bounds, namely the detectionTimeUpperBound, mistakeRecurrenceTimeLowerBound and mistakeDurationUpperBound, declared in the Agent entity on his behalf. Bork will react to that change by calculating a new adjusted notify interval matching that bounds.

Tolerating the Discovery

To limit the agents sending heartbeats only if a instance of Bork is running and is in correct state to process data, the agents should look after the Discovery entity. This entity that corresponds to the Discovery schema, denotes with the sendHeartbeat property if the agents should or should not send heartbeats. Doing so, unnecessary traffic can be prevented. When Bork shuts down the property is changed to false. A subscription like the following could be used to track the state of that property.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json', 'Fiware-Service': 't_01', 'Fiware-ServicePath': '/iml'}'> <URL: 'http://localhost:1026/v2/subscriptions'>

{
    "description": "Agent Discovery Subscription to be informed about whether heartbeats should be sent or not",
    "subject": {
        "entities": [
            {
                "idPattern": ".+",
                "type": "Discovery"
            }
        ]
    },
    "notification": {
        "http": {
            "url": "http://localhost:54321/DiscoveryChange"
        },
        "attrs": [
            "*",
            "dateCreated",
            "dateModified"
        ],
        "attrsFormat": "keyValues"
    }
}

Note: This entity is created by Bork in the Fiware-ServicePath and Fiware-Service that is configured with the docker-compose file. In this example that is the service t_01 and service path /iml. See the installation guide for more information.

Using Bork with an IDAS-Agent

Using an json IDAS-Agent collaborate with Bork needs some adjustments outlined below. Prerequisite is, that you have a running IDAS instance. See the linked readthedocs or the iotagent-json on docker hub if you haven't. The iotagent-ul and the lightweightm2m-iotagent can't handle structured attributes which are needed for the used data mode, see issue 549. Another problem using iotagent-ul is the need of updating serval properties at the same time, for example with "s|4#d|10" (see issue 391), uses two requests to update the entity at the context broker. The logic used by Bork insists, however, that the heartbeat properties sequenceNumber and dateSent are updated at the same time. Using the iotagent-json, create a service, like following.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json'}'> <URL: 'http://localhost:4041/iot/services'>

{
 "services": [
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "http://orion:1026",
     "entity_type": "Heartbeat",
     "resource":    "/iot/json"
   }
 ]
}

Note: The property apikey is the token needed to send updates from devices corresponding to that service. The service and servicePath are left out here, but could be added as headers.

And create a device, like following.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json'}'> <URL: 'http://localhost:4041/iot/devices'>

{
 "devices": [
   {
     "device_id":   "motion001",
     "entity_name": "IML:SAN-123a-Heart",
     "entity_type": "Heartbeat",
     "timezone":    "Europe/Berlin",
     "attributes": [
       { "object_id": "s", "name": "sequenceNumber", "type": "Integer" },
       { "object_id": "d", "name": "dateSent", "type": "DateTime" }
     ],
     "static_attributes": [
       { "name":"refAgent", "type": "StructuredValue", "value": {"id": "IML:SAN-123a", "type": "Agent"}}
     ]
   }
 ]
}

Note: The service and servicePath are left out here, but could be added as headers.

Then a heartbeat can be send as a normal update to the IDAS instance, using the token and the device id.

<Method: 'POST'> <Headers: '{'Content-Type': 'application/json'}'> <URL: 'http://localhost:7896/iot/json?k=4jggokgpepnvsb2uv4s40d59ov&i=motion001'>

{
    "s": 5,
    "d": "2019-06-24T07:59:33.00Z"
}

Listen on changes of the Discovery or HeartbeatConfig entity

This is currently not supported by Bork/IDAS. This means to IDAS-Agents, that they can't be informed about the Discovery entity existence, as seen in the Tolerating the Discovery section. Since the agent cannot subscribe to the HeartbeatConfig entity either, it is not possible for him to use Mode 2. The implementation of the missing functionalities could be implemented via the command functionality of IDAS. However, this would mean more profound changes to Bork, since in this case the agents would have to create the two entities themselves.