How to Recover from Device Errors with OmniServer Error Messages

With any communication server for collecting process data, there has to be a reliable mechanism built-in for dealing with error responses from the underlying devices.  This is true even for OmniServer as a user-configurable communication server.

Continuing our "OmniServer Did You Know?" blog series, this blog post focuses on how to use error messages and notification items in an OmniServer protocol to properly handle error responses or conditions received from your non-standard process devices.

In an ideal world, your devices would always communicate reliably and have no issues carrying out their designated purposes, whether that is collecting weight data, printing product labels or other important tasks.  However, as we are all aware, eventually a device will inevitably get into an error state, at some point.  So most protocols account for the potential error conditions that may be possible by providing a message or messages that the device will send in such a situation.

If you missed our any of our other "OmniServer Did You Know?" posts, get caught up here!

This is why OmniServer protocols have a component referred to as an "Error Message" where such message formats can be implemented and, when they are received from the device, OmniServer can proceed appropriately.

How Can I Do This?

In the example below, OmniServer sends out a command/request message to get the value of DATA (using the READ_PV message). The instrument can potentially send back a NAK signal if, for some reason, the device cannot complete the request.

With the implementation in this example, OmniServer will record this in the three different ERROR items, then attempt to make a correction by issuing the INITIALIZE command/request message. Should that INITIALIZE message also fail, OmniServer will alert the client via the NOTIFY_ERROR host message.

Here is how it works:

• We create the five data items shown below. The data types are: DATA, ERROR_COMMAND, ERROR_NUMBER; ERROR_MESSAGE; RECEIVED_ERROR.
  
  
• Next, we create three command/request messages. The first one, NOTIFY_ERROR, will have no Request nor Response message, only a Notification Item. Why is this?
  
  When this message is performed (via chains from messages described later), OmniServer will have no Request to send nor a Response to expect. Since "nothing" happens, OmniServer marks the message as complete, and sets the Notification Item to One.
  
  Later, other messages will tie into this message via the Failure Chain to inform the client that a failure has taken place.
  
  
  
  
• The second message is the INITIALIZE message. Its job is to send an initialization string to the device once the original command/request message has received a failure. It will only be called via the failure chain of the original command/request message (described in the next section), and it will call NOTIFY_ERROR in its failure chain if it also receives an error.
  
  The important part of this message is the Chains and Triggers section, where you set the Failure Chain to the message NOTIFY_ERROR. This means that, if this message fails, OmniServer will send out the message NOTIFY_ERROR - which, as explained above, will set the RECEIVED_ERROR item to One.
  
  
• The final message is the one that retrieves the data. In our example, this will read the Present Value from the device.
  
  As with the INITIALIZE message above, the important thing here is the Chains and Triggers section. The Failure chain for this message will be INITIALIZE. This means that if this message fails for any reason, OmniServer will send out the INITIALIZE message - allowing OmniServer to initialize the device in case of an error.
  
  
• Finally, we need to design our two Error Messages. The Error Messages in OmniServer are designed as "secondary responses" to Host Messages, so that messages that are not part of the normal response can be evaluated and acted upon.
  
  The first of the two Error Messages, NAK_RECEIVED, is used to trap any NAK messages that comes as a result of a request from the READ_PV message. The important thing here is the Action section on the General Tab. This tells OmniServer how to deal with the message when it is received. In this case, OmniServer is told that the message has "Failed", meaning OmniServer will immediately go to the message's "On Failure" chain (see above).
  
  
• The second error message takes care of the possibility that even the INITIALIZE message will fail. In this case, it will instruct the INITIALIZE message to immediately fail and process the On Failure chain, which will send out the NOTIFY_ERROR message.
  
  As with the previous Error Message, the important selection here is the Action selection, which tells OmniServer to immediately fail the original command/request message should this Error Message be received.
  
  
• Finally, save the protocol.

Once you have tags in your client application associated with the five items in the protocol, OmniServer will send out the READ_PV message in order to get the DATA item. As per normal "Read" operations, it will continue to send this message as long as the client is connected and requesting the items for that message.

However, if the NAK response is received (as described in the NAK_RECEIVED message), OmniServer will look at the Action selected in the error message. The action tells OmniServer to "Fail" the message, in which case OmniServer proceeds to the "On Failure" chain of the READ_PV message. This then processes the INITIALIZE message.

And, if the INITIALIZE message fails, it will be captured by the NAK_ON_INIT message, which fails the message and processes the NOTIFY_ERROR message which results in the "Received_Error" tag in the client being set to 1/True indicating the error.

So here are all the possible combinations of behavior for the protocol items:

• While no errors are occurring:
  • Data=<current value from device>
  • Error_Number=0
  • Error_Message=<blank>
  • Error_Command=0
  • Received_Error=0
  • Everything is normal.
    
    
• If there is an error in the READ_PV Host Message:
  • Data=<last known value>
  • Error_Number=<error number from device>
  • Error_Message=<error message from device>
  • Error_Command=<number of command from device>
  • Received_Error=0
  • Error in processing READ_PV on the device. Corrective action taken by server.
    
    
• If there was an error in the INITIALIZE Host Message:
  • Data=<last known value>
  • Error_Number=<last known value>
  • Error_Message=<last known value>
  • Error_Command=<last known value>
  • Received_Error=1
  • Could not initialize device. Major error detected. Corrective action needs to be taken by the client or user.

Why Would I Need To Do This?

Many instruments will send a error condition in response to an inquiry by OmniServer, or even send out alert signals in case of a detected problem with the device.

Using OmniServer's Error Messages, Notification Items and message Failure Chains, you can design your protocol in such a way that OmniServer will not only inform your client program what error has occurred, but also take corrective measures that attempt to resolve the communication issue without interrupting communications, when possible.

Are There Any Limitations?

You do have to be careful of "endless loops". For example, if we changed the "On Failure" chain for the command/request message INITIALIZE from NOTIFY_ERROR to INITIALIZE, that would create an endless loop.

The logic is this: If INITIALIZE does not get back the appropriate response, then NAK_ON_INIT will execute. It tells OmniServer to issue a Fail for the INITIALIZE message.

OmniServer will then execute the On Failure message for INITIALIZE, which is INITIALIZE. So everything starts back at the beginning. And since there is no way to stop the process (since the odds are that if the device failed once on INITIALIZE it will fail again), OmniServer will be put into an endless loop. Usually the only way out is to restart the OmniServer runtime or to reboot the computer.

Also, this post assumes that the manufacturer of the device designed the protocol with error handling.  While this is typically the case, it's unheard of for a protocol to not have error messages as part of the protocol.  In those situations, the technique used above won't be applicable.

Where can I get more information?

Would you like to start handling your device's error messages in OmniServer?  As always, we're happy to help answer questions you may have.

Email us at support@softwaretoolbox.com with your questions and subscribe to our blog for more quick and easy OmniServer tutorials and tips.  Then, make sure to download the latest OmniServer trial to try it yourself for free.