Sending a device command is weird

Help
1

If I send a command to a device using the name set and following payload:

{
  "on": true,
  "relay": "left"
}

The result on my end is something like this:

{
  "topic": "structure/xxxxx/command", 
  "payload": "{\"name\":\"set\",\"payload\":{\"on\":true,\"relay\":\"left\"},\"time\":{\"$date\":\"2016-04-13T20:46:19.768Z\"}}", 
  "qos": 0, 
  "retain": false, 
  "_msgid": "a4f3545b.5b0ca8" }
}

Why is the payload stringified? This is somewhat unexpected, and requires extra work on my end. FWIW, “my end” is Node-Red, and I need to use a parse node (to call JSON.parse() on the payload field), followed by a change node to normalize the data.

Granted, JSON is not the only thing you can send over MQTT, so this makes sense, but since the UI requires a valid JSON object, I’d expect the command to be parsed before publishing. From the MQTT specification, payload can be anything, yes? I’m an MQTT novice, so pardon my ignorance.

I would expect this message to look something like this:

{
  "topic": "structure/xxxxx/command", 
  "payload": {
    "name": "set",
    "data": {
      "on": true,
      "relay": "left"
    },
    "time": {
      "$date": "2016-04-13T20:46:19.768Z"
    }
  },
  "qos": 0,
  "retain": false,
  "_msgid": "a4f3545b.5b0ca8"
}
2

While I’m here, what’s the significance of $date?

3

Since JSON doesn’t have built-in support for dates, we use the EJSON spec to represent date objects. Basically when time is parsed using EJSON, the value will be an actual JavaScript date object. A little more data can be found in our MQTT docs.

1 Like
4

You are correct, an MQTT payload can be anything. Losant uses JSON as the payload data format. Since MQTT payloads can be anything, your MQTT client doesn’t know that it should automatically parse the data, so it just keeps it as a string. I’m not sure what language you are using as a client, but we do offer a Node.js client that takes care of the parsing for you.

5

Since JSON doesn’t have built-in support for dates, we use the EJSON spec to represent date objects. Basically when time is parsed using EJSON, the value will be an actual JavaScript date object. A little more data can be found in our MQTT docs.

I see; time is the field and $date then represents the “type”. I’ve had to solve this particular problem before; I hand-rolled something at the time, but good to know this exists.

You are correct, an MQTT payload can be anything. Structure uses JSON as the payload data format. Since MQTT payloads can be anything, your MQTT client doesn’t know that it should automatically parse the data, so it just keeps it as a string. I’m not sure what language you are using as a client, but we do offer a Node.js client that takes care of the parsing for you.

Right, but since the Losant UI requires this to be JSON, it follows the MQTT client would expect JSON as well (right?). So why does it need to be stringified before publish?

6

Losant simply applies an opinionated JSON protocol inside MQTT payloads. If you’re using an off-the-shelf MQTT client, it will have no way to know that the broker it’s connected to is using JSON. MQTT clients simply send and receive strings. So in order to send and receive in a way that Losant understands, objects must first be converted to JSON strings before being sent and converted from a JSON string when it’s received.

The JSON protocol only applies to topics that start with losant. You can, if you want, publish and subscribe to to any other topics using Losant’s broker as perform any arbitrary MQTT communication. Losant requires a well-formatted protocol inside MQTT because we need to be able to properly parse and understand the messages to support things like data storage and visualizations.

7

Okay, this is what was confusing. As per the MQTT spec, I understood “payload can be anything” to be literally anything. You seemed to confirm this; however, that’s not actually the case, as you’re telling me now that only strings are allowed, is that correct?

8

Sorry for the confusion - you are correct. MQTT payloads are simply byte arrays. When the payload hits our workflow engine, it is converted to a string in order to execute the workflow. So in order to use custom MQTT topics in conjunction with our workflows, the data will have to be strings.

If you want to use custom MQTT topics to simply publish/subscribe between devices, the data will be sent and received as-is, so you can use raw bytes as needed for this use-case. Here’s an example using the Node.js mqtt client.

var mqtt = require('mqtt');

var client = mqtt.connect('mqtts://broker.losant.com', {
  clientId: '5716ac3ecd56800100cffb21',
  username: '...',
  password: '...'
});

// Connect and send some raw data after 1 second.
client.on('connect', function() {
  setTimeout(function() {
    client.publish('my/custom/topic', new Buffer([0,34,26,32,13,53]), function(err) {
      console.log(err);
    });
  }, 1000);

});

// Subscribe to the custom topic.
client.subscribe('my/custom/topic');

// Will log a Buffer containing the raw bytes.
client.on('message', function(topic, message) {
  console.log(message);
});

If you were to use an MQTT workflow trigger to do something whenever this data is published, you’ll see that those raw bytes are converted to a string that’s mostly garbage.

mqtt-string.png1200Ă—940 50.3 KB

The conversion for running workflows does not affect the original message at all and all subscribed devices will still receive the raw data.

Thanks for bringing this up! I think there’s something we can do in order to facilitate raw data processing in workflows.