Request for Workflow to be able to ignore old data

Feature Request
1

Hi

Had a situation, where the Losant Edge Agent was offline for a while.
Then when it comes back online we have old data being streamed into the workflow with new data.

In this specific case of a workflow it meant old data and new data is being intermixed, when the workflow was designed to only deal with current inbound data (or at least in order delivery of date).

We could see the interleaved arrival as dashboards would flick between 19min ago, and a second ago, and 18min ago etc…

It would be nice to have the workflow be able to ignore old data. I know I can put logic into the workflow however it would seem cumbersome and would require a lot of code repeated everywhere.

Thanks

Tim

2

Hi Tim,

In our latest release of our edge agent, we added a new flag called QUEUE_OFFLINE_MESSAGES. If you set this to false it will no longer send old messages, it will just discard them when the device is not online.

Hope this helps,
Erin

3

Hi

Thanks for that, in this case I am referring to cloud workflows, and filtering out/ignoring out of order data.

Cheers

Tim

4

It’s an interesting idea and something we’ll have to think about. Could potentially be added as an option to the Device State trigger. If a new state comes in that is older than a previously triggered state, skip it.

The workaround for now is to use Workflow Storage. Set the key to the device ID, which is available on the payload at {{ triggerId }}. Then set the value to the date on the payload. Each time a message comes in you can compare what’s in workflow storage to the incoming value. If it’s newer, accept the message and update the storage value. If it’s older, do nothing.

5

HI Brandon.

Yes I think your suggestion for device state would be ideal.

I have already implemented your workaround.

But it does involve multiple nodes (get/store) transform date/time string from the time property of the payload to a timestamp etc and a comparison. So there for has a lot of moving parts.

The new feature of being able to create custom nodes does make this re-useable.

6

You should totally contribute that custom node to the catalog :slight_smile: