I’ve been running a Temp,Humdity, VOC dashboard that is receiving data from a Particle Photon and I’m pushing the data to Losant via the Particle Integration.
Looks like even though the Particle Publish is still working the dashboard is not updating anymore as of 15 hours ago for some reason.
The particle integration does not seem to be having any issues, as far as our metrics show - we are actually processing more particle messages than yesterday. We also have a couple applications using particle, and they are working correctly. If you go to the particle integration in your application, what is it’s connection status?
Could you check the status of your particle integration itself? In the integrations section in your application, you can see the connection status of the particular integration - like the screenshot here.
If it says that it is connected, could you give me your application ID or your integration ID (they will be in the url), and I can investigate more on our side.
I dug into the health of your particular integration’s connection to Particle, and it looked fine, but no data was coming over the channel from Particle. I then caused the integration to disconnect and then reconnect to Particle, and now data seems to be streaming in again (as a note, you can always cause an integration to disconnect/reconnect just by clicking “save” on the integration’s page).
We have not seen this behavior before - we might have to add some additional health checks on our connections to Particle (i.e., if no messages come in over the channel over X time, disconnect and reconnect). I’m going to file a ticket on our side to investigate further.
As a note I had to do as you stated above and click the “Save” button on the integrations page to cause the data being pushed to Losant via the Particle Publish to start displaying on my Dashboard again. I was not accepting successfuly sent Publish events again for some reason.
@Brandon_Cannaday I’ll keep monitoring this and let you know if it keeps happening.
Did Particle have any insights on a cause of this when you talked with them?
It’s basically a Voc, Temp, Humidity sensor sending data every 5 mins 24/7.
I can say it’s been reliable other than these couple events.
We actually have some additional Particle integration health checks going out this week that should help - essentially auto reconnecting if no messages have been received in a while. We did talk to Particle, and they did not have any insight into the source of the issue, but were going to investigate on their side to see if they could find anything (have not heard anything back yet). It seems like it might be an issue with long-running connections - the time lengths between Dec 20th -> Jan 3rd (your first issue) and Jan 3rd -> Jan 16th are suspiciously similar.
Very similar app to what RWB is running, fine for several days, no changes. 12 hour outage last night on the Losant dashboard. Integration with Losant indicates Particle re-connected at 12:30 local time, when the Losant dashboard came back to life. Is there a way to show the historical activity on the Particle integration for debugging purposes?
We don’t have historical tracking on the particle integrations, we only keep track of the latest connection time (or latest connection error, if the connection is failing).
The reconnection today is probably because we rolled out new Particle integration code, with the newer health checks I was referring to above - so that caused every integration to reconnect. These new health checks will (hopefully) detect when Particle is no longer sending events over a currently open stream, and will cause us to close and establish a new connection.
If a Particle connection has previously received events, and has not received any new events in the past 5 minutes, we consider it to be unhealthy, and close it and open a new one.
I’m hoping that I can eventually find a more direct way to detect these “zombie” connections - it doesn’t appear like Particle sends any kind of “keepalive” messages, which would be optimal for detecting this.
Has anyone else been experiencing Particle Integration issues recently? I’ve been running a Particle data feed into a Dashboard for many months without issue, until several days ago. The data will flow in just fine for a few hours (the Particle devices report every 15 minutes) and then for no apparent reason they just stop. Disabling the Integration, saving, and then Enabling it again causes the feed from Particle to start working again. It is highly repeatable and happens after just a couple hours of completely normal operation. Nothing is changing on the Particle side, so this seems to be some new issue in Losant.
and while we are on the subject of Particle Integration, is there a reason that Losant chose to ‘pull’ data from Particle, rather than using their Webhook which offers a ton of useful features. Allowing Particle to push data would seem to me to be a far better option than pulling. Losant is doing a lot of pulls when there are no new events to retrieve which seems highly inefficient.