Intermittent Blank URLs in Emails

I’ve been generating daily reports using the Output HTTP Node and sending the URL via SendGrid. This setup has worked well for over a year, but recently I’ve started intermittently receiving blank URLs in the email. I’m not sure whether the issue is within Losant or SendGrid.

I cannot reproduce it when triggering the workflow manually, so is there a way to store the workflow debug output in a file for troubleshooting later?

I would do something like the following …

  1. Add a Conditional Node before your HTTP and have it check the value of the template you are using for the URL, i.e. an expression of {{working.myUrl}} or whatever the value is.
  2. The true path goes to your HTTP Node.
  3. The false path goes to an Event: Create Node. Give it a subject, description, and level of your choosing; the important bit is that the “Additional Data” field is populated as a payload path of . - which means “the whole payload”.

There is one gotcha there in that if the payload is excessively large, the event creation may fail. I would test this first to ensure you are catching the issue and creating an event with a payload close to what you expect in real cases.

As an added bonus, you could also set up an Event Trigger to fire another workflow when this occurs and, off that trigger, alert you to the problem in some way (i.e. using an SMS Node or an Email Node). Under normal circumstances we recommend against using these nodes in favor of ones tied to an account you own, but I’d consider this “non-critical functionality” so it should suffice for this use case.

Thanks @Dylan_Schuster. It took me a while to confirm the issue since it’s intermittent, but it seems like the URL is blank before being sent to SendGrid, which indicates the problem lies within Losant.

As mentioned before, I’m using the Output HTTP node to send a dashboard snapshot to a Losant webhook. Then, in the workflow triggered by that webhook, I use both the SendGrid and Email nodes to send the link from {{data.body.downloadUrl}}. However, both emails are coming through with an empty link, which points to an issue within Losant.

Is there a way to add more debugging or logging to help pinpoint the root cause?

So in the cases when it’s failing …

  1. The Output HTTP Node - which I assume is calling the Dashboard: Send Report API endpoint with a callbackUrl pointing to a Losant Webhook - is returning a successful response.
  2. The Webhook Trigger is firing, and a blank URL is being sent to SendGrid using the SendGrid Node.

The problem being intermittent indicates that your string template for rendering the report URL from your payload is correct. So, some ideas …

  • Could there be something else in your application hitting your webhook URL? Webhooks are public but the chances of something outside of your control making requests to it are slim.
  • Is an error response being sent to the webhook, meaning it would not have a value at data.body.downloadUrl?
  • Similarly, are you verifying that data.body.downloadUrl exists in the payload before invoking the SendGrid Node? You could do this with a Conditional Node checking an expression of !!{{data.body.downloarUrl}}, which will cast the value to a boolean and will reject cases where downloadUrl is falsy (undefined, null, 0, empty string).

As for catching problematic payloads, my advice is the same as before:

The false path goes to an Event: Create Node. Give it a subject, description, and level of your choosing; the important bit is that the “Additional Data” field is populated as a payload path of . - which means “the whole payload”.

If you can catch such a payload we can investigate further as it will have the Dashboard: Send Report response in the body.

Yes, you’re right. The webhook is returning a successful 202 response, and the Webhook Trigger is firing, but data.body.downloadUrl was blank. I verified this by emailing myself the link using both SendGrid Node and Email Node.

The webhook URL is exclusively used for this Output HTTP Node in my application. I could try changing the webhook URL, but I’d prefer to understand the root cause before making any changes.

I also added the Conditional Node check as you suggested, but interestingly, it didn’t trigger. Does that mean data.body.downloadUrl wasn’t falsy at the time it was created?

So I popped into your application and you do indeed have the Conditional Node set up as I described, and I tested it with a Virtual Button press.

However, I also see that you have several applications that are essentially clones of one another, all with the same workflows that kick off a report generation and send it via callback URL.

My first thought is that something got missed when making these changes across all the applications - a Conditional Node didn’t get changed, or a callback URL is referencing a webhook from a different application, or a context variable is referencing a device from a different application than the dashboard belongs to. Something like that. I know I’ve made mistakes like those before.

As to why dashboardUrl would not be set in the callback body … If we encounter an error when building the dashboard report, we will still hit the callback URL provided. But instead of data.body.dashboardUrl, you’d have a property of data.body.error, whose value would be a string describing the encountered error.

To try and deduplicate some of this work - simplifying the management as well as reducing the possibility of errors - I would consider housing the webhook in a separate “management” application. The callback URL in each application doesn’t have to go to a webhook in the same application that made the request. And if there are application-specific values you need when the callback fires, you can pass those as query parameters in the callback request and read those in the webhook trigger.

For example, set the callback URL to https://triggers.losant.com/my-trigger-key?emailSubject=ABC Company Subject&emailRecipient=bobby@abccompany.com. Then, those properties are available in the webhook’s payload under data.query.emailSubject and data.query.emailRecipient.

The most likely error you would encounter when building a report is a context validation error - such as providing a deviceId context variable value from Application A in a dashboard that belongs to Application B. Any other errors you would encounter would bubble up to our engineering team as a bug to investigate.

Update, we did find a case of one of your dashboard reports failing to send; and looking in one of your applications, you did catch it in a logged event.

You’re not the only user affected by this - the occasional unexpected report generation failure. We’re investigating further and will let you know what we find.

That said I would still leave the error handling we described in this thread in place so you can be alerted to issues in the future.

Thanks for bringing this to our attention.

We’ve identified a fix for this and expect it to be released next week. I’ll follow up with you once it is out.

Thanks again for reporting the issue and apologies for the error.

Thanks @Dylan_Schuster for the follow-up on this. Glad to hear the Losant team identified the issue and will be releasing a fix soon!