Polling external API beyond the 60 secs limit of workflow


#1

Trying to figure out a way to poll an API for a status update that may take longer than the 60 secs workflow timeout to resolve. I have read some other workarounds about using a Timer trigger and a conditional block to monitor the duration since the last poll interval as a way to get around the 60 sec limit, but this would result in a ongoing polling of the external API.

Ideally, I would use this workaround if I could programmatically start (and terminate) a Timer trigger when I want to start polling an API for status updates. Once to final status is returned, I could then stop the timer trigger for this workflow. Short of this, I would have to build a Timer workflow that references a “polling” data table and pick up “jobs” (API endpoints and expected response schemas) that could be populated by other workflows that need to poll results. The Timer workflow could then purge the “job” row from the “polling” data table when the successful result is returned.

I am playing around with a Delay node + circular webhook workflow at the moment to see if that will work as an alternative.

Any thoughts on other / better approaches this problem (other than only working with external API’s that support webhooks)??


#2

Unfortunately, the solution that you want only currently exists as a ticket in our system :slight_smile: We have a spec for a concept of a workflow equivalent of Javascript’s setTimeout / cancelTimeout, which would solve your problem in a reasonably neat way. I will add a +1 on that ticket.

One thought is that you don’t specifically need to use a data table for “jobs”, it might be easier to use the get/store value nodes, but in general your idea of a “job queue” is probably the cleanest way to solve the problem in a generic way at the moment.