Hi Team,
Our custom domain is intermittently returning a Cloudflare Error 522 (Connection Timed Out) — meaning Cloudflare’s edge reaches the origin, but the request never completes.
We’ve confirmed the DNS/CNAME setup matches what Losant specifies, the Experience Version is active and correctly mapped, and the underlying workflows/code appear to be functioning normally otherwise. Trying to figure out whether this points to something on Losant’s infrastructure side or a config issue on ours (e.g., Cloudflare SSL/TLS mode).
Please reach out to me via secure email chain if team needs more information regarding the issue.
Best regards,
Surya
We’ve had another user report a very similar issue. We’re looking into this currently but have not been able to isolate a cause yet.
Would you mind PM’ing me details about your Cloudflare setup and any request logs you can extract from the service? That might help us get to the bottom of the issue.
Update on this, we are able to reproduce the issue, though intermittently. We’re investigating now and will keep you updated on what we discover.
We may reach out for more information from you.
Thanks agin.
Just another followup: We’ve done quite a bit of testing and have confirmed that the issue actually lies with Cloudflare, where certain edge locations (Chicago, Paris, Seattle) have intermittent failures when resolving to Losant’s servers.
One of our other customers is sending the results of our testing to Cloudflare and is hopefully working with them to resolve the issue. We don’t have a timeline currently.
Unfortunately the workaround is to take Cloudflare out of the mix and not proxy your subdomain’s requests through it.
I will update this thread as we get more information.
We contacted cloudfare support team. They are requesting for fresh timeout error information such as :
-
A fresh failing URL and the full Ray ID from the 522 page, plus the UTC timestamp of the failure.
Ray ID: a3e9bd1d5bfe2251
UTC timestamp: 2026-09-21 14:25:41
URL: mxcloud.live/
-
Whether the failures are concentrated in one or a few data centers, or spread across all locations, from your analytics during the same window.
col : Dallas (shown as “Working” but this is the colo that served the failed request to origin)
-
Origin or hosting-side firewall, connection, and load metrics for the same UTC window.
-
If possible, an MTR or traceroute from the origin toward the Cloudflare connecting IP seen in the origin logs during a failure.
Could I please have the information for the 3 and 4 questions so I can provide all the information to the cloudfare support team.
Regards,
Surya
I’ll see if I can get what you’re asking for out of the engineer who is handling this on our end.
FYI we are still actively engaged in this issue with another customer and are currently going back and forth with Cloudflare support to get this resolved. Your issue and theirs are related and fixing one will fix the other.
Apologies for the delay in getting to the bottom of this, but we are dependent on Cloudflare support to identify where the requests are going within their system before it even reaches Losant’s servers.
Thanks for the update — good to know this is already being tracked and tied to the other customer’s case. I understand it depends on Cloudflare identifying the issue on their end first, so no worries on timing.
One thing I wanted to flag: even I run into these timeouts myself as an engineer using our own domain, having to reload the page each time. What I’m most concerned about is our MX Cloud users experiencing the same thing day to day. I’d really appreciate this being resolved as soon as possible so they have a smooth, reliable experience on the platform.
Please keep me posted as you hear more from Cloudflare, and let me know if there’s anything further you need from our side in the meantime.
Hi Team,
Wanted to share an update from Cloudflare support on the 522 timeout case.
They reviewed the two fresh failures we captured:
-
URL: mxcloud.live/
Ray ID: a3e9bd1d5bfe2251
UTC timestamp: 2026-09-21 14:25:41
Colo: Dallas
-
URL: dev.mxcloud.live/
Ray ID: a3f1fa246ba5f0ad
UTC timestamp: 2026-09-22 14:25:26
Colo: Dallas
Along with our comparison across domains - mxcloud.live and dev.mxcloud.live (both proxied through Cloudflare) have hit 522s at the timestamps above, while mxdev.onlosant.com (not proxied) has never returned one.
Based on this, Cloudflare has ruled out origin-load/capacity as the cause and confirmed this points to a Cloudflare-to-origin path issue localized to Dallas. They’ve escalated the case to their engineering team for further investigation.
The one piece still missing is origin-side evidence for the two UTC windows above: firewall logs, connection metrics, or routing information from Losant’s infrastructure. Cloudflare specifically asked if this could be forwarded once available, since it would help their engineering team pinpoint the issue faster.
Could you let me know if your engineer has been able to pull this for those windows, or give an idea of timing? Once I have it, I’ll pass it straight along to Cloudflare to keep the case moving.
Thanks again for staying on top of this alongside the other customer’s case - happy to share anything further needed from our side.
Regards,
Surya
So we don’t have those kinds of low level network logs, but I can share the information for testing that we did for the other ongoing Cloudflare support ticket:
- We can confirm we are not blocking any Cloudflare IP ranges in our firewalls or other protections. In addition, to our knowledge the underlying infrastructure provider (Google Cloud - us-central) also does not block anything on their own.
- Traffic is not hitting network-level rate limiting - we see no indication that our load balancer is dropping any traffic due to rate limiting.
- Backing VMs are not dropping network traffic due to network or other load - failures are reproducible throughout the day, independent of load.
- We see no indication of the failed requests in our application level logs - which makes sense for a Cloudflare 522 syn/ack error.
- Keepalives are enabled.
- It is not possible to run an MTR from the Losant to Cloudflare IPs that actually matches how inbound traffic works. This is because traffic over connections initiated from within Losant’s infrastructure travels through a NAT, which is significantly different routing than how inbound traffic routes (a network load balancer).
- We confirmed that outbound traffic in response to a connection through the load balancer does correctly go out with the IP of the load balancer, so it is not a case of asymmetric traffic routes.
- We did tcpdumps and analyzed traffic from Cloudflare IP ranges while we were making requests that produced the issue, and our logs showed that every incoming SYN from Cloudflare was correctly replied to with an ACK. We were unable to specifically correlate lines in the tcpdump logs to the failures as there is plenty of other working Cloudflare traffic at the same time. However, in the logs we did not see any of the expected “retry pattern” that would come from a 522 error (Cloudflare is specifically supposed to retry 8 times at a specific frequency over the 19-second window) - which suggests (assuming we are looking for the correct pattern, and that Cloudflare performs its retries from the exact same source IP) that our backing VMs are not receiving the SYN at all (as opposed to our reply SYN+ACK not being received by Cloudflare).
Here is a table of our testing against different locations from two weeks ago (although not for mxcloud.live, this was for a sample site we set up).
| Colo |
Location |
Failure rate |
| EWR |
Newark |
~40% (8/20) |
| SEA |
Seattle |
~21.7% (55/254 across two sessions) |
| ATL |
Atlanta |
~13.0% (7/54) |
| ORD |
Chicago |
~19.2% (147/767 across 5 sessions) |
| DFW |
Dallas |
~8.7% (4/46) |
| CDG |
Paris |
~14.3% (57/400 across two sessions) |
| BOS |
Boston |
0% (0/263) |
| PDX |
Portland |
0% (0/146) |
| YYZ |
Toronto |
0% (0/277 across multiple sessions) |
| DUB |
Dublin |
0% (0/28) |
| MAN |
Manchester |
0% (0/30) |
| CVG |
Cincinnati |
0% (0/400 across two sessions) |
| CMH |
Columbus |
0% (0/400 across two sessions) |
| MCI |
Kansas City |
0% (0/200) |
One interesting item, YYZ started having failures yesterday in some additional testing - so it is possible that this table is no longer accurate in other respects.
We also did some additional testing about a week ago that added a proxy in between Cloudflare and Losant:
1. **Cloudflare-fronted** (cpf.dylanschuster.me): client → Cloudflare → 6aa056add96ed0811e13d972.onlosant.com (i.e. 146.148.110.247).
2. **Cloudflare-fronted, via intermediate proxy** (cpf-proxy.dylanschuster.me): client → Cloudflare → a proxy server we control at 136.110.173.223 → 6aa056add96ed0811e13d972.onlosant.com.
3. **Direct to origin** (6aa056add96ed0811e13d972.onlosant.com): client → 6aa056add96ed0811e13d972.onlosant.com. No Cloudflare involved at all.
In four separate sessions, paths 1 and 2 happened to land on the same Cloudflare edge location as each other at the same time:
| Requests Initiated From |
Cloudflare Location |
Path 1: Cloudflare → origin |
Path 2: Cloudflare → proxy → origin |
Path 3: direct to origin |
| Residential Indianapolis |
ORD |
71/200 failed (35.0%) |
0/200 failed (0.0%) |
0/200 failed (0.0%) |
| A server in AWS us-east-2 |
CMH |
0/200 (0.0%) |
0/200 failed (0.0%) |
0/200 failed (0.0%) |
| A server in AWS eu-west-3 |
CDG |
50/200 failed (25.0%) |
0/200 failed (0.0%) |
0/200 failed (0.0%) |
| A server in GCP us-west-1 |
SEA |
40/200 failed (20.0%) |
0/200 failed (0.0%) |
0/200 failed (0.0%) |
Hope this information helps!
Hi team,
Just wanted to say thanks again for putting all that testing together - it really helped. I shared it with Cloudflare and they said it was enough on its own to point their network team in the right direction. They’ve added everything, including the proxy test and the colo table, to the open escalation on their end.
One more thing they mentioned that could help narrow it down even further - if you’re able to grab the Cloudflare connecting IP along with the Ray ID next time there’s a fresh failure, that would be useful. I think that’s something only visible from your origin logs at the time it happens, so not sure if it’s easy to pull or not.
No rush on this, just passing it along in case it helps for round two. Thanks again for sticking with us on this.
Regards,
Surya
