T3 Connect creator blames failures on Cloudflare tunnel rate limits
Theo attributes T3 Connect failures to tunnel provisioning and decommissioning rate limits, not tunnel counts. He says broken alerts hid the severity and reports service recovery.
TL;DR
- T3 Connect's remote-access service should have recovered, its creator said in an incident update.
- Tunnel provisioning and decommissioning hit endpoint rate limits, according to his clarification, despite headroom on tunnel counts.
- Broken alerts concealed the incident's severity, the creator acknowledged in a support reply.
T3 Connect's relay allocates tunnels without proxying ordinary HTTP or WebSocket sessions. Cloudflare's API documentation lists a global budget of 1,200 requests per five minutes per user, alongside separate limits for some API calls.
Tunnel provisioning limits
Theo, T3 Connect's creator, put the deployment at roughly 60,000 tunnels and pinpointed the bottleneck to the endpoint used to provision and decommission them.
T3 remained below the tunnel-count limits it knew about, he said in his incident update. He also said Cloudflare had confirmed the issues on its side and was working on fixes.
The endpoint's actual rate-limit threshold was not disclosed. Tunnel-count headroom proved a poor proxy for lifecycle capacity.
Broken alerts and reported recovery
Onboarding was among the affected flows: Theo apologized to a user in an early support reply while saying T3 Connect was down.
He later acknowledged that broken alerting had hidden the incident's severity, accepted responsibility, and said the connection command should work again.
Client versions and thundering herds
T3 cannot guarantee that users run the latest client version. Theo said a thundering herd becomes difficult to contain because the team cannot control code already running on users' machines, forcing changes to be planned ahead of time.
Startup retries for temporary HTTP 408, 429, and 5xx failures can continue for up to ten minutes, according to the project's remote-access documentation. Theo did not attribute the herd to a specific retry implementation.
Tailscale fallback and diagnostics
Theo said in the same update that he personally keeps Tailscale as a fallback and welcomed users doing the same.
The documented Tailscale HTTPS path requires both devices on the same tailnet:
- New command-line server:
t3 serve --tailscale-serve. - Already-running server:
t3 pair --tailscale.
For machines still having trouble, Theo asked users to run npx t3 triage on the affected host and report back.