High latency was experienced on our primary internet carrier’s network.
Immediate Remediation
We failed over to our secondary carrier which resolved the latency issue.
Resolution
Our primary internet carrier vendor resolved the routing issues that was caused by the large fiber span being cut.
We failed back to our primary internet carrier
Analysis Findings/Review
Extreme weather in another region caused trees to fall causing several fiber spans to be cut. This caused some routing issues within the primary carrier’s network that was causing the latency.
The vendor had to re-splice all of the affected fiber links. This repair took a few days, but they were able to route around the impairment until those spans were fixed.
Recommendations
This was caused by weather. RedHelm Cloud has redundant carriers for this purpose. Impact was minimized by having carrier redundancy.
Posted Jul 16, 2026 - 12:41 EDT
Resolved
The failback to our primary carrier is complete. Internet access through our primary carrier is stable.
Posted Jul 07, 2026 - 21:44 EDT
Monitoring
Our primary carrier has reported that their issue is resolved. We will fail back to our primary carrier on Tuesday, July 7th at 8 PM EDT.
Posted Jul 07, 2026 - 01:54 EDT
Update
We are still waiting for our primary carrier to resolve their issue. Internet remains stable on our secondary carrier. We will remain on our secondary carrier until the issue is resolved.
Posted Jul 06, 2026 - 18:49 EDT
Update
The carrier has identified our latency as part of a higher-level issue that they are still investigating. The internet remains stable on our secondary carrier. We will provide additional updates are they are available.
Posted Jul 06, 2026 - 13:05 EDT
Investigating
We started experiencing high latency with our primary internet carrier in the PIT01 datacenter. We have failed over to our secondary carrier and internet appears stable. The root cause is being investigated.
Posted Jul 06, 2026 - 10:25 EDT
This incident affected: PIT01 (Pittsburgh, PA) (Internet Connectivity).