As of 12:50 UTC all services have returned to normal operation. The root cause of the connectivity issues was due to a hardware fault in a router linecard. This issue is now resolved.
Posted Aug 18, 2018 - 12:52 UTC
Update
We are still experiencing intermittent network connectivity issues - graphs may return partial renders, datapoints are being ingested as normal and the processing of some alerts may still be delayed. We continue to work with our provider to resolve this issue.
Posted Aug 18, 2018 - 12:21 UTC
Update
We continue to experience intermittent connectivity issues, and we have switched our health checking to a more network fault tolerant approach to mitigate the impact of the issue as we work with our provider to identify the root cause of this issue.
Posted Aug 18, 2018 - 11:51 UTC
Update
We are still experiencing connectivity issues which may result in partial graph renders, delayed alerts and delays in processing ingested datapoints. We continue to work with our provider to resolve this issue, and we are working to identify the affected network paths to mitigate the issue in the interim.
Posted Aug 18, 2018 - 11:29 UTC
Identified
As of 11:01 UTC we are experiencing network connectivity issues across several of our services. Ingestion, alerting and graph rendering are all affected by these connectivity issues. We are working with our provider to resolve these issues.
Posted Aug 18, 2018 - 11:11 UTC
This incident affected: Graph rendering, Ingestion, and Alerting.