Monitoring - A fault within our hosting providers infrastructure has resulted in severe network issues across all our services.

Between 10:25 and 10:29 UTC this resulted in the following impact:
* Ingestion: Our ingestion layer successfully processed all data-points sent, however, during the outage we saw a large backlog of data between the ingestion layer and the aggregation layer. This would have lead to gaps in graphs, and possibly erroneous triggered alerts, until this data was successfully replayed at 10:45 UTC.
* Graph Rendering: 50% of renders failed during the affected period. This will have caused alerts to trigger in error.
* Website: The website may have been unavailable for a short time during the incident.
* Alerting: Valid Alerts may not have been triggered during the incident.
Jun 17, 10:49 UTC

Monitoring - From 15:13 UTC TO 15:30 UTC, approximately 10% of connections made to ingestion endpoints would have failed, this also affected 10% of UDP traffic. Any received data has not been lost.

We have tracked down the issue to a recent configuration change and it has been reverted.
Jun 14, 15:40 UTC