We're currently seeing 100% packet loss on some routes via one of our transit providers, which is causing some disruption to internet connectivity. We're currently investigating as a matter of urgency and will provide further details ASAP.
UPDATE 11:17 UTC: We're no longer seeing packet loss on the affected routes, though are continuing to monitor performance closely. We're continuing to investigate the root cause.
UPDATE 11:42 UTC: Initial investigation suggests the incident was caused by a line card reboot on one of our provider's core routers in London. Further investigation is currently underway and we'll provide updates in due course.
UPDATE 13:05 UTC: The provider is continuing to monitor the affected router closely and it has remained stable since the incident. Senior engineers from their team are continuing to investigate the underlying cause.
UPDATE 14:10 UTC: Investigation has indicated that the line card reboot was triggered by a potential system bug in the traffic flow analysis process on the affected router. Traffic flow analysis has been disabled temporarily while the provider works with the hardware vendor to verify the bug and produce a permanent resolution.
UPDATE 2019-08-06: The affected router has now been stable for 24 hours. The traffic flow analysis feature which triggered the line card reboot remains disabled and our provider is continuing to work with the hardware vendor to identify a permanent resolution. Any necessary corrective actions will be performed during a future scheduled maintenance window.