The primary data centre is currently having some network connectivity issues and this is affecting a small proportion of our servers causing timeouts and unresponsiveness.
We are monitoring the situation and will keep users updated with more information as and when we have it.
Update 10:38
Date: 18/10/2010
Time: 09:20
Duration so far: 1 hour, 18 minutes
Affected Service(s): IS-03093, IS-03384, IS-03587, IS-03711, IS-03712, IS-03713, IS-03714, IS-03715, IS-03716, IS-03717, IS-03718, IS-03719, IS-05965, IS-06110, IS-07420, IS-07931, IS-08077, IS-08313, IS-08532, IS-08817, IS-08859, IS-08985, IS-09830, IS-09906, IS-11329, IS-11743, IS-11959, IS-12110, IS-12290, IS-12441, IS-12812
There is currently a network issue affecting RSH North. At this time we believe this to be caused by a network routing problem at one of our peering connections. This problem is only affecting a proportion of traffic via RSH North.
Update 11:17
The issue affecting RSH North is ongoing. A proportion of the traffic into RSH North will currently be experiencing increased latency and packet loss at varying levels. In some instances this may appear to be complete connectivity loss.
We are investigating the cause and will provide further updates as soon as we possibly can.
Update 13:24:
The initial problem with a peering point that occurred earlier this morning has led to a problem within the VSS routing cluster that services RSH North. Our network team quickly eliminated as many causes as possible. This issue was then escalated to Cisco and we are currently involved in joint investigation to try and discern the underlying problem. As soon as we have any progress from this work we will inform you immediately.
Update 13:59:
Network connectivity has now been restored across all the servers however we are waiting fr an official all clear from the data centre before showing this issue as resolved.