The data centre are continuing to have severe network issues that will be affecting all our servers.
We are still waiting for the all clear from the data centre but in the mean time users may experience a slowness in connection, timeouts or lag when browsing sites or using services.
Please bear with us during this period as unfortunately the issue is out of our control but we are working closely with the data centre to get this resolved asap.
Update 18:22
The following has now been received from the data centre:
Below is a summary update of the issues experienced by clients in RSH-North today and yesterday, 25th and 24th September 2009.
Fundamentally we have experienced serious issues that have affected all clients in RSH-North. This has been due to the Cisco equipment at the core of our Spectrum House services not responding as per specification and documentation.
As has already been explained, we have been in direct contact with Cisco, working on a Level 1 priority request to solve the issues that have affected our clients today. We take full responsibility for our vendor selection and do not wish to appear to be passing blame “conveniently”. We pride ourselves in the level of service that we provide and also in the quality of communication that we send to clients. We accept that neither have been anywhere close to our usual standard during this prolonged incident. However, we would like to take this opportunity to clarify a few points that we are aware have been questioned and discussed by our clients and competitors:
- Spectrum House routers were configured in a redundant VSS cluster
- CPU usage was recorded as being very high for normal usage
- Cisco have offered two possible solutions to the issue, including a firmware update provided to us yesterday. These solutions have failed to resolve the issues experienced
- Our priority has, at all times, been to provide stable service for as much time as possible, to as many clients as possible. This has been the endeavour and sometimes this has not been possible. We do not shirk the responsibility for this matter and recognise the impact that it has on our clients’ businesses and both our and their reputations.Our network team are continuing to investigate why some of our clients are still experiencing outages, interrupted service and packet-loss and are doing so with Cisco.
As soon as we have further updates we will be posting them directly here.