Open Mind Status

As part of our long term commitment to continually raising the level of hosting service that we provide , we are undertaking a major capital investment programme to improve and upgrade our entire network.

The net result of this programme will be a network with greater capacity and redundancy, complete with a ‘future proofing’ capability enabling us to offer you new services like IPv6 and MPLS in the very near future.

We are planning to implement this upgrade in two phases over the coming weeks, with the first scheduled window being 1st December between the hours of 01:00 am and 06:00 am GMT. During this window clients will experience a very short period of connectivity interruption of less than two minutes as we change the routing configuration from the current equipment to four fully redundant network segments. The second phase, details of which will be released in a separate communication, will involve the migration of the rack uplinks to the new network segments. Once again as a consequence of implementing Phase 2, clients will also experience a period of less than two minutes of connectivity interruption.

We apologise for any inconvenience this may cause, and hope that you’ll appreciate that we are undertaking this action to ensure that we can continue to provide the best hosting service available.

On 2nd October 2010 between the hours of midnight and 8am, the data centre housing our shared, reseller and dedicated server clients will be carrying urgent and substantial maintenance to the primary power supply system and this is advanced notification of the expected downtime schedule that will affect the majority of our client services. A full breakdown, schedule and client action is given below:

Summary

* The data centre will be enacting our procedures for a controlled power down and power restore to RSH North. This will happen between the hours of 00:00 and 08:00 on Saturday 02/10/10.
* These procedures are being put into action to allow essential repairs to the main power systems in RSH North. For further details on these repairs please see ‘Technical explanation of cause’.
* As part of procedures for restoration of power there will additional staff on site. They will be checking that all services have power restored and pro-actively investigating any issues reported by our monitoring system.

Sequence of events

00:00:

* Our monitoring will begin automatically executing a soft shutdown on affected servers.

01:00

* Mains power will be systematically shut down.
* Your service will not be available during this time.

05:00

* We aim to begin restoring power systematically across the site, in stages.
* Technical staff will begin pro-actively investigating and restoring service to any problematic servers based on the monitoring system.

08:00

* we aim to have all servers back online.

Technical explanation of cause

* The data centre recently installed a fourth 500kva UPS to add resilience and capacity to our power systems.
* The data centre was designed with this in mind, so the appropriate connections exist on the Schneider UPS Output Panel (that feeds all servers from a common busbar) to connect this UPS.
* During the installation of the UPS, engineers discovered there was a fault with the panel which meant the additional UPS could not be connected.
* To repair this fault, we need to electrically isolate the panel, as engineers cannot work on the panel while it is live for safety reasons.
* The manufacturer of the panel, Schneider, is sending a team of engineers to repair the panel so it can accept the fourth 500kva UPS.

Scheduling an automated shutdown on a server
We are strongly advising all unmanaged dedicated server clients to put in place on of the following methods dependent on the OS installed.

Shared/reseller hosting clients and managed dedicated server owners do NOT need to take any action as we will be powering the server down for you. VPS servers without a “VS” prefix will NOT be affected by this maintenance window and do NOT need to take any action.

Windows Server 2003

1. Click Start, All Programs, Accessories, System Tools, Scheduled Tasks
2. Double-click on Add Scheduled Task, and Click Next
3. Click Browse… to select the program to run
4. Enter C:\Windows\System32\Shutdown.exe, and click OK
5. Under “Perform this Task”, click “One Time Only”, and press Next
6. Enter the date and time to shutdown your computer: 00:00 on 02/10/2010, and click Next (BE SURE TO ALTER THE TIME IF YOU’VE CHANGED YOUR TIME ZONE!)
7. Enter your password in both fields and click Next
8. Check “Open advanced properties for this task when I click Finish”, and click Finish
9. Add “/s /t 30″ to the end of the Run box, so it looks like the following:
C:\WINDOWS\system32\shutdown.exe /s /t 30
10. Click OK, Reenter your password, and click OK
Windows Server 2008

1. Click Start, All Programs, Accessories, System Tools, Task Scheduler
2. Click “Create Basic Task” on the actions pane on the right-hand side
3. Enter “Shutdown” as the Name, and click Next
4. Under “When do you want the task to start”, select “One Time”, and click Next
5. Enter the date and time to shutdown your computer: 00:00 on 02/10/2010, and click Next (BE SURE TO ALTER THE TIME IF YOU’VE CHANGED YOUR TIME ZONE!)
6. Select “Start a Program”, and click Next
7. Enter “shutdown” (without quotes) in the “Program/Script” box, and enter “/s /t 30″ (without quotes) in the “Add Arguments” box, and click Next
8. Check “Open the Properties dialog for this task when I click Finish”, and click Finish
9. Click “Run whether the user is logged on or not”, and click OK
10. Enter your password, and click OK
Linux

Ensure you have the “at” service installed, for running tasks on a schedule. This can be installed in the following way:
Debian 5: aptitude install at
Debian 4: apt-get install at
CentOS: yum install at
Fedora: yum install at

1. Log into your server using SSH
2. Enter: “at 00:00 Oct 02″ (without quotes) and press Enter
3. After the at> prompt, enter “/sbin/shutdown -h now” (without quotes) and press Enter
4. Enter CTRL-D
5. Enter “atq” and press Enter to check that

DNS services

* During the maintenance the DNS name server services provided by us will cease to operate with the exception of ns2, ns5 and ns6.nixcontrolpanel.com name servers.
* It will not be possible to change your DNS records during the maintenance.
* Please ensure any DNS changes for holding pages are made prior to 00:00.

We realise that this is a large scale maintenance window and you will have received notification on how to claim a credit for your account despite our network SLAs having no provision for scheduled maintenance such as this.

Maintenance Type: Infrastructure
Expected effect on your service: No Effect
Expected downtime duration: 0 minutes
This will occur between 08:00 and 18:00 on 05/07/2010 (UK Time)

The following servers are relevant to this message:

IS-03093, IS-03384, IS-03587, IS-03711, IS-03712, IS-03713, IS-03714, IS-03715, IS-03716, IS-03717, IS-03718, IS-03719, IS-05965, IS-06110, IS-07420, IS-07759, IS-07931, IS-08077, IS-08313, IS-08532, IS-08817, IS-08859, IS-08985, IS-09830, IS-09906, IS-10952, IS-11329, IS-11743

On the 5th of July 2010 there will be a maintenance window to expand the capacity of the UPS systems. An additional UPS control unit and its associated batteries will be added to the existing cluster. This is to ensure the UPS system continues to provide fully redundant power backup as the site grows. This work creates an “at risk” period for your service. This maintenance is not service-affecting, but during the maintenance period services are at an increased risk.

During the maintenance period the critical load will be transferred seamlessly from the UPS to the raw mains. When the maintenance is complete it will be transferred back seamlessly.

We have confirmed with Southern Electric that they have no maintenance scheduled during this period.

We are currently migrating the primary Open Mind Hosting domain to a new server so you may experience minor issues accessing your account whilst the IP changes propagate.

This should be fully resolved within 24 hours.

We are pleased to report that all systems are currently stable.

We are currently experiencing a major outage at the data centre our newtork is located within.

This is affecting all shared/dedicated clients.

Currently we are waiting for the data centre to update us as to the nature of the problem and expcted resolution.

Updae 01:45 – The network is now back online, we are communicating with the data centre to find the cause of the outage.

This is an important update, please read it carefully and contact us if you have any questions.

We will be performing maintenance which will affect your service. This will last about 20 minutes, and during this time, your service will go offline.

Date: 21 July 2009
Window: 02:00 AM for 1 Hour
Duration: < 20 minutes

This maintenance is to upgrade the IOS firmware on our RSH.North core router, from our new Maidenhead facility. Currently the network is configured with dual uplinks from our core router to the edge switches in each rack. This upgrade will enable clustering capabilities on the core router and once the core router will be configured as a cluster, it will increase the availability of the service in case of a core router failure.
This will improve redundancy across our network ensuring maximum availability for all our services.

We apologise for any inconvenience this may cause, please do not hesitate to contact us if you have any queries or questions regarding this maintenance window.