31 May 2012 10:20:57
[Maidenhead Colocation] Power outage in Maidenhead data centre - Closed
Posted: 29 May 2012 22:23:38

IT looks like a few severs rebooted, so suggests a power problem. Trying to get some more information now.

IT would affect web and email hosted services as well as VOIP.

30 May 2012 00:09:53

It has been confirmed that this was a problem with power. It is being worked on.

There are a few minor problems, like one of our spam checking servers is offline (the other servers should cope), one of our customer facing SMTP relays died (no longer in DNS so not customer affecting) and customised voicemail messages are not working on our VoIP service.

The accounts system web interface is down too (although the accounts system itself is fine).

These are not major problems, and we'll continue to work on them in the morning.

30 May 2012 00:54:11

It looks like there are some ongoing connectivity problems with Palsant's transit feed. They are manifesting themselves as brief outages, but it doesn't look too serious.

We'll post details when we know more.

30 May 2012 08:58:27

Power went off at 22:07:47 and was restored at 22:10:59

Engineers are on site this morning to resolve remaining issues with some servers.

Transit seems to be stable now.

30 May 2012 09:55:53

Most services are working correctly - some issues with voicemail/announcement messages at present, but this is being worked on. One of the outgoing mail servers has not come back to life and will be worked on this morning so that the mail queue can be sent ASAP.

30 May 2012 18:30:13
All services are working again now. The data centre have completed their investigations and have determined that the outage was caused by a faulty UPS component which failed during planned maintenance. The power feed to the data centre is currently running without UPS backup and should be seen as at-risk in the event of a mains power outage. The data centre staff expect to have the UPS back in-line by 22:00 this evening.
Started 29 May 2012 22:07:47 by AAISP automated checking
Closed 31 May 2012 10:20:57