We're investigating problems with our web hosting.
There is a problem with the file server that hosts the files for web hosting.
The RAID array for the web hosting has broken. This is a simple two disk RAID1 mirror. It looks like the mirror became degraded overnight but for an unknown reason the RAID controller also removed the remaining drive! We are attempting to rebuild it, but if that fails we'll need to restore from backup.
We've managed to bring the RAID array back online, we are running a filesystem check as it looks like some files, or file attributes are corrupt.
Unfortunately, due to the problem we are having with web hosting we are needing to reboot the file server that manages email and web. This is happening now, and will mean access to email will be unavailable for a few minutes. We do apologise for this.
The file system did not come back after the reboot. A repair has been completed and the file system is mounted again.
Sadly, we're needing to restore websites from backup, This will be starting soon but it will take some time for data to be fully restored. We'll update on the progress at around 1PM though.
Websites are being restored from our Wednesday morning backup. About 16% of the files have been restored so far. We expect this to take a few more hours to complete.
Over 25% now.
Over 50% now.
Backups have been restored, and FTP/rsync are now live again. Web access logs have been lost - sorry for any inconvenience.
Some customers are unable to ftp or rsync files to their web space. This is being worked on now though.
The job to repair the ftp/rsync issues at this moment in time is still running.
FTP/rsync is now repaired, and access to the web space should be back to normal. Unfortunately, during the restore process, weblogs were not recreated for any domains using them. This means that customers will need to log back in to the web space to re-enable logs. Sorry for any inconvenience that causes!