This is a follow-up to https://aastatus.net/2222 and https://aastatus.net/2229
As we were not able to complete the planned work that was scheduled for Thursday 17th March we are planning a third visit on Thursday 24th March at 2am.
We are planning to replace our core network switches in Telehouse early Thursday morning. This is part of our network upgrades to increase our capacity. The work is being carried out in the early hours of Thursday to minimise impact on customers.
We do expect Internet connectivity to be affected temporarily as we move cables around. We aim to minimise disruption but there will be times when broadband customers will lose PPP and be offline for a few minutes. Other customers and wholesalers who we provide connectivity to will notice that BGP sessions will be taken down and brought back up one by one. Staff will be on IRC during this time and this status post will also be updated.
The work is expected to start from around 2am and will last approximately 4 hours.
We do apologise for the short notice of this work. The date it was dependent on some other factors outside of our control so it was difficult to give a longer notice period.
Just a note to say that we expect the most disruptive period for customers will be between 2am and 2:30am. We will post further updates here though.
Work is underway... Next update by 2:30am.
Work is underway... Next update by 2:30am.
All cables have been moved, we are reviewing the status of the network.
The network is not yet stable, next update at 3am.
We're still working on this, please to bare with us.
Lines are starting to log back in and we are reviewing the network. Next update at around 4:30am
DSL lines are logging back in and we are tracking down some routing problems at the moment. Next update at 5am
Routing is looking better, we are still going through reviewing the network though.
**Customers who are still off line, eg no PPP, should try rebooting their modem/router so as to force a reconnect.**
There are a few remaining routes that we are working on fixing. Next update at 5:45am
Most things are now back to normal.
Our phone lines will be open from 8am as usual so do phone in if your line has still not come back up.
The disruptive work took a lot longer than expected, and we will be reviewing this, but I can confirm the switch change has been completed and we have our 80Gb/s link across London now. This does not yet fix the capacity issues, but allows them to be fixed. We need to review exactly where we are with the switches and the fibre link over the next few days before we can move to the next step. Thankfully this should not involve lots of staff in the datacentre in the middle of the night next time. Well done everyone. Thanks to all of our customers for your patience.