Major work in London affecting most customers - Third visit
MAINTENANCE Closed London Data Centres
STATUS
Closed
CREATED
Mar 18, 04:52 PM (8 years ago)
AFFECTED
London Data Centres
STARTED
Mar 24, 02:00 AM (8 years ago)
CLOSED
Mar 24, 06:00 AM (8 years ago)
REFERENCE
2231 / AA2231
INFORMATION
  • INITIAL
    8 years ago by Andrew

    This is a follow-up to https://aastatus.net/2222 and https://aastatus.net/2229

    As we were not able to complete the planned work that was scheduled for Thursday 17th March we are planning a third visit on Thursday 24th March at 2am.

    We are planning to replace our core network switches in Telehouse early Thursday morning. This is part of our network upgrades to increase our capacity. The work is being carried out in the early hours of Thursday to minimise impact on customers.

    We do expect Internet connectivity to be affected temporarily as we move cables around. We aim to minimise disruption but there will be times when broadband customers will lose PPP and be offline for a few minutes. Other customers and wholesalers who we provide connectivity to will notice that BGP sessions will be taken down and brought back up one by one. Staff will be on IRC during this time and this status post will also be updated.

    The work is expected to start from around 2am and will last approximately 4 hours.

    We do apologise for the short notice of this work. The date it was dependent on some other factors outside of our control so it was difficult to give a longer notice period.

  • UPDATE
    8 years ago by Andrew

    Just a note to say that we expect the most disruptive period for customers will be between 2am and 2:30am. We will post further updates here though.

  • UPDATE
    8 years ago by Andrew

    Work is underway... Next update by 2:30am.

  • UPDATE
    8 years ago by Andrew

    Work is underway... Next update by 2:30am.

  • UPDATE
    8 years ago by Andrew

    All cables have been moved, we are reviewing the status of the network.

  • UPDATE
    8 years ago by Andrew

    The network is not yet stable, next update at 3am.

  • UPDATE
    8 years ago by Andrew

    We're still working on this, please to bare with us.

  • UPDATE
    8 years ago by Andrew

    Lines are starting to log back in and we are reviewing the network. Next update at around 4:30am

  • UPDATE
    8 years ago by Andrew

    DSL lines are logging back in and we are tracking down some routing problems at the moment. Next update at 5am

  • UPDATE
    8 years ago by Andrew

    Routing is looking better, we are still going through reviewing the network though.

  • UPDATE
    8 years ago by Andrew

    **Customers who are still off line, eg no PPP, should try rebooting their modem/router so as to force a reconnect.**

  • UPDATE
    8 years ago by Andrew

    There are a few remaining routes that we are working on fixing. Next update at 5:45am

  • UPDATE
    8 years ago by Andrew

    Most things are now back to normal.

  • UPDATE
    8 years ago by Shaun

    Our phone lines will be open from 8am as usual so do phone in if your line has still not come back up.

  • RESOLUTION
    8 years ago by Andrew

    The disruptive work took a lot longer than expected, and we will be reviewing this, but I can confirm the switch change has been completed and we have our 80Gb/s link across London now. This does not yet fix the capacity issues, but allows them to be fixed. We need to review exactly where we are with the switches and the fibre link over the next few days before we can move to the next step. Thankfully this should not involve lots of staff in the datacentre in the middle of the night next time. Well done everyone. Thanks to all of our customers for your patience.

  • Closed