INITIAL

0 seconds ago by Andrew

The following is from TalkTalk and relates to this incident post: https://aastatus.net/2401 and also similar work on 3rd August https://aastatus.net/2415

"The wholesale LTS platform has been suffering from packet loss and congestion over the last few months. Juniper have now advised that this is due to a limitation with the hardware we have and we need to carry out some essential vendor recommended maintenance on our DSL Interconnects platform/switch at Telehouse North in order to fix customer slow throughput.

This RFC is to rebuild ldn-vc1.thn from a mixed EX4200/EX4550 estate to a entirely EX4550 virtual switch.

We have a limitation with our EX4550's - running in a mostly EX4200 VC (mixed mode), whereby the VC cables/modules will not support any more bandwidth past 16gbps (each way). This limitation is because we are running a mixed mode VC. We therefore need to upgrade this VC to run all EX4550s which will mean the VC cables/modules will support 32gbps and the hope is that low speed issues will be resolved."

During this work AAISP will move traffic over to our Harbour Exchange interconnect so as to minimise the impact on our customers, however there still may be drops between midnight and 4am.

UPDATE

6½ years ago by Andrew

TalkTalk traffic has been moved over to Harbour Exchange, away from where this overnight work is happening.

UPDATE

6½ years ago by Andrew

A number of TT lines dropped their PPP session at 00:38, byt 01:30 most had reconnected. The planned work window is still open and there may be a further drop.

UPDATE

6½ years ago by Andrew

It looks like TalkTalk completed their work at around 06:30. We've not yet received an update from TalkTalk as to how it went from their side, but we'll update this post when we do.

There may be a very small number of customers who are still off line, these may need a router reboot so as to force it to log back in.

UPDATE

6½ years ago by Andrew

TalkTalk called us shortly after 9am to confirm that they had completed the work in Telehouse successfully. We will move traffic over to Telehouse later today and will be reporting back the outcome in status post: https://aastatus.net/2401

UPDATE

6½ years ago by Andrew