1 Aug 06:30:00
[Broadband] TalkTalk DSL core network changes will cause drops from midnight to 4AM 1st August - Completed
Details
Posted: 22 Jul 20:21:08

The following is from TalkTalk and relates to this incident post: https://aastatus.net/2401 and also similar work on 3rd August https://aastatus.net/2415

"The wholesale LTS platform has been suffering from packet loss and congestion over the last few months. Juniper have now advised that this is due to a limitation with the hardware we have and we need to carry out some essential vendor recommended maintenance on our DSL Interconnects platform/switch at Telehouse North in order to fix customer slow throughput.

This RFC is to rebuild ldn-vc1.thn from a mixed EX4200/EX4550 estate to a entirely EX4550 virtual switch.

We have a limitation with our EX4550's - running in a mostly EX4200 VC (mixed mode), whereby the VC cables/modules will not support any more bandwidth past 16gbps (each way). This limitation is because we are running a mixed mode VC. We therefore need to upgrade this VC to run all EX4550s which will mean the VC cables/modules will support 32gbps and the hope is that low speed issues will be resolved."

During this work AAISP will move traffic over to our Harbour Exchange interconnect so as to minimise the impact on our customers, however there still may be drops between midnight and 4am.

Update
31 Jul 16:25:24
TalkTalk traffic has been moved over to Harbour Exchange, away from where this overnight work is happening.
Update
1 Aug 03:25:09
A number of TT lines dropped their PPP session at 00:38, byt 01:30 most had reconnected. The planned work window is still open and there may be a further drop.
Update
1 Aug 07:17:14

It looks like TalkTalk completed their work at around 06:30. We've not yet received an update from TalkTalk as to how it went from their side, but we'll update this post when we do.

There may be a very small number of customers who are still off line, these may need a router reboot so as to force it to log back in.

Update
1 Aug 10:42:11
TalkTalk called us shortly after 9am to confirm that they had completed the work in Telehouse successfully. We will move traffic over to Telehouse later today and will be reporting back the outcome in status post: https://aastatus.net/2401
Update
1 Aug 10:45:19
Closed 1 Aug 06:30:00
Previously expected 1 Aug 04:00:00