Mon 17 Sep 2018, 16:40
Affected
Broadband
Details
Posted: Mon 17 Sep
We just had one of out LNS routers restart which has caused all circuits on it to PPP reconnect. We are investigating the cause, most lines are already reconnected.
Update
Mon 17 Sep 2018, 16:36
Most lines have reconnected, just waiting for the remaining ones.
Update
Mon 17 Sep 2018, 19:18
Circuits on E.Gormless had a similar drop at around 19:00 - we can see the cause of this. IPv6 routing for customers on E.Gormless was also intermittent from 16:15 through to 19:15.
Resolution G.Gormless restarted unexpectedly, we're investigating the cause - we apologise for the disruption this caused.
Started Mon 17 Sep 2018, 16:28
Closed Mon 17 Sep 2018, 16:40

Fri 10 Aug 2018, 06:00
Affecting
VoIP and SIMs
Details
Posted: Wed 08 Aug
Customers using the 'mobiledata' APN on voice SIMs with AAISP will experience up to 2 minutes of downtime between 22:00 and 22:20. This is for work being performed by our carrier. The proposed Maintenance window is between: 09 Aug 2018 21:45 and 10 Aug 2018 06:00
Started Thu 09 Aug 2018, 22:00
Closed Fri 10 Aug 2018, 06:00
Previously expected Fri 10 Aug 2018, 06:00

Tue 14 Aug 2018, 06:00
Affecting
VoIP and SIMs
Details
Posted: Wed 08 Aug
Loss of resiliency for approximately 30 minutes at around 23:00 is expected whilst our carrier performs maintenance. The proposed Maintenance window is between: 13 Aug 2018 22:00 and 14 Aug 2018 06:00
Update
Wed 15 Aug 2018, 10:51
The scheduled maintenance was unsuccessful and will be re-scheduled in due course
Started Mon 13 Aug 2018, 22:00
Closed Tue 14 Aug 2018, 06:00
Previously expected Mon 13 Aug 2018, 22:00

Thu 05 Jul 2018, 16:00
Affected
Broadband
Details
Posted: Wed 04 Jul

We've been investigating a possible problem with TalkTalk VDSL circuits not able to reach the full speed potential when doing downloads. This is more prevalent on circuits which have near 80M sync speed. The throughput of these circuits seem to be unable to download at IP rates faster than 69Mb/s. We are investigating this with TalkTalk, and have also been testing both TalkTalk and BT based circuits with the help of other customers.

We will update this post as things develop.

Update
Wed 04 Jul 2018, 17:57
This Kitz Forum thread is also relevant to this: https://forum.kitz.co.uk/index.php/topic,21750.0.html
Update
Wed 04 Jul 2018, 18:58

Regular readers may remember that we migrated our TalkTalk lines on to a brand new TalkTalk network in April of this year. Many lines saw a drop in latency, which has been well received by customers.

With the help of TalkTalk, this evening we tested a circuit via the old network. Although the latency increased (from 6.4ms to 11.8ms) the throughput as measured by using the iperf3 tool showed an increase from 67.4Mb/s to 71.9M/s. (This is a line with a 75.6M sync rate)

We are still testing things further so as to rule out other factors that could be involved.

We have passed this on to TalkTalk and they are investigating further too.

Update
Thu 05 Jul 2018, 14:43

Some good news. TalkTalk have been doing some tests and changes on a single line as part of their investigations.

This is what we're looking for when looking at our CQM graphs of traffic:

A constant download of lots of http and torrents at 75.2M (sync is a bit below 76M), filling the link (red line at bottom)

iperf alone gives 71.9 when before it was getting 67.4 yesterday.

The change TT have made has only been applied to this one line, we'll update this post when the change has been made across the board.

Update
Thu 05 Jul 2018, 15:34

It looks like this has been fixed, we'd appreciate any feedback

TalkTalk have rolled out a change to the "overhead accounting mode in use for VDSL customers." Customers can now try reconnecting (restart PPP or reboot the router), and try speed tests and downloads.

Started Wed 04 Jul 2018, 09:00
Closed Thu 05 Jul 2018, 16:00

Sat 15 Sep 2018, 23:15
Affected
Broadband
Details
Posted: Sat 15 Sep
We had reports of packet loss this evening from 11pm through to 11:15pm - our monitoring also saw PPP reconnects on a number of lines.
Update
Sat 15 Sep 2018, 23:44
We're still investigating this, it looks like some (not all) TalkTalk circuits were affected, but as yet we cannot see a common factor between the affected circuits. We have asked TalkTalk if they are aware of anything happening - usually planned work would happen in the early hours of the morning rather than late evening. We'll post an update when we have one.
Resolution TalkTalk confirmed that thus was due to 'multiple link failures caused by planned work by our carriers Transmission Network (Zayo). This work was scheduled between 22:00 and 06:00 however it was not expected to have any customer impact.' (however, it did!)
Started Sat 15 Sep 2018, 23:00
Closed Sat 15 Sep 2018, 23:15

Fri 28 Sep 2018, 03:00
Affecting
General
Details
Posted: Fri 14 Sep
Between 03:00 - 05:00 on the 28th September we will be updating software on core infrastructure. Expect service disruption during this time.
Planned start Fri 28 Sep 2018, 03:00 by AA Staff
Update expected Fri 28 Sep 2018, 05:00
Expected close Fri 28 Sep 2018, 05:00 (Estimated Resolution Time from AAISP)

Sat 08 Sep 2018, 20:20
Affected
Broadband
Details
Posted: Sat 08 Sep
Some TalkTalk lines in the Leicestershire and Derbyshire regions are suffering from some sort of congestion (high packet loss and latency) they are aware and are investigating. Exchanges affected are :- Swadlincote, Burton on trent, Beaumont, Goscote, Saint Barnabus, Glenfield, Montford, Leicester Central, Thurnby, Oadby, Measham, Belgrave, Birstall, Tutbury, Stoneygate, Kirby Muxloe, Newark, Overseal and Barton under Needwood
Update
Sat 08 Sep 2018, 16:58
TalkTalk say: "Following investigations by our IP Access team we believe this incident to be fixed. This incident will remain open under monitoring while retest takes place." However we are still seeing packtloss on lines!
Update
Sat 08 Sep 2018, 17:07
We've contacted TT regarding this, as the fault us clearly still ongoing.
Update
Sat 08 Sep 2018, 17:11
Resolution Service looks to be back to normal now - TalkTalk identified the fault in their network at about20:20 this evening.
Started Sat 08 Sep 2018, 16:20
Closed Sat 08 Sep 2018, 20:20
Previously expected Sat 08 Sep 2018, 20:20

Fri 07 Sep 2018, 08:30
Affecting
Broadband and Ethernet
Details
Posted: Wed 22 Aug
We are upgrading routers over night, and then a rolling LNS upgrade over night. More information about LNS updates here: https://support.aa.net.uk/LNS_Switches
Update
Fri 24 Aug 2018, 17:47
Upgrades are going well (slightly later than planned this morning) and we start the rolling LNS upgrades tonight.
Resolution The LNS upgrades have completed.
Started Thu 23 Aug 2018, 00:00
Closed Fri 07 Sep 2018, 08:30
Previously expected Mon 10 Sep 2018, 00:00

Tue 04 Sep 2018, 15:30
Affecting
Broadband
Details
Posted: Wed 08 Mar

We continue to work with TalkTalk to get to the bottom of the slow throughput issue as described on https://aastatus.net/2358

We will be performing some routing changes and tests this afternoon and this evening, we are not expecting this to cause any drops for customers, but this evening there will be times when throughput for 'single thread' downloads will be slow. Sorry for the short notice, please bear with us, this is being a tricky fault to track down.

Update
Wed 08 Mar 2017, 22:39
Sorry, due to TalkTalk needing extra time to prepare for their changes this work has been moved to Thursday 9th evening.
Started Thu 09 Mar 2017, 20:00
Closed Tue 04 Sep 2018, 15:30

Fri 24 Aug 2018, 12:30
Affected
VoIP and SIMs
Details
Posted: Fri 24 Aug
Our upstream SMS provider is having problems, they say "At the moment our systems and networks team are actively working to resolve the issues with our SMS services, sadly we do not have an ETA for resolution" Updates to follow.
Update
Fri 24 Aug 2018, 12:51
The outbound SMS issues have now been restored, and messages which were sent during the issue were queued and are now being delivered.
Started Fri 24 Aug 2018, 11:00
Closed Fri 24 Aug 2018, 12:30