Recent posts
Timeline view of events on our network and systems

Events from the AAISP network from the last few months on a scrollable timeline. Mouseover for brief details, click incident to view the full post.

...start date, newest first ...last updated first
    Maximise all events
NEWS Info KCI
AFFECTING
KCI
STARTED
Feb 06, 04:54 PM (2½ hours ago)
DESCRIPTION
We are retiring our use of the Twitter API. This means that tweets and DMs we send from our systems will no longer be sent - this includes Status Page updates and broadband up/down notifications. (Though the up/down notification DMs haven't been working for some time now.) We are looking into alternatives, such as Mastodon, and will announce that in due course.
Last updated: 2¼ hours ago
View full details

MINOR Closed L2TP
AFFECTED
L2TP
STARTED
Feb 04, 11:35 PM (1¾ days ago)
CLOSED
Feb 05, 06:37 AM (1½ days ago)
DESCRIPTION
Over night we have seen restarts on the new LNS used for L2TP access causing people to have to reconnect. Customers have been routed to the older LNS for now, and the situation is stable.
Resolution: Engineers will be working on the issue on the new LNS, and we expect to move back in the week at some point. Diagnostics have given a good clue as to the cause of the problem.

Closed: 1½ days ago
View full details

MINOR Closed L2TP
AFFECTED
L2TP
STARTED
Feb 04, 10:30 PM (1¾ days ago)
CLOSED
Feb 04, 10:49 PM (1¾ days ago)
DESCRIPTION
We're investigating a problem with L2TP connections.
Resolution: Further drops happens through the night and the backup router is now in use. More details: https://aastatus.net/42511

Closed: 1¾ days ago
View full details

MAINTENANCE Completed LNS Upgrades
AFFECTING
LNS Upgrades
STARTED
Feb 02, 04:00 AM (4½ days ago)
CLOSED
Feb 04, 09:00 AM (2¼ days ago)
DESCRIPTION
We are performing upgrades on the 'Witless' set of LNSs this week. This will mean a PPP reconnect in the early ours of the morning for customers with CityFibre, BT FTTP, G.Fast and some 80M FTTC.
Resolution: This series of upgrades has been completed.

Closed: 2¼ days ago
View full details

MINOR Closed L2TP
AFFECTED
L2TP
STARTED
Jan 31, 02:25 PM (6 days ago)
CLOSED
Jan 31, 03:05 PM (6 days ago)
DESCRIPTION
We're seeing a small number of L2TP customers drop and reconnect - this has been happening since 14:25 and seems to be those connection that are using Virgin as the underlying ISP. It seems as though there is packet loss affecting Virgin's network.
Resolution: Connections have stopped disconnecting. We had other reports of packetloss on Virgin connections unrelated to our services, so it seems there was some sort of problem at Virgin's side.

Closed: 6 days ago
View full details

MINOR Closed BT DSL
AFFECTED
BT DSL
STARTED
Jan 30, 12:15 PM (7¼ days ago)
CLOSED
Jan 30, 12:20 PM (7¼ days ago)
DESCRIPTION
A small number of BT lines dropped and reconnected at 12:15. This does seem specific to BT and we're asked them to investigate. The lines did reconnect a minute or so after.
Resolution: BT say this was due to a ' transmission incident' at FARADAY exchange, saying: "Loss of service to 47 x 100GigE, 4 x 40 GigE, 138 x 10GigE, 1 x 1 GigE and 1 x STM-64 circuits. {Total 6291 GigE}. Suspected fibre disturbance however, unable to confirm."

Closed: 7¼ days ago
View full details

MAINTENANCE Completed L2TP
AFFECTING
L2TP
STARTED
Jan 27, 03:00 AM (10½ days ago)
CLOSED
Jan 27, 03:52 AM (10½ days ago)
DESCRIPTION
We will be performing some overnight work on our L2TP (l2tp.aa.net.uk) service. The work involves moving the main IP address that customers connect to from a FB6000 router to a FB9000 router. This work will cause connections to drop for a couple of minutes between 3AM and 4AM.
Resolution: This has been completed. L2TP circuits are now being terminated on a different router. Any problems please contact Support.

Closed: 10½ days ago
View full details

MAJOR Closed L2TP and Routing
AFFECTED
L2TP and Routing
STARTED
Jan 26, 08:57 AM (11¼ days ago)
CLOSED
Jan 26, 09:40 AM (11¼ days ago)
DESCRIPTION
We say some L2TP sessions drop at around 08:57 - we are investigating this and a possible routing problem.
Resolution:

Closed: 11¼ days ago
View full details

MINOR Closed BT DSL
AFFECTED
BT DSL
STARTED
Jan 18, 04:05 PM (19 days ago)
CLOSED
Jan 18, 05:00 PM (19 days ago)
DESCRIPTION
We saw a number of BT lines drop at around 16:05. Most have reconnected.
Resolution: This was an incident at BTs FARADAY location affecting multiple ISPs and many broadband lines in the south-east of the UK. BT have not yet given further details about the cause.

Closed: 19 days ago
View full details

MINOR Closed BT DSL
AFFECTED
BT DSL
STARTED
Jan 18, 01:30 PM (19 days ago)
CLOSED
Jan 18, 01:35 PM (19 days ago)
DESCRIPTION
As part of routine maintenance to move broadband traffic off a link we have to BT, a small number of lines disconnected and reconnected a few moments later.
Resolution:

Closed: 19 days ago
View full details

MAINTENANCE Completed TalkTalk
AFFECTING
TalkTalk
STARTED
Jan 09, 10:11 AM (28¼ days ago)
CLOSED
Jan 11, 03:00 AM (26½ days ago)
DESCRIPTION
TalkTalk are doing work on their infrastructure in Telehouse which includes a reboot of one of their core routers. We will move traffic away from this link the day before so as to prevent our customers from being affected by this work.
Resolution: This was completed without incident.

Closed: 26½ days ago
View full details

MINOR Closed TalkTalk
AFFECTED
TalkTalk
STARTED
Jan 09, 09:00 AM (28¼ days ago)
CLOSED
Jan 09, 01:00 PM (28¼ days ago)
DESCRIPTION
Since 9AM today we are seeing latency and packet loss on a number of TalkTalk circuits. The common factor seems to be a particular route the traffic takes within TalkTalk's network - this indicates congestion caused by a broken fibre or a faulty device within TalkTalk.
Resolution: From TalkTalk: "Resolution Details: Due to an ongoing Transmission issue we are having within our core network we have lost multiple 100G and 10G links , causing our core network to run at reduced capacity. During this period, we noticed an interlink between our edge routers was congesting causing issues for customers. We have moved traffic away from this interlink to restore service to normal operations. Technical / Suspected Root Cause: Traffic congestion in edge network. Traffic moved away to alleviate congestion."

Closed: 28¼ days ago
View full details

MAINTENANCE Assumed Completed BT and TT
AFFECTING
BT and TT
STARTED
Jan 09, 12:01 AM (28¾ days ago)
DESCRIPTION
Both TalkTalk and BT have overnight work planned for various areas of the country over the next couple of weeks. A small number of customers may see their connection drop for a few minutes between the hours of midnight and 5AM.
Last updated: 25¼ days ago
View full details