Recent posts
Timeline view of events on our network and systems

Events from the AAISP network from the last few months on a scrollable timeline. Mouseover for brief details, click incident to view the full post.

MAINTENANCE Assumed Completed L2TP
AFFECTING
L2TP
STARTING
Nov 24, 03:15 AM (1¾ days )
DESCRIPTION
On Monday 24th November, at 3:15AM we will be upgrading the software on one of our L2TP servers (l2tp.aa.net.uk) This will cause some L2TP customers to drop and reconnect twice as this happens. (Twice, as whilst the router is upgraded connections will reconnect to the other L2TP server, and then will moved back once the upgrade has completed.)

MAINTENANCE Completed BT
AFFECTING
BT
STARTED
Nov 19, 09:00 PM (2¼ days ago)
CLOSED
Nov 20, 07:05 AM (1¾ days ago)
DESCRIPTION
BT have planned work on one of the fibre pairs on one of our hostlinks. We will take this link out of service to avoid disruption to customers.
Resolution: The disruption between 02:29 and 02:37 was caused by BT's Planned work. However, a communications error on our side meant we shut down the wrong link to BT. We do apologise for this! We do not expect any further drops though.

MINOR Closed BT
AFFECTED
BT
STARTED
Nov 18, 11:00 PM (3¼ days ago)
CLOSED
Nov 18, 11:50 PM (3 days ago)
DESCRIPTION
There were several 'blips' overnight that affected some BT customers between 11PM and midnight. Service has been stable since, we are investigating the cause
Resolution: This was caused by BT planned work, which went wrong. They are planning work on the other pair of fibres on this hostink during the early hours of Thursday 20th November, but we will take this out of service beforehand.

MINOR Open ZyXEL
AFFECTING
ZyXEL
STARTED
Nov 18, 04:40 PM (3½ days ago)
DESCRIPTION

ZyXEL have published a security advisory which covers some of the ZyXEL routers we have provided our customers, the DX3301 and VMG3927.

New firmware has been released. Customers can either load this manually, or use the 'Upgrade Firmware' button found on the router information page on our Control Pages.

We have received further clarification that the vulnerabilities affect the router's web interface when exposed to the internet. Routers we configure for customers will have the web interface only allowable from our office IP range, and so this vulnerability is deemed low risk. We still recommend that customers upgrade. The upgrade process will take a few minutes but it does not reset the configuration.

We will make direct contact with customers with these routers, and support staff will mention this if and when they are talking to customers and notice the firmware is out of date.


MINOR Closed BT
AFFECTED
BT
STARTED
Nov 16, 10:20 AM (5¾ days ago)
CLOSED
Nov 16, 03:45 PM (5½ days ago)
DESCRIPTION
At 10:20 a number of BT lines dropped, we're investigating.
Resolution: At 10:20 about 30% of customers dropped there connection and reconnected within a few minutes. Service was back to normal by 10:28. One of our 4 links to BT had loss of light. BT replaced a faulty LIM (Line Interface Module) card at Greenwich later in the afternoon.

MINOR Closed BT
AFFECTED
BT
STARTED
Nov 11, 07:55 PM (10¼ days ago)
CLOSED
Nov 12, 07:15 AM (9¾ days ago)
DESCRIPTION
at around 19:56 a fairly large number of BT circuits dropped. We're investigating.
Resolution: Around a quarter of our customers on BT had a short service interruption caused by a faulty card within BTs network that caused loss of light on one of our interconnects to BT. These customers reconnected within a couple of minutes. The fault was caused by a faulty card at BTs end. The card was swapped overnight, and we bought the interconnect back in to service at 7:15AM.

MAINTENANCE Assumed Completed IRC
AFFECTING
IRC
STARTED
Oct 26, 10:00 AM (26¾ days ago)
CLOSED
Nov 26, 02:00 PM (4¼ days ago)
DESCRIPTION
[minor] We run a pool of three IRC servers, and usually have two live at any one time. One of these IRC servers is being rebooted on Sunday midday as part maintenance at the hosting provider - it's not in DNS at the moment but IRC clients will still be connected to it and may well drop and reconnect on Sunday.
Resolution:

MINOR Closed AWS
AFFECTED
AWS
STARTED
Oct 23, 09:45 AM (29¾ days ago)
CLOSED
Nov 20, 10:13 AM (1¾ days ago)
DESCRIPTION
We've had reports of customers having problems routing to some IPs within Amazon
Resolution: Separate to the routing problem, both of our Amazon interconnects are being migrated to different equipment at Amazon's end. They are out of action for the moment. We still have peering to Amazon over LINX.

MAJOR Closed CityFibre
AFFECTED
CityFibre
STARTED
Oct 22, 09:55 PM (30¼ days ago)
CLOSED
Oct 22, 10:22 PM (30¼ days ago)
DESCRIPTION
We are investigating a problem with our CityFibre services.
Resolution: We do apologise for this outage. We are looking into making our interconnect to CityFibre more resilient.

MAINTENANCE Completed Network
AFFECTING
Network
STARTED
Oct 09, 01:00 AM (1¼ months ago)
CLOSED
Oct 22, 09:02 AM (30¾ days ago)
DESCRIPTION

We'll be performing routine overnight software upgrades on our pool of LNSs and BGP routers, starting Thursday 9th October. One LNS will be done per night, and this involves moving lines between the LNSs and upgrading when lines have been moved off. Customers may see a PPP drop and reconnect in the early hours of the morning as this work is carried out.

Upgrades to our BGP routers will also happen overnight but should have little to no impact. Resolution: This work has been completed.