Last 10 posts, sorted by last updated
Order posts by limited to posts

MAINTENANCE Planned Broadband
AFFECTING
Broadband
STARTING
Feb 17, 01:00 AM (11½ days )
DESCRIPTION

Most of our LNSs have extra diagnostic and debugging hardware installed which has been used to help us track down the problems we were having with them in early 2024. We are now in a position where we can remove this. We have planned to do this work over two nights: the early hours of February 17th and 18th.

This work will involve us moving customers off the LNSs as we do the work. In practice this means customer connections will experience a drop and reconnect at around 1AM and 5AM on the 17th and 18th February.


MINOR Open Email
AFFECTING
Email
STARTED
Feb 03, 09:00 AM (1¾ days ago)
DESCRIPTION

Over the past week we have seen a huge number of 'bots' trying to guess customer email credentials in order to try to send email through our outbound email servers: smtp.aa.net.uk.

The attempts were being blocked due to wrong passwords being used, but this caused significant load on our severs due to all the database lookups involved. To address this, we are blocking IP addresses that are listed on the Spamhaus 'Exploits Blocklist (XBL)' and the Spamhaus 'Combined Spam Sources (CSS)' lists. -These are typically IP address known to have hijacked in some way or known spam senders.

This has reduced the load on the email servers significantly, however we are are still blocking around 1.5 million unique IP addresses each day.

We have had a small number of legitimate customers affected by this as their IP address is on these blocklists. (IPs can be looked up on https://check.spamhaus.org). In these cases, please do contact support and we can discuss workarounds.


MINOR Open Éowyn
AFFECTING
Éowyn
STARTED
Jan 23, 07:00 PM (12½ days ago)
DESCRIPTION
Openreach have notified us of the precautions they are taking in light of storm Éowyn. This involves them not sending Engineers on Friday to areas with 'red' warnings. This may may missed appointments for install or repair activities whilst these warnings are in place.

MAINTENANCE Assumed Completed Email
AFFECTING
Email
STARTED
Jan 14, 09:00 AM (21¾ days ago)
DESCRIPTION
We will be doing work on our email servers which should not affect customers. The work involves rebuilding one of the two back end file stores and will be taking place across multiple days.

MAINTENANCE Assumed Completed authoritative DNS
AFFECTING
authoritative DNS
STARTED
Jun 03, 10:35 AM (8 months ago)
DESCRIPTION

This is only relevant to customers who run their own authoritative DNS servers and use our secondary-dns.co.uk as an additional nameserver.

Overview: We run a "secondary" DNS service for customers where they run the master DNS server and we are secondary slaves. We have a project underway that involves migrating all our authoritative DNS services to a new platform. As part of this we are needing to disable some of the automation we do for adding and updating the customer's master IP address automatically.

The change: From June 17th, If you run your own master DNS server for your domain(s) and secondary-dns.co.uk is a slave, if you change the IP address of your master you will need to contact support@aa.net.uk to request us to update our side.

We have more information about our Authoritative DNS project on our Support Site: https://support.aa.net.uk/New_Authoritive_DNS


MAINTENANCE Completed BT
AFFECTING
BT
STARTING
Feb 12, 01:00 AM (6½ days )
CLOSED
Jan 29, 11:35 AM (6¾ days ago)
DESCRIPTION

BT are performing "Simultaneous Upgrades" which will affect all BT ADSL, VDSL and FTTP services during the early hours of February 12th. It is disappointing that this work is a multi-site activity as we have multiple links to BT in multiple datacentres so as to prevent outages when BT have work to do.

They say:

Start: 2025-02-12 00:01. We're simultaneously upgrading some 21C network software across multiple sites. The PW window is from 00:01 until 07:00, and we will start at 00:01 with non-disruptive pre-checks. From 01:00 we will start to re-boot the devices to the new version of code. This will cause an outage to all customers of up to 15 minutes whilst the device restarts. However, if roll-back is required, the outage could exceed 1hr. All work will be complete by 07:00.

In practice, we expect customers with BT provided connections to drop and reconnect a couple of times in the early hours of February 12th.


Resolution: CANCELLED! This notification fas sent to us by BT in error - their work is not expected to affect our services at all.

MINOR Closed Routing
AFFECTED
Routing
STARTED
Jan 27, 08:28 PM (8½ days ago)
CLOSED
Jan 27, 09:38 PM (8¼ days ago)
DESCRIPTION
We're investigating a large number of drops and reconnects of our L2TP service (l2tp.aa.net.uk)
Resolution:

MINOR Closed Database
AFFECTED
Database
STARTED
Jan 24, 07:02 AM (12 days ago)
CLOSED
Jan 24, 07:22 AM (12 days ago)
DESCRIPTION
Some VoIP calls and the control pages were affected by a Database issue this morning between 7AM and 7:20AM.
Resolution: Our database problem seems to be caused by a recent Linux kernel update. We're tracking the upstream bug and have reverted to a previous kernel.

MINOR Closed SIP2SIM
AFFECTED
SIP2SIM
STARTED
Jan 23, 02:00 PM (12¾ days ago)
CLOSED
Jan 23, 03:30 PM (12½ days ago)
DESCRIPTION
We are investigating a problem with our SIP2SIM servers registering to customer's own PBXs. This isn't affecting SIP2SIM customers where we are also providing the VoIP service/number.
Resolution:

MINOR Closed VoIP
AFFECTED
VoIP
STARTED
Jan 23, 10:03 AM (12¾ days ago)
CLOSED
Jan 23, 10:14 AM (12¾ days ago)
DESCRIPTION
We're investigating a problem with VoIP registrations having problems.
Resolution: Early indications looks to point the cause at our REDIS cluster having failing over when the master had a problem. Further investigations will continue.