Recent posts
Timeline view of events on our network and systems

Events from the AAISP network from the last few months on a scrollable timeline. Mouseover for brief details, click incident to view the full post.

MAJOR Closed Email - POP3 and IMAP
AFFECTED
Email - POP3 and IMAP
STARTED
Dec 12, 10:15 AM (4½ days ago)
CLOSED
Dec 12, 10:30 AM (4½ days ago)
DESCRIPTION
We're investigating a problem on one of our email back end file servers - this will be affecting customers accessing their email by POP3 and IMAP at the moment
Resolution: The RAID card on one of the file servers locked up and needed a hard reboot. No mail was lost and service has been restored. From the new year we will be migrating customers off this file server and on to our new email back ends. More information about this will be posted in the new year - but will not need customers to make any changes at their side.

MINOR Open Broadband
AFFECTING
Broadband
STARTED
Dec 11, 09:00 PM (5 days ago)
DESCRIPTION
We're seeing low levels (1-5%) of packet loss and low throughput (eg 16-60 variable on a 70M VDSL) in the evenings on some TalkTalk connected lines in the SO postcode area. We've passed the affected lines on to TalkTalk for them to investigate further.

MAINTENANCE Completed Server Maintenance
AFFECTING
Server Maintenance
STARTED
Dec 11, 07:45 PM (5 days ago)
CLOSED
Dec 11, 10:15 PM (5 days ago)
DESCRIPTION
We are needing to upgrade a reboot a few of our servers - most of these will not affect customers but there may be short periods of time where the Support website (support.aa.net.uk) becomes unavailable and when other services are considered 'at risk' at times in the evening.
Resolution: This work has been completed.

NEWS Info Opening Times
AFFECTING
Opening Times
STARTED
Dec 10, 09:00 AM (6½ days ago)
DESCRIPTION

Here are our opening times over the Christmas and new year holidays:

2018-12-24 Closing in the afternoon
2018-12-25 Closed
2018-12-26 Closed
2018-12-27 Open 09:00 - 17:00
2018-12-28 Open 09:00 - 17:00
2018-12-29 Closed
2018-12-31 Open 09:00 - 17:00
2019-01-01 Closed
2019-01-02 Back to normal hours

When we're open, our staff will be available in all the usual ways (Telephone, email, SMS, IRC etc).


MINOR Closed L2TP customers
AFFECTED
L2TP customers
STARTED
Dec 06, 06:00 PM (10¼ days ago)
CLOSED
Dec 06, 07:31 PM (10¼ days ago)
DESCRIPTION
We're investigating why a small number of L2TP customers have having troubles reconnecting since a software upgrade was performed.
Resolution: The issue has been resolved - sorry for any inconvenience caused.

MINOR Closed SIP2SIM Data problems
AFFECTED
SIP2SIM Data problems
STARTED
Dec 06, 08:00 AM (10½ days ago)
CLOSED
Dec 07, 03:30 AM (9¾ days ago)
DESCRIPTION
Due to problems in the O2 mobile network today, our SIP2SIM customers may be having problems with data and calls.
Resolution: From our carrier (at 3AM) O2 have confirmed that their 4G network has been restored. Both O2 and Manx Telecom will continue to monitor.

MAJOR Closed Broadband (BT)
AFFECTED
Broadband (BT)
STARTED
Dec 02, 07:20 PM (14¼ days ago)
CLOSED
Dec 02, 08:30 PM (14 days ago)
DESCRIPTION
We're aware that some broadband lines have been dropping this evening, we're investigating.
Resolution: This fault was caused by a 'Transmission Incident' at Stepney exchange - BT had to reset equipment and this affected other ISPs who interconnect with BT via Stepney. Due to an error at BTs side, their usual incident notification messages were not sent to us; if they had been sent then they would have appeared on this status page and our staff would have been more aware at the time. We do apologise for the inconvenience that this caused our customers. The actual cause of the fault has not been sent to us yet, but we'll update this post when we do.

MINOR Closed TalkTalk DSL Circuits
AFFECTED
TalkTalk DSL Circuits
STARTED
Nov 30, 12:00 PM (16½ days ago)
CLOSED
Dec 11, 10:32 AM (5½ days ago)
DESCRIPTION

Over the past couple of months we have seen a very small number of TalkTalk lines not log back in to us after maintenance or outages within the TalkTalk network. In these cases we need to involve TalkTalk staff to manually clear the stuck session.

The cause of this problem has been down to some of TalkTalk's routers being unable to cope with the high number of reconnects after an outage has been fixed or planned work has been completed.

To solve this problem, TalkTalk are upgrading the software in these routers. They expect to have updated all their routers within a couple of weeks.


Resolution: TalkTalk have very nearly finished the upgrades that prevent these stale sessions from happening, we don't expect this to be a problem anymore.

MAINTENANCE Open General
AFFECTING
General
STARTED
Nov 22, 03:00 AM (24¾ days ago)
DESCRIPTION
We are upgrading some routers over night again, and we then plan to do a rolling LNS upgrade. More information about LNS switchovers: https://support.aa.net.uk/LNS_Switches

MINOR Closed DNS, Email and Web Hosting
AFFECTED
DNS, Email and Web Hosting
STARTED
Nov 20, 07:00 PM (26¼ days ago)
CLOSED
Nov 20, 10:26 PM (26 days ago)
DESCRIPTION
We're investigating a problem with customers accessing their email by POP3, IMAP or webmail. This is looking like a problem with the authentication process.
Resolution: Access to email was restored at 8:30pm, and queued mail has now been delivered. We apologies for the inconvenience this caused those affected. The cause was database access problems from one of our backend mail servers. A workaround was applied, and we'll be reviewing the failure in order prevent this from happening again.

MAINTENANCE Completed Ethernet Services (via BT)
AFFECTING
Ethernet Services (via BT)
STARTED
Nov 20, 12:01 AM (27 days ago)
CLOSED
Nov 20, 03:24 AM (26¾ days ago)
DESCRIPTION
BT have advised us of planned work that will affect customers with Ethernet circuits as follows:

Summary:Upgrade IOMv1 to IOMv3c AND SWAP 10 PORT CARD

Start: 20/11/18 00:01

End:20/11/18 06:00

Maximum expected Downtime:0 Days 3 Hours 0Minutes

We're upgrading some network hardware in the LONDON POPLAR WEST INDIA DOCK region to provide more capacity, and faster throughput. We expect an outage of 50-60 minutes to end-users if all goes well, but this could extend to 2hrs if we have a problem. We plan to be finished by 04:00 , however if we encounter additional problems, service outage could extend to 06:00


Resolution: BT have confirmed that this work was completed by 03:24