Last 10 major outages posts, sorted by last updated
Order posts by limited to posts

MAJOR Closed General
AFFECTED
General
STARTED
May 04, 03:10 PM (27¼ days ago)
CLOSED
May 04, 04:00 PM (27¼ days ago)
DESCRIPTION
We're investigating a connectivity problem.
Resolution: Service was restored at around 4PM. The disruption was caused by a number of our our BGP routers crashing due to unusual network traffic. This is similar to what we saw a couple of weeks ago when one of our LNS routers crashed. We have since upgraded the software on all our LNS routers and that has helped - our LNSs did not crash during this incident. We are planning to upgrade the software on our BGP routers this week. It is fair to say that with staff working from home that the time taken to recover from this incident was impaired and we'll be looking at ways to improve this too. We would like to apologise to our customers for this outage.

Closed: 27¼ days ago
View full details

MAJOR Closed General
AFFECTED
General
STARTED
Feb 27, 08:00 AM (3 months ago)
CLOSED
Feb 27, 09:15 AM (3 months ago)
DESCRIPTION
We're investigating networking problems with our Maidenhead datacentre. This will be affecting hosted services and access to some of our services such as VoIP and outbound email.
Resolution: Sorry for the delay providing more information. Due to the nature of the incident we prefer not to go in to too much detail publicly. Staff are willing to talk to customers who may be concerned about this, so please do get in touch. The incident lasted just over an hour and did cause various amounts of disruption. There are lessons we can learn from this incident and will be working on further improvements.

Closed: 3 months ago
View full details

MAJOR Closed Broadband
AFFECTED
Broadband
STARTED
Jan 22, 11:10 PM (4¼ months ago)
CLOSED
Jan 23, 09:05 AM (4¼ months ago)
DESCRIPTION
We're investigating a problem with some broadband lines showing 50% packetloss from just after 11PM.
Resolution: The problem was resolved just after 9AM. the reason for this outage is in https://aastatus.net/33308 "Networks have investigated and advised they found a number of errors on a card in LOH which was erroring and causing back plain errors."

Closed: 4¼ months ago
View full details

MAJOR Closed General
AFFECTED
General
STARTED
Dec 18, 09:40 AM (5¼ months ago)
CLOSED
Dec 18, 12:04 PM (5¼ months ago)
DESCRIPTION
We're investigating a faulty switch in Telehouse. This will have some impact on services, we're reviewing this at the moment.
Resolution: The switch is back in service. We're still investigating the initial cause of the problem.

Closed: 5¼ months ago
View full details

MAJOR Closed Broadband
AFFECTED
Broadband
STARTED
Nov 19, 12:01 AM (6¼ months ago)
CLOSED
Nov 19, 01:00 AM (6¼ months ago)
DESCRIPTION
We are seeing line drops on some TalkTalk lines since midnight. We're investigating...
Resolution: This was a fault within TalkTalk. The 'routing engine' in one of their routers failed.

Closed: 6¼ months ago
View full details

MAJOR Closed Broadband
AFFECTED
Broadband
STARTED
Oct 09, 02:11 AM (7¾ months ago)
CLOSED
Oct 09, 09:06 AM (7¾ months ago)
DESCRIPTION

On 09/10/19 at 02:11 our monitoring picked up that a few thousand broadband lines dropped.

After some investigation it appears the drops occurred due to all the B-sides of our BTW hostlinks flapping causing all sessions over these links to re-establish

Lines did immediately start to reconnect over our other hostlinks; however due to the number of attempted re-connections in some cases it took ~30minutes for lines reconnect

We are not aware of any BTW planned works scheduled for this time and are currently chasing BT for an explanation for this blip

Resolution:

Closed: 7¾ months ago
View full details

MAJOR Closed Broadband
AFFECTED
Broadband
STARTED
Aug 09, 04:53 PM (9¾ months ago)
CLOSED
Aug 09, 05:46 PM (9¾ months ago)
DESCRIPTION
We have identified a blip in our network at 16:53 with multiple customers going offline and not recovering within a reasonable period of time. We are currently investigating and will update this post with any news.
Resolution: We believe that all affected circuits are currently back online. Our carriers suggest that this outage was due to power issues across the UK. If you are still experiencing issues, then we would suggest rebooting your router, or contacting Support at 03333 400 999 / support@aa.net.uk.

Closed: 9¾ months ago
View full details

MAJOR Closed Email
AFFECTED
Email
STARTED
Aug 01, 05:40 PM (10 months ago)
CLOSED
Aug 01, 06:45 PM (10 months ago)
DESCRIPTION
We're investigating authentication problems whilst accessing email.
Resolution:

Closed: 10 months ago
View full details

MAJOR Closed Maidenhead connectivity
AFFECTED
Maidenhead connectivity
STARTED
Jun 24, 09:54 PM (11¼ months ago)
CLOSED
Jun 26, 12:49 AM (11 months ago)
DESCRIPTION
We're investigating problems with services hosted at our Maidenhead Datacentre - this is affecting VoIP, sending email, Control Pages and some other services as well as customer servers we host.
Resolution: Power was lost at the datacentre as as result of something going wrong with low risk UPS/Generator maintenance by the datacentre.

Update 1 July: The datacentre say that whilst the UPS was disconnected for planned maintenance they incurred a coincidental loss of utility mains power, and thus power to the site was lost.


Closed: 11 months ago
View full details

MAJOR Closed Broadband
AFFECTED
Broadband
STARTED
Jun 27, 01:35 AM (11 months ago)
CLOSED
Jun 27, 02:16 AM (11 months ago)
DESCRIPTION
Looks like all lines in TT backhaul dropped. Lines are reconnecting. We’ll update later when we have more details.
Resolution: Line reconnections seem to have stopped and traffic levels back to normal.

Closed: 11 months ago
View full details