Order posts by limited to posts

18 Aug 14:51:51
Details
18 Aug 15:03:02
We have received advice from Three that they will be applying firmware upgrades to resolve issues identified on their new infrastructure. The maintenance will be carried out of the following days; 23rd August 2016 - 23:10 to 24th August 2016 - 04:30 BST 31st August 2016 5th September 2015 We are still awaiting a full schedule from Three regarding the times for the 31st August and 5th September and will post a further announcement once the times have been confirmed. This affects only our Data SIMs.
VoIP and SIMs Users Affected 15%
Started 18 Aug 14:51:51 by Carrier
Update was expected Thursday 15:51:51
Expected close 5 Sep 04:30:00 (Estimated Resolution Time from Carrier)

5 Aug 15:30:00
Details
5 Aug 15:31:35

We are making some changes to our Ip transit connectivity over the next few days. This should be pretty seamless as just mean traffic taking new routes. However, there is always a risk of issues.

Today we are doing a bit or preparation which means some minor routing changes for about 5% of our traffic, and which we also expect to have no impact.

Started 5 Aug 15:30:00

20 Jul 09:58:43
[Maidenhead Colocation] - Web and email outage - Open
Details
04 Apr 2013 15:47:25

This is ongoing. We're investigating.

Update
04 Apr 2013 16:07:41

This should now be fixed. Please let support know if you see any problems, or have any questions.

Update
05 Apr 2013 09:15:04

This was resolved yesterday afternoon.

Update
20 Jul 09:58:43
This may be related to a wider issue on the internet with a power outage at a major london data centre. Thos routing problems are still ongoing.
Started 04 Apr 2013 15:46:11

5 Feb 16:32:23
Details
5 Feb 16:32:23
Some call recordings from today have been lost. We've identified the problem and have worked around it until we can fix it. We should be able to fix it with no customer impact. If you are expecting a call recording that you have not received, support can check whether the call was affected by this problem. The call recordings lost are in the minority and most calls were recorded as expected. This problem is unrelated to yesterday's problems with lack of ring tone!
Started 5 Feb 16:25:24

12 Jun 2015 11:06:49
Details
12 Jun 2015 11:00:23
Our office connectivity blipped. We have internet again, but our phones are down. We're investigating. This is not customer affecting.
Update
12 Jun 2015 11:06:49
Phones are working again.
Previously expected 12 Jun 2015 14:57:07

13 Apr 2015
Details
09 Apr 2015 13:43:39

We have been advised by Three of works on their network which will involve rerouting traffic from one of their nodes via alternate paths in their core. Although connections should automatically reroute, there will be brief amounts of packet loss. As a result, some customers may experience dropped connections. Any device reconnecting will automatically be routed via a new path.

This only affects our data only SIMs.

Started 13 Apr 2015

12 Mar 2015 09:48:01
Details
12 Mar 2015 09:48:01
Our wiki at http://wiki.aa.org.uk/ will be down for a while today due to an internal PEW. Sorry for any inconvenience.

08 Jan 2015 12:39:06
Details
08 Jan 2015 12:49:24
We're going to remove the legacy fb6000.cgi page that was originally used to display CQM graphs on the control pages. This does not affect people who use the control pages as normal, but we've noticed that fb6000.cgi URLs are still being accessed occasionally. This is probably because the old page is being used to embed graphs into people's intranet sites, for example, but accessing graph URLs via fb6000.cgi has been deprecated for a long time. The supported method for obtaining graphs via automated means is via the "info" command on our API: http://aa.net.uk/support-chaos.html This is likely to affect only a handful of customers but, if you believe you're affected and require help with accessing the API, please contact support. We will remove the old page after a week (on 2015-01-15).
Update
09 Jan 2015 08:52:28
We'll be coming up with some working examples of using our CHAOS API to get graphs, we'll post an update here today or monday.
Update
12 Jan 2015 16:19:58
We have an example here: https://wiki.aa.net.uk/CHAOS
Started 08 Jan 2015 12:39:06 by AA Staff
Previously expected 15 Jan 2015 17:00:00

03 Jun 2014 17:00:00
Details
03 Jun 2014 18:20:39
The router upgrades went well, and now there is a new factory release we'll be doing some rolling upgrades over the next few days. Should be minimal disruption.
Update
03 Jun 2014 18:47:21
First batch of updates done.
Started 03 Jun 2014 17:00:00
Previously expected 07 Jun 2014

14 Apr 2014
Details
13 Apr 2014 17:29:53
We handle SMS, both outgoing from customers, and incoming via various carriers, and we are now linking in once again to SMS with mobile voice SIM cards. The original code for this is getting a tad worn out, so we are working on a new system. It will have ingress gateways for the various ways SMS can arrive at us, core SMS routing, and then output gateways for the ways we can send on SMS. The plan is to convert all SMS to/from standard GSM 03.40 TPDUs. This is a tad technical I know, but it will mean that we have a common format internally. This will not be easy as there are a lot of character set conversion issues, and multiple TPDUs where concatenation of texts is used. The upshot for us is a more consistent and maintainable platform. The benefit for customers is more ways to submit and receive text messages, including using 17094009 to make an ETSI in-band modem text call from suitable equipment (we think gigasets do this). It also means customers will be able to send/receive texts in a raw GSM 03.40 TPDU format, which will be of use to some customers. It also makes it easier for us to add other formats later. There will be some changes to the existing interfaces over time, but we want to keep these to a minimum, obviously.
Update
21 Apr 2014 16:27:23

Work is going well on this, and we hope to switch Mobile Originated texting (i.e. texts from the SIP2SIM) over to the new system this week. If that goes to plan we can move some of the other ingress texting over to the new system one by one.

We'll be updating documentation at the same time.

The new system should be a lot more maintainable. We have a number of open tickets with the mobile carrier and other operators to try and improve the functionality of texting to/from us. These cover things like correct handling of multi-part texts, and correct character set coding.

The plan is ultimately to have full UTF-8 unicode support on all texts, but that could take a while. It seems telcos like to mess with things rather than giving us a clean GSM TPDU for texts. All good fun.

Update
22 Apr 2014 08:51:09
We have updated the web site documentation on this to the new system, but this is not fully in use yet. Hopefully this week we have it all switched over. Right now we have removed some features from documenation (such as delivery reports), but we plan to have these re-instated soon once we have the new system handling them sensibly.
Update
22 Apr 2014 09:50:44
MO texts from SIP2SIM are now using the new system - please let support know of any issues.
Update
22 Apr 2014 12:32:07
Texts from Three are now working to ALL of our 01, 02, and 03 numbers. These are delivered by email, http, or direct to SIP2SIM depending on the configuration on our control pages.
Update
23 Apr 2014 09:23:20
We have switched over one of our incoming SMS gateways to the new system now. So most messages coming from outside will use this. Any issues, please let support know ASAP.
Update
25 Apr 2014 10:29:50
We are currently running all SMS via the new platform - we expect there to be more work still to be done, but it should be operating as per the current documentation now. Please let support know of any issues.
Update
26 Apr 2014 13:27:37
We have switched the DNS to point SMS to the new servers running the new system. Any issues, please let support know.
Started 14 Apr 2014
Previously expected 01 May 2014

11 Apr 2014 15:50:28
Details
11 Apr 2014 15:53:42
There is a problem with the C server and it needs to be restarted again after the maintenance yesterday evening. We are going to do this at 17:00 as we need it to be done as soon as possible. Sorry for the short notice.
Started 11 Apr 2014 15:50:28

07 Apr 2014 13:45:09
Details
07 Apr 2014 13:52:31
We will be carrying out some maintenance on our 'C' SIP server outside office hours. It will cause disruption to calls, but is likely only to last a couple of minutes and will only affect calls on the A and C servers. It will not affect calls on our "voiceless" SIP platform or SIP2SIM. We will do this on Thursday evening at around 22:30. Please contact support if you have any questions.
Update
10 Apr 2014 23:19:59
Completed earlier this evening.
Started 07 Apr 2014 13:45:09
Previously expected 10 Apr 2014 22:45:00

25 Sep 2013
Details
18 Sep 2013 16:32:41
We have received notification that Three's network team will be carrying out maintenance on one of the nodes that routes our data SIM traffic between 00:00 and 06:00 on Weds 25th September. Some customers may notice a momentary drop in connections during this time as any SIMs using that route will disconnect when the link is shut down. Any affected SIMs will automatically take an alternate route when they try and reconnect. Unfortunately, we have no control over the timing of this as it is dependent on the retry strategy of your devices. During the window, the affected node will be offline therefore SIM connectivity should be considered at risk throughout.
Started 25 Sep 2013

Details
06 Jul 2015 12:49:42
We have been advised by Three of works on their network (22:30 8th July to 04:50 9th July 2015) Which will involve rerouting traffic from one of their nodes via alternate paths in their core. Although connections should automatically reroute, there will be brief amounts of packet loss. As a result, some partners may experience dropped connections. Any device reconnecting will automatically be routed via a new path. We apologise for any inconvenience this may cause and the short notice of this advisory.

Details
12 Feb 2015 15:57:26
We have received the below PEW notification from one of our carriers that we take voice SIMS from. We have been advised by one of our layer 2 suppliers of emergency works to upgrade firmware on their routers to ensure ongoing stability. This will cause short drops in routing during the following periods: 00:00 to 06:00 Fri 13th Feb 00:00 to 04:00 Mon 16th Feb Although traffic should automatically reroute within our core as part of our MPLS infrastructure, some partners may experience disruption to SIM connectivity due to the short heartbeats used on SIM sessions.

12 Aug 18:21:20
[Broadband and Ethernet] - World of Warcraft IPv6 issues - Closed
Details
24 Jul 19:42:01

It seems that once again Blizzard have some IPv6 issues that are impacting World of Warcraft.

This should be simple for them to fix. They have connectivity to their network, which seems to be very good (via LINX) with low pings, but the individual servers their end do not seem to be working on the IPv6 addresses the client is trying.

This could simply be a change of IPv6 SLAAC address following hardware changes, or could be something else, we don't know. They are not interested in investigating.

Normally a failure in one IP protocol is not a huge issue as clients fall back to the other version, e.g. trying IPv6 first then legacy IPv4. However the WoW client seems not to be doing this, simply hanging and eventually giving an error.

Obviously this has been reported to Blizzard, but I am sorry to say that they are taking the backwards step of taking no notice of IPv6 issues, effectively withdrawing support for the current version of Internet Protocol, IPv6

As such, customers may have to change their config on their game client not to try IPv6. This is obviously not something we would normally recommend as it removes the usual redundancy of having both protocols. It also gives the mistaken impression that IPv6 is somehow unreliable, rather than the more accurate impression that Blizzard are unreliable.

Hopefully they will fix it eventually. We have yet to see any official statement as to withdrawal of support for IPv6, a move that is very unusual for any company to take.

Update
25 Jul 06:57:26
Blizzard have now said they have engineers working on this. No ETA.
Update
26 Jul 20:19:57
An update from Blizzard suggests they have made changes to address this, but tests suggest it is not yet resolved, so still chasing.
Update
27 Jul 10:38:54
Blizzard have made a number of changes to fix this issue. It is not quite fixed for everyone according to my tests, so they are working on it still.
Update
7 Aug 08:54:09

Blizzard have said: "As mentioned previously, the game client is designed to take advantage of the IPV6 options simply by ticking the IPV6 option ingame and restarting the game. As IPV6 comes available, you will be automatically join it."

This is odd, as we have had IPv6 since 2002 and Blizzard have had it since 2011, so by that statement we should "join it".

At this point it is still not working in my test, still a routing loop within Blizzard's network.

As Blizzard mistakenly marked my ticket resolved, I have opened a new one. Sorry we don't have any more update on this.

Update
7 Aug 14:33:25
There is a Blizzard EU forum post on this matter. http://eu.battle.net/forums/en/wow/topic/17612333772?page=1#post-8
Resolution Blizzard have finally resolved the issue.
Started 22 Jul
Closed 12 Aug 18:21:20

5 Aug 15:00:16
[Broadband and Ethernet] - L2TP services disrupted - Closed
Details
5 Aug 14:43:08

L2TP services were disrupted and reconnected following a replacement of one of our LNSs that handled L2TP. The new box causes services to immediately switch from backup to live service automatically which caught us by surprise slightly.

There will be another brief disruption shortly then all should be well again.

Sorry for the inconvenience and very short notice.

Resolution The main L2TP server is fully back in service now.
Started 5 Aug 14:35:00
Closed 5 Aug 15:00:16
Previously expected 5 Aug 14:50:00

12 Aug 18:55:27
[Broadband and Ethernet] - Router issue - Closed
Details
12 Aug 18:04:47

One of our edge routers appears to be playing up, we are investigating.

As normal traffic re-routed. We should have this sorted shortly.

Update
12 Aug 22:51:15
Router was taken out of service until we could resolve the issue.
Resolution Router is back in service, thanks to staff going in to data centre this evening.
Started 12 Aug 18:00:00
Closed 12 Aug 18:55:27

9 Jun 22:00:00
[DNS, Email and Web Hosting] - Email problems to gmail addresses - Closed
Details
9 Jun 17:01:30
We're noticing a large number of email to gmail accounts are being bounced back by gmail servers this afternoon. We are investigating the cause of this.
Update
9 Jun 19:53:32
We have put in a temporary fix (sending via IPv4 instead of IPv6) and messages are new being delivered to gmail addresses without being bounced.
Started 9 Jun 16:00:00
Closed 9 Jun 22:00:00

18 Aug 16:00:00
Details
18 Aug 16:30:43

This is affecting customers who we provide email services to for their domains.

We have noticed a small number of emails on the 17th and 18th August were being marked as spam when they are not spam. The problem should be fixed as of 16:30 on 18th August, but if you are missing email please check your spam folder(s) as the email may be in there.

Started 17 Aug 13:00:00
Closed 18 Aug 16:00:00

12 Aug 19:30:17
[Broadband and Ethernet] - New peering for gamers - Info
Details
12 Aug 18:58:16

I am pleased to confirm we have peering with Blizzard, and are working on peering with Riot Games as well.

The Blizzard peering includes IPv6, as they have just fixed their EU realms IPv6.

Peering means we have a more direct route for traffic to and from our customers. We always have various fallbacks, but peering means the best connectivity for lowest latency in your games.

We always take latency seriously on broadband lines, and this just adds an extra edge.

Started 12 Aug 18:55:57

10 Aug 18:00:00
Details
29 Jul 17:19:16
There may be disruption to some outbound SMS between 07:00 and 09:00 BST on Tuesday the 9th of August 2016. Although the window is wide, the disruption is only expected to last 30 seconds. This is due to upstream carrier upgrades.
Started 9 Aug 07:00:00 by Carrier
Closed 10 Aug 18:00:00
Previously expected 9 Aug 09:00:00

8 Aug 10:37:44
[DNS, Email and Web Hosting] - SSL Certificates Updated - Info
Details
8 Aug 10:37:44
We're updating SSL certificates for our email servers today. The old serial number is 11C4A8. The new serial number is 124247. Users who don't have the CAcert root certificate installed may see errors. This does not affect webmail. Details on http://aa.net.uk/cacert.html

3 Aug 16:18:26
[VoIP and SIMs] - SMS billing - Info
Details
3 Aug 15:51:00

Customers may have noticed that some of the SMS itemisation has been wrong on the bills in terms of the time of the message.

Basically, some of the time sent logs were recorded in UTC even in the summer, but the time received was in local time, and so were some of the logs such as SIP2SIM usage.

We have corrected this today, so from now you should see all itemisation on local time.

Customers are reminded that if using our SIP2SIM service there is an airtime charge for sending a text (e.g. 2p) and if the text is to an external number such as someone else's mobile then there is an SMS charge (e.g. 6p), and these are separately itemised on the bill. At least after this change they should be next to each other and not an hour apart.

Sorry for any confusion.

Started 3 Aug 15:49:20

3 Aug 09:39:11
[Broadband and Ethernet] - Price reduction - Info
Details
2 Aug 19:50:54

We are removing the extra £5/month charge for Home::1 customers on BT back haul with 80/20 capped FTTC. This means instead of being £40/month the 80/20 FTTC will be £35/month from your next bill.

This is consistent with our Talk Talk backhaul Home::1 terabyte services where the speed of FTTC does not impact the pricing.

Customers on BT backhaul Home::1 with 40/10 caps that have forecast speeds that could exceed 40Mb/s down or 10Mb/s can order a regrade on our broadband order form for £15. This does not restart the minimum term and usually happens next working day. This is no guarantee that your line will sync over 40/10, it simply changes the caps applied to the line sync.

Started 2 Aug 19:45:00

3 Aug 11:31:20
Details
25 Jul 11:36:34
SIP2SIM may be affected for up to an hour between the hours of 20:00 on the 2nd of August and 06:00 on the 3rd of August. This is due to upstream GGSN upgrade work and the resulting testing of failover.
Started 2 Aug 20:00:00 by Carrier
Closed 3 Aug 11:31:20
Previously expected 3 Aug 06:00:00

Thursday 14:42:10
Details
27 Jul 15:08:24

We use two data centres in London, Equinix and (Telecity). Equinix have planned engineering works on their power feeds on the morning of 2nd August as follows:

Investigations into the power incident that occurred on Wednesday 20th July 2016 have concluded that the outage was caused by the malfunction of the protection module on the primary circuit breaker for the A power riser in Building 8/9 of Equinix's LD8 (Harbour Exchange) data centre. An identical protection module is installed on the B riser power supply. It is now essential that both circuit breakers are replaced. In order to carry out this work safely it is necessary to power down the affected circuits in accordance with the following schedule in the early hours of 2nd August 2016:

A Power riser:
Power down: 00:00
Reenergise: 01:00 or sooner once work is complete

B Power riser:
Power down: 02:00
Reenergise: 03:00 or sooner once work is complete

All our equipment is dual power - so we do not expect there to be any problems during this period.
Started 2 Aug
Closed Thursday 14:42:10
Previously expected 2 Aug 03:00:00

28 Jul 16:47:17
Details
20 Jul 16:56:12
On Tuesday the 26th and Wednesday the 27th of July 2016, between 22:00 and 23:00, upstream router maintenance is expected to cause minor disruption to data SIM connectivity. The disruption should not be for more than a few minutes a time.
Started 26 Jul 22:00:00
Closed 28 Jul 16:47:17
Previously expected 27 Jul 23:00:00

20 Jul 12:41:10
Details
20 Jul 08:18:13
We can see a major routing issue, and we are investigating. Not entirely clear the extent of the problem at this stage.
Update
20 Jul 08:21:55
At the moment we are seeing what appears to be an issue with the links between London and Maidenhead. This can impact some Ethernet customers, VoIP, hosting, etc. However, a number of systems have a fallback for this, so many are still working, e.g our VoIP services are OK.
Update
20 Jul 08:22:23
The link is partially back, and we are investigating the cause.
Update
20 Jul 10:19:15
This is looking like a major fault within BT's core network. This is affecting many ISPs in the UK. We'll post more details when we can.
Update
20 Jul 10:19:48
also see http://www.bbc.co.uk/news/technology-36844712 and http://www.ukinternetreport.co.uk/?p=d&id=215 Which are reporting the problem within BT's network.
Update
20 Jul 10:30:51
This fault is affecting our communications with BT's systems. This means that our sales and support staff have very limited access to BT's ordering, diagnostic and fault reporting problems. so please to bear with us as staff will be struggling to do their work at the moment. (TalkTalk lines are ok though)
Update
20 Jul 10:35:40
This is looking related to a power issue at a major london datacentre
Update
20 Jul 10:48:05
We've had an update from BT regarding this issue, they say: There is an intermittent incident affecting the Openreach B2B and XML Gateways, as a result of this issue you may be experiencing a delay in the receipt of KCI's for both Fulfilment and Assurance requests as well as failures in Dialogue Services.
Update
20 Jul 12:40:00
It looks like everything affected by this outage has returned to normal now.
Update
20 Jul 12:40:06
It looks like everything affected by this outage has returned to normal now.
Update
20 Jul 12:40:23
It looks like everything affected by this outage has returned to normal now.
Started 20 Jul 08:04:00
Closed 20 Jul 12:41:10

12 Jul 21:21:00
Details
12 Jul 21:24:48
We've had some lines dropping and reconnecting this evening, between 20:50 and 21:20. Things seem back to normal now, but we're investigating the cause.
Update
12 Jul 21:26:13
During the times above customers would have experienced routing problems, perhaps packetloss and VoIP problems. (although my Netflix stream was unaffected smiley )
Update
12 Jul 21:34:11
Sorry about that everyone, early indicators is that a few DLS lines were being DOSd, and although our automated systems did drop the PPP and blackhole their IPs across the most of the internet, it did cause problems for some customers. We'll continue to monitor the network this evening and will do some more investigations in the morning as to the cause and how to help prevent this sort of thing in the future.
Started 12 Jul 20:50:00
Closed 12 Jul 21:21:00

8 Jul 04:00:00
[Broadband and Ethernet] - TalkTalk Outage - Closed
Details
8 Jul 04:17:51
We lost TalkTalk lines shortly after 2am, and they were down for around 100 minutes. Our links to TalkTalk were either down or not passing traffic for this time. Things are returning to normal now, and we're monitoring the links and chasing TalkTalk for an explanation.
Update
8 Jul 08:00:48
Service was restored at around 4am. We are investigating the cause. It looks like there was planned work within TalkTalk at their Telehouse location, but there was some problems with lines re-connecting via our Telecity connection. More information to follow as we get it.
Started 8 Jul 02:10:35 by Customer report
Closed 8 Jul 04:00:00

4 Jul 12:48:25
Details
30 Jun 15:10:48
We are aware of problems affecting our L2TP and data SIM service, this is being investigated by our engineers now.
Update
30 Jun 16:13:47
Apologise this has not yet been resolved, we are seeing further problems and it's still being investigated.
Started 30 Jun 15:09:09
Closed 4 Jul 12:48:25
Previously expected 30 Jun 19:09:09

19 Jun 13:39:55
[Broadband and Ethernet] - L2TP direct service - Info
Details
3 Apr 16:36:34

Now that we have the new network upgrade we are hoping to properly launch the new L2TP specific service. Previously this was a somewhat informal service with somewhat complicated billing.

This new service is an L2TP over IP service for people using other broadband lines to access us and use our IPs and services including fixed IPv4 and IPv6 blocks. The plan is £10/month for a service capped at 100Mb/s with 1TB monthly allowance, clamping to 3Mb/s from next month's allowance on reaching allowance. Very similar to Home::1. No date yet but we are hoping to make this our standard L2TP service some time in April.

Update
17 Apr 19:34:56
I'm sorry this is taking so long to sort - it is basically a tariff change, and order form, but we have been very keen to ensure the infrastructure is in place for this. I still hope to do this by end of month.
Update
12 May 11:09:15
Information: http://aa.net.uk/broadband-l2tp.html
Order form: https://order.aa.net.uk/chaos2l2tp.html
Started 1 Apr
Previously expected 1 May

17 Jun 18:38:00
Details
17 Jun 16:38:45
Talktalk have informed us that they are experiencing issues with one of their network routers in London. This has impacted all Talktalk lines in the surrounding area, covering ADSL, VDSL(FTTC) and EoFTTC lines since approximately 15:21. Affected areas are Canonbury, Lower Holloway, Kentish Town, Upper Holloway, Kingland Green and Clerkenwell.
Update
17 Jun 19:56:24
All affected customers appear to be back online now, and TalkTalk have advised us that a fibre break has been repaired.
Resolution Openreach have repaired a break in TalkTalk's fibres.
Started 17 Jun 15:21:00 by TalkTalk
Closed 17 Jun 18:38:00
Cause TT

17 Jun 04:00:00
Details
16 Jun 21:23:10
Talktalk have informed us that they are experiencing issues with their network core in Swansea. This has impacted all Talktalk lines in the surrounding area, covering ADSL, VDSL(FTTC) and EoFTTC lines since approximately 19:53. Talktalk have engineers on their way to the equipment to ensure it is correctly restarted, and currently believe the outage is due to a power failure in the BT exchange which hosts the equipment. BT Openreach have been informed of the power failure, and are also investigating. The following area codes and exchanges are impacted by this outage: 01792, 01554, 01269, 01639, Llanboidy, Amman Valley, Pontardawe, Port Talbot, Llangennech and Crosshands
Update
16 Jun 21:48:54
We have received reports that the outage is also effecting the 01656 area code, and lines around Bridgend. We will post further updates when we have information from Talktalk.
Resolution The cause of this incident has not been determined. BT confirmed they had a fibre related issue in Merthyr Tydfil and restored our service by moving the Talk Talk circuits to a spare fibre cable. As no further issues have been reported, we can confirm closure to this incident with further investigations taking place under the Problem Management process.
Started 16 Jun 19:53:00 by TalkTalk
Closed 17 Jun 04:00:00
Previously expected 16 Jun 22:30:00

17 Jun 10:39:29
Details
6 Jun 14:00:34
Our carrier for Data only SIMS are performing a software update on their LNS, this will mean a short outage for data SIMS for a minute or two between 10PM and 11PM on Thursday 9th June.
Update
10 Jun 10:10:01
Aql will be performing another upgrade on the remaining infrastructure on Thursday 16th June 2016 at 22:00. This part of the migration to the new LTE infrastructure. They have advised that the patch has been tested in lab conditions, and no issues are expected, however in the event of a fault they revert the patch and investigate. As with last night some of our sessions may disconnect and reroute while the patch is applied and the LNS restarted, however no more than five minutes disruption is anticipated.
Update
17 Jun 10:40:45
Looks like planned works completed successfully last night. We will post further information on our carriers LTE upgrades as we receive them.
Started 16 Jun 22:00:00 by Carrier
Closed 17 Jun 10:39:29

4 Jun 12:33:49
Details
3 Jun 09:33:49

We now have new options for VDSL (Fibre to the cabinet capped at 80/20) with terabyte allowance and free install. This no longer forces you to include the phone line copper pair.

We also now have the option if you are taking over the phone line and it has a BT number you want to keep where we can port the number to VoIP. This is a cost for phone line install or take over.

This gives a lot more choice on the Terabyte services. It also allows ordering where the line is on a satellite exchange and could not get TT MPF (phone line) before. Do try the order form to see if it is available to you now.

http://aa.net.uk/broadband-home1-terabyte.html

Update
4 Jun 12:33:49
We have worked out that around half of the customers currently with ADSL services could upgrade to VDSL. We are working on showing the estimated speed for FTTC upgrade on the control pages and bills.
Started 3 Jun

27 May 20:15:12
Details
27 May 18:59:26

A number of customers were emailed that their 1st of month DD was cancelled.

This process was stopped as soon as we realised what was happening so not many people will be affected. You will receive normal DD collection notices in due course for your monthly bills or have collection within 3 working days as per exiting notice (so may be 3rd instead of 1st) as per DD rules.

Our DD management system is coded to be as cautious and customer focused as possible and a minor error meant it was not happy with the DDs for regular collections being in place before the invoice was issued. This has been corrected now.

Sorry for any confusion caused.

Started 27 May 18:00:00
Previously expected 27 May 22:56:44

27 May 04:30:00
Details
27 May 02:24:53
At around 1:51 today TT dropped offline. We are investigating.
Update
27 May 02:56:38
We've seen reports of other ISPs lines via TT also having the same problem.
Update
27 May 03:16:06
This looks very much like a major outage within TalkTalk. Our BGP session to them that carry our traffic is down at one datacentre and up and our second datacentre but with no prefixes being advertised to us from TalkTalks side.
Update
27 May 03:58:05
We are seeing some lines starting to reconnect.
Update
27 May 03:58:26
We are seeing some lines starting to reconnect.
Update
27 May 04:01:19
No word from TT as to the cause of this, but we'll update this post in due course.
Update
27 May 04:14:03
Majority of lines are back on now - waiting for the last ones to reconnect.
Resolution It looks like this was caused by some TT planned work of which we were unaware, and clearly did not go to plan. We are trying to get a full explanation from TT. In the mean time the problem is resolved. If you are off line, try restarting your router to force it to reconnect.
Started 27 May 01:51:00
Closed 27 May 04:30:00

19 May 21:55:00
Details
20 May 09:03:45
On the evening of 19th May a small number of lines all in the Guildford area (GU postcodes) were offline between around 21:35 and 21:55. This only affected TalkTalk connected circuits. We'll investigate the cause of this with TalkTalk and will update this post shortly.
Resolution TalkTalk have confirmed that they did have a major network outage. This was resolved by TalkTalk rebooting one of their main BRAS aggregation devices. They do apologise and have raised the problem with their vendor.
Started 19 May 21:35:00
Closed 19 May 21:55:00
Cause TT

14 May 17:36:17
Details
14 May 17:36:17

I have added some more features to card printing.

https://cards.aa.net.uk

It now included linear barcodes including code 39, code 128, EAN13, UPC, EAN8, EAN5 and ITF

Started 14 May 17:34:55

20 May 14:30:35
Details
13 May 21:45:49
Our upstream carriers have advised us of scheduled maintenance taking place between 02:00 - 03:00 on Friday 20th May 2016 affecting Data SIMs only.
Some customers may notice a momentary drop in connections during this time as any Data SIMs using that route will disconnect when the link is shut down. Any affected Data SIMs will automatically take an alternate route when they try and reconnect. Unfortunately, we have no control over the timing of this as it is dependent on the retry strategy of your devices. During the window, Data SIM connectivity should be considered at risk throughout.
Voice SIMs are not affected by this maintenance.
Started 13 May 20:54:24
Closed 20 May 14:30:35

12 May 17:00:28
Details
12 May 16:58:38

We use IRC all the time here at AAISP, both internally between staff teams and in helping our customers. Many customers have their own IRC client setup on their computer whilst others pop in to the IRC channel by using the web-based version we provide via our website.

To help improve the experience for customers using our web-based irc client we have a new one to try:

https://webirc.aa.net.uk/

We'll be updating our links on the website to the new one next week, but feel free to try out the new client.
Started 12 May 16:50:00

5 May 16:52:53
Details
5 May 14:42:29
We've seen several incidents, at 12:25, 12:33 and 14:26 where we lost connectivity over all of the VLANs we currently utilise on our Datahop circuits in Telehouse North. This has affected a few of our wholesalers who connect in to us over Datahop VLANs. This looks like it's relating to a fault within datahop. The link is up at the moment.
Update
5 May 16:25:36
We've had an update from datahop saying that they have replaced a switch in their network. Service has remained stable since 14:30 this afternoon.
Update
5 May 16:54:00
Datahop have completed their switch replacement work. This incident is now closed.
Resolution Faulty switch within Datahop has been replaced.
Started 5 May 12:30:00
Closed 5 May 16:52:53

28 Apr 16:18:20
Details
28 Apr 15:30:05
We are seeing a number of broadband lines in the south of London suffering from severe packet loss and high latency. This looks like a fibre problem with TalkTalk. This has been reported and as soon as we have an update we will let you know.
Update
28 Apr 15:36:41
Here is an example graph
Update
28 Apr 16:18:44
TalkTalk have fixed this - that was quick.....
Started 28 Apr 15:27:37
Closed 28 Apr 16:18:20
Previously expected 28 Apr 19:27:37

2 May 17:36:35
[Broadband and Ethernet] - At risk: Router upgrades - Completed
Details
28 Apr 09:29:41
We may do some router upgrades and changes over the weekend as part of ongoing network tidy-up. This should not be disruptive.
Update
30 Apr 08:52:48
The first changes have been done, with most likely impact (if any) being on our office connections. If all goes well today we'll do more changes tomorrow (Sunday) morning. As I say, these should have no noticeable impact, but are to improve some of our internal routing logic.
Update
1 May 05:47:14
The next stage was done this morning. Any issues please let us know on irc. We have more changes later today (Sunday).
Update
1 May 09:53:27
Some further work was done before 8am today (Sunday). We plan to do the next stage tomorrow (Monday) morning. So far all seems to be going to plan.
Update
2 May 11:01:20
One small causality was our NAT64 gateway, which is being restored now.
Update
2 May 16:51:30
We have identified a possible peering MTU issue and are investigating. There may be more work this evening (Monday). Again, we expect no disruption, but a risk.
Update
2 May 17:18:19
Yes, we have identified an issue with MTU on some peers, and this is being rectified now.
Update
2 May 17:37:08
MTU fixes now applied
Resolution The main planned work was all completed before 8am this morning, all looks well. Any issues contact support or mention on irc or text the MSO number. We'll be keeping an eye on things today anyway.
Started 30 Apr
Closed 2 May 17:36:35
Previously expected 3 May

28 Apr 04:55:55
Details
27 Apr 11:21:25
We are making a correction to routing. This will impact external connectivity for a few seconds but should not cause lines to drop PPP. We aim to do this over night or early tomorrow morning. Sorry for any inconvenience.
Resolution Changes complete
Started 28 Apr
Closed 28 Apr 04:55:55
Previously expected 28 Apr 08:00:00

27 Apr 06:09:11
[Broadband and Ethernet] - At risk: Routing changes - Completed
Details
26 Apr 16:58:57
We are tidying up some of our internal routing logic following from our major network upgrades recently. This is expected to have no impact, but there is a risk it could impact Ethernet and broadband customers and maybe other services. The plan is to get this all done well before 8am so that we can sort any issues that come up or reverse changes that are made.
Resolution The work is complete and all looks well - any issues, please let us know. P.S. an issue with BGP connections via DSL was resolved.
Started 27 Apr
Closed 27 Apr 06:09:11
Previously expected 27 Apr 09:00:00

26 Apr 17:00:00
Details
26 Apr 16:40:46

Some Ethernet customers have seen a routing blip today. This is mainly those that have a secondary block of IPs routed via your Ethernet link and related to that additional block.

We should have posted an "at risk" notice about some very minor routing changes tidying up after all of the recent work in our core network. I genuinely did not expect any impact - we have done simple routing changes many times in the past without experiencing any issues.

Unfortunately the changes uncovered an underlying problem with the redundancy logic, and hence something that should have had no impact did in fact have an impact. This was fixed quickly within minutes of the report.

There is other work we expect to do tomorrow morning before 9am, including sorting the underlying issue we have uncovered today. If there are any issues you encounter with routing in the morning, please do contact support.

Sorry for any inconvenience caused.

Started 26 Apr 13:00:00
Closed 26 Apr 17:00:00
Previously expected 26 Apr 17:00:00

19 Apr 12:42:22
[General] - DRIPA - Info
Details
16 Jul 2014 10:13:15
Just to be clear on our policy here - when DRIPA comes in to force, and if A&A become subject to a retention notice for all customers, we aim to work on all practical legal means to minimise the amount of data retained under that legislation - making full use of the bad wording in the Schedule in the 2009 regulations where possible. We also aim to clearly publish what is retained under such a notice and what steps we have taken to minimise such data. Such steps may mean separate companies running email or other services, or even hosting some servers outside the UK, if those are practical steps we can take. Why? Because blanket mass surveillance is illegal under EU law as it is against our basic human right to privacy as decided by a court, that's why! Feel free to comment on my blog.
Started 15 Jul 2014