Outages
Maintenance
Our Network:
Major Outages

0

Minor Outages

0

Happening Now

1

Future Planned

0

Open Events:
STATUS
Open
CREATED
Oct 07, 12:11 PM (12¾ days ago)
AFFECTING
Network
STARTED
Oct 09, 01:00 AM (11¼ days ago)
REFERENCE
42793 / AA42793
MASTODON
INFORMATION
  • INITIAL
    11¼ days ago by Andrew

    We'll be performing routine overnight software upgrades on our pool of LNSs and BGP routers, starting Thursday 9th October. One LNS will be done per night, and this involves moving lines between the LNSs and upgrading when lines have been moved off. Customers may see a PPP drop and reconnect in the early hours of the morning as this work is carried out.

    Upgrades to our BGP routers will also happen overnight but should have little to no impact.

  • UPDATE
    2½ days ago by Andrew

    All of our LNSs have been upgraded. We have a few more BGP routers and L2TP routers to upgrade, which will happen on Monday and Tuesday next week. (20th & 21st October)

  • NEXT UPDATE...

    Due in 1 day

EXPECTED CLOSE
Oct 22, 06:00 AM ( In 1¾ days )
STATUS
Open
CREATED
Feb 11, 03:50 PM (8¼ months ago)
AFFECTING
DNS
STARTED
Feb 11, 03:30 PM (8¼ months ago)
REFERENCE
42746 / AA42746
MASTODON
INFORMATION
  • INITIAL
    8¼ months ago by Andrew

    This notification is for customers who run their own authoritative DNS servers and use our secondary-dns.co.uk as secondary (slave).

    As part of our DNS infrastructure project we will start to initiate Zone Transfers from an additional set of IP addresses.

    You will still send your NOTIFY to secondary-dns.co.uk but you will start to see AXFR requests from an additional set of IP addresses.

    Therefore, please update your ACLs to allow the following addresses in addition to what you have at the moment:

    • 194.4.172.3
    • 194.4.172.4
    • 194.4.173.3
    • 194.4.173.4
    • 2001:8b6:1:0:194:4:172:3
    • 2001:8b6:1:0:194:4:172:4
    • 2001:8b6:2:0:194:4:173:3
    • 2001:8b6:2:0:194:4:173:4

    Please update your ACLs by March 11th.

    We will have an overlap of using the existing DNS servers and the new DNS servers. As part of our testing we will start to use the new IP addresses from February 17th

Recently Closed Events:
STATUS
Closed
CREATED
Oct 15, 01:11 PM (4¾ days ago)
AFFECTED
VoIP
STARTED
Oct 15, 11:00 AM (4¾ days ago)
CLOSED
Oct 17, 05:00 PM (2½ days ago)
REFERENCE
42794 / AA42794
MASTODON
INFORMATION
  • INITIAL
    4¾ days ago by Andrew

    Some customers have been experiencing irregular phone call behaviour - such as when incoming calls are set to ring multiple phones, some phones carry on ringing after the call has been successfully answered elsewhere.

  • UPDATE
    4¾ days ago by Andrew

    We have been investigating the problem, and have planned a software upgrade on some of our back-end servers for 6AM Thursday morning which will then enable some further configuration changes thereafter to improve the situation.

  • UPDATE
    4 days ago by Andrew

    Our early morning work is sell under way, there will be a brief disruption to services at around 06:40-07:00 which will cause new calls to not connect for few short moments.

  • UPDATE
    4 days ago by Andrew

    The disruption was short lived and didn't affect calls that were in progress.

  • UPDATE
    3¾ days ago by Andrew

    We've had some further problems this afternoons with some call and registration failures. This is related to our work to improve our back-end systems. This is being worked on.

  • UPDATE
    3½ days ago by Andrew

    We are still seeing call setup or registration problems and are working to resolve this. More information to follow shortly.

  • UPDATE
    3½ days ago by Andrew

    Some background information: Our VoIP system has a cluster of back-end servers which manage SIP messaging which involves tracking registered phones, call routing information, and so on. We had needed to improve the performance of this system as we were seeing occasional timeouts. There were a few steps required to carry out this work. We needed to upgrade the OS of our servers (Completed on Thursday morning), and then change our code to run in a new, more efficient, way. We also need run our code in a 'migration' mode which means running both the old method and new method at the same time, so that the new system can have its databases populated and indexed. The plan was to run that for a period of time and we started this earlier this morning. Even though this procedure was tested earlier in the week, today's work hasn't quite gone to plan, and we're currently experiencing an increase of timeouts which is affecting the service. This is being investigated.

  • UPDATE
    3½ days ago by Andrew

    We are putting in place a short term change which should help reduce the load, we are also reviewing the cause of the timeouts as there may be a separate root cause of the timeouts.

  • UPDATE
    3½ days ago by Andrew

    Our efforts to reduce load have not completely helped, we have still seen intermittent call problems this afternoon. We are working on other avenues to tackle this.

  • UPDATE
    3½ days ago by Andrew

    We have code changes prepared that we'll apply early Friday morning.

  • UPDATE
    3 days ago by Andrew

    New code is being installed now.

  • UPDATE
    2¾ days ago by Andrew

    The service has been running well this morning.

  • RESOLUTION
    2½ days ago by Andrew

    Some call and registration failures have been the result of slow back-end systems. We have made various optimisations and code changes which have been working well throughout the day (Friday). We also have avenues for further improvements which we be implemented in the near future. We do apologise to those who had calls disrupted due to this.

  • Closed
Broadband blip graph
About our status page

This is the status page of Andrews & Arnold Ltd.

Our status page shows outages (problems) and maintenance (planned work) that happen on our own network and systems and also that of our suppliers networks and systems. We try and ensure this site is updated as soon as possible with incidents as they happen. Live discussion of issues is usually available on IRC.

The last update was 17 Oct 17:26:12

Contacting us
Our support number is 033 33 400 999, or you can email support@aa.net.uk or text 01344 400 999 to raise a support ticket.

Spotted a Major Service Outage? (MSO)
A Major Service Outage disrupts the service of multiple customers simultaneously. If you believe that a problem affects multiple customers, and is not mentioned here already, text the number above. Begin the text with "MSO". This alerts multiple staff immediately, waking them if necessary. False alarms (i.e. raising MSO for a single line being down) may result in your number being prevented from raising MSO alerts in future. More info.

Regular Maintenance
Thursday evenings, from 10pm, are designated as a general maintenance window where we will perform non-service affecting updates.