We'll be performing routine overnight software upgrades on our pool of LNSs and BGP routers, starting Thursday 9th October. One LNS will be done per night, and this involves moving lines between the LNSs and upgrading when lines have been moved off. Customers may see a PPP drop and reconnect in the early hours of the morning as this work is carried out.
Upgrades to our BGP routers will also happen overnight but should have little to no impact.
All of our LNSs have been upgraded. We have a few more BGP routers and L2TP routers to upgrade, which will happen on Monday and Tuesday next week. (20th & 21st October)
Due in 1 day
This notification is for customers who run their own authoritative DNS servers and use our secondary-dns.co.uk as secondary (slave).
As part of our DNS infrastructure project we will start to initiate Zone Transfers from an additional set of IP addresses.
You will still send your NOTIFY to secondary-dns.co.uk but you will start to see AXFR requests from an additional set of IP addresses.
Therefore, please update your ACLs to allow the following addresses in addition to what you have at the moment:
Please update your ACLs by March 11th.
We will have an overlap of using the existing DNS servers and the new DNS servers. As part of our testing we will start to use the new IP addresses from February 17th
Some customers have been experiencing irregular phone call behaviour - such as when incoming calls are set to ring multiple phones, some phones carry on ringing after the call has been successfully answered elsewhere.
We have been investigating the problem, and have planned a software upgrade on some of our back-end servers for 6AM Thursday morning which will then enable some further configuration changes thereafter to improve the situation.
Our early morning work is sell under way, there will be a brief disruption to services at around 06:40-07:00 which will cause new calls to not connect for few short moments.
The disruption was short lived and didn't affect calls that were in progress.
We've had some further problems this afternoons with some call and registration failures. This is related to our work to improve our back-end systems. This is being worked on.
We are still seeing call setup or registration problems and are working to resolve this. More information to follow shortly.
Some background information: Our VoIP system has a cluster of back-end servers which manage SIP messaging which involves tracking registered phones, call routing information, and so on. We had needed to improve the performance of this system as we were seeing occasional timeouts. There were a few steps required to carry out this work. We needed to upgrade the OS of our servers (Completed on Thursday morning), and then change our code to run in a new, more efficient, way. We also need run our code in a 'migration' mode which means running both the old method and new method at the same time, so that the new system can have its databases populated and indexed. The plan was to run that for a period of time and we started this earlier this morning. Even though this procedure was tested earlier in the week, today's work hasn't quite gone to plan, and we're currently experiencing an increase of timeouts which is affecting the service. This is being investigated.
We are putting in place a short term change which should help reduce the load, we are also reviewing the cause of the timeouts as there may be a separate root cause of the timeouts.
Our efforts to reduce load have not completely helped, we have still seen intermittent call problems this afternoon. We are working on other avenues to tackle this.
We have code changes prepared that we'll apply early Friday morning.
New code is being installed now.
The service has been running well this morning.
Some call and registration failures have been the result of slow back-end systems. We have made various optimisations and code changes which have been working well throughout the day (Friday). We also have avenues for further improvements which we be implemented in the near future. We do apologise to those who had calls disrupted due to this.
This will search through the various outages and maintenance reports we have received from our suppliers which may be affecting you.
You can also see a list of outages and maintenance work we receive from our suppliers that may affect your specific broadband service on the Control Pages.
The search box above should find posts affecting your service however, you can use a Keyword search.
This is the status page of Andrews & Arnold Ltd.
Our status page shows outages (problems) and maintenance (planned work) that happen on our own network and systems and also that of our suppliers networks and systems. We try and ensure this site is updated as soon as possible with incidents as they happen. Live discussion of issues is usually available on IRC.
The last update was 17 Oct 17:26:12