Outages
Maintenance
Our Network:
Major

0

Minor

0

Happening Now

2

Future Planned

0

Suppliers' Network:
Open Events:
STATUS
Open
AFFECTING
Email - POP3 collections
STARTED
Feb 11, 08:00 AM (11¾ days ago)
REFERENCE
27990 / AA27990
INFORMATION
  • INITIAL
    11¾ days ago by Andrew

    We have new email servers that we are moving customer to. The move is happening 'behind the scenes' and no server settings on customer email programs will need to be changed. However, there is a minor change that impacts POP3 users regarding a change in how messages we mark as spam are delivered. If you you use POP3 to collect email then please read: https://support.aa.net.uk/POP3_Tips for further information.

    Please do get in touch with us if you have any questions or concerns about this though.

STATUS
Open
AFFECTING
Email Services (POP3 and IMAP)
STARTED
Jan 17, 04:00 PM (1 month ago)
REFERENCE
27530 / AA27530
INFORMATION
  • INITIAL
    1 month ago by Andrew

    We're starting the process of moving customers to new back end storage servers. This is a process that is being done as seamlessly as possible and should not be noticed by customers. Customers do not need to make any change to their email programs.

    The reason for this work is fourfold: increase the storage capacity we have available, improve resilience, improve performance and improve the ease of further capacity upgrades in the future.

    We expect this work to take a few weeks to complete.

  • UPDATE
    11¾ days ago by Andrew

    Just an update on the progress. So far we have moved about a quarter of the mailboxes over to the new servers. We hope to be able to speed up the process this week.

  • UPDATE
    13 hours ago by Andrew

    We are still moving mailboxes to the new severs and have currently moved two thirds. We expect the remaining to take another week or so. There have been a few problems over the past couple of weeks with the old servers which has caused some short outages. We do apologise for this and are moving customers as quickly as possible.

  • NEXT UPDATE...

    Due in 4¼ days

EXPECTED CLOSE
Feb 10, 08:00 PM ( 12¼ days ago)
Broadband blip graph

The graph shows the last few hours of logins and logouts of ADSL, VDSL, SIMs and L2TP circuits.

The current time is on the left. Green is login, red is logout.

If there are spikes, then this shows a large number of logouts, which may indicate an outage or planned work happening.

You can click on a spike to search for incidents or maintenance that were open around that time.

About our status page

This is the status page of Andrews & Arnold Ltd.

Our status page shows outages (problems) and maintenance (planned work) that happen on our own network and systems and also that of our suppliers networks and systems. We try and ensure this site is updated as soon as possible with incidents as they happen. Live discussion of issues is usually available on IRC.

The last update was Today 02:10:13

Contacting us
Our support number is 033 33 400 999, or you can email support@aa.net.uk or text 01344 400 999 to raise a support ticket.

Spotted a Major Service Outage? (MSO)
A Major Service Outage disrupts the service of multiple customers simultaneously. If you believe that a problem affects multiple customers, and is not mentioned here already, text the number above. Begin the text with "MSO". This alerts multiple staff immediately, waking them if necessary. False alarms (i.e. raising MSO for a single line being down) may result in your number being prevented from raising MSO alerts in future. More info.

Regular Maintenance
Thursday evenings, from 10pm, are designated as a general maintenance window where we will perform non-service affecting updates.

Timeline view of events on our network and systems

Events from the AAISP network from the last few months on a scrollable timeline. Mouseover for brief details, click incident to view the full post.

Congestion Reports

Every night we analyse the packet loss and latency from us to our carriers and also from us to each of our customers.

A&A to Backhaul Carriers

This table show un-errored seconds on our connections in to our carriers for the past 7 days. Our aim is for all of these to be at 100%.

Links to carriers

We have links to back-haul carriers BT and TT.

Our target is that we operate a uncongested links to the carriers. This means that if the general trend is that the link will be getting full we order more capacity from carriers. The capacity we order is expensive, takes time to change and has minimum terms. This means it can be difficult to perfectly manage the traffic. We allow some head room but not more than necessary. It is technically impossible to guarantee that the link is always uncongested (not without having a pipe the size of all internet links in the world added together) but by careful monitoring and allowing enough headroom we can aim for that target.

This table shows an analysis of how well we meet our target.

Unerrored seconds

The unerrored seconds report is the simplest statistic to understand. If we drop even a single packet in a second then this counts as an errored second. Given that we can be handling hundreds of thousands of packets every second, this can be a very sensitive measure of congestion. Bear in mind we will drop the larger packets first which will normally be TCP which re-sends the dropped plackets. This helps ensure VoIP and interactive uses of the internet are unaffected.

Daytime unerrored seconds are based only on 9am to 6pm Monday to Friday

Non-premium

We also consider 100 second samples. If we have reduced the throughput of non-premium lines at all (as a result of dropped packets for several seconds in a row) then the whole 100 seconds is considered to have affected non-premium customers. Because we are looking at 100 seconds at a time it is possible for very short busts of traffic to cause this to show a worse figure than the unerrored seconds. Even if traffic was reduced by 0.5% for part of the 100 second period the whole 100 seconds counts as errored.

Premium

Considering the 100 seconds samples, there is a limit to how much we will reduce the non-premium customers. If that limit is reached then we consider the whole 100 second sample is having an impact on premium lines. This is a sign of much more severe bursts of congestion for several minutes. Bear in mind, a single 100 second sample in a day is more than 0.1% of the day, and even if this does happen VoIP and interactive services should be unaffected.

Read more.
Updated 21 hours ago. Link congestion stats
Link Week days 9-6 24 hour Non-premium Premium
TalkTalk 100% 100% 100% 100%
BT 100% 100% 100% 100%
Backhaul Carrier to Customers

This report can pinpoint congestion in exchanges, BRAS's or particular areas of the country.

Method of analysis

Every line is sent an LCP echo every second and loss and latency measured. This is summarised per 100 second sample and archived for every line for each whole day. The previous day is then analysed each morning and the above table is updated.

Each line is considered to pass, fail or be inconclusive. Inconclusive includes lines where samples have too much upload or download which could themselves cause higher loss or latency to be observed. Lines with less than 18 hours on-line are excluded totally.

Latency is considered against a reference base-line latency for the circuit, and this has to be present for several hours to be considered. As such, lines without clean latency monitoring (some makes of router) are excluded.

For an area to show in the above table there must be no lines that pass at all, and at least 50% of the lines must fail (the rest being inconclusive). Where we have very few lines we require 80% to fail (and none to pass) to show on this list. The area must also have enough lines, so some smaller exchanges will not be detected.

The size stated is what percentage of our total lines are in the affected area.

The block graph under each report for yesterday is based on an average of valid samples for each line for each hour, and then taking the lowest value of all lines in the affected area, so the loss and latency shown is something that is affecting every line in an area.

Updated 21 hours ago.
TypeAreaSizeNote
Recently detected, but has been OK since Tue 19 Feb:
BT 21CN LI, all BRASs 0.20% Packet loss approx midnight to 9pm

We are not able to automatically identify all congestion spots so this does not mean there are not problem areas, do contact Support if you think you are suffering from congestion.