Last 10 posts, sorted by last updated
Limit to posts

MAINTENANCE Assumed Completed Maidenhead datacentre
AFFECTING
Maidenhead datacentre
STARTING
Mar 05, 08:00 AM (11¼ days )
DESCRIPTION

From the datacentre:

As part of Pulsants planned maintenance regime, we have been informed by the power company that they need to undertake a inspection of the infrastructure that delivers power on site, in alignment with these works, we have decided that we will align these works with our annual darken building test to ensure all infrastructure works correctly in the event of a full loss of mains power and to minimise further interruptions.

WHY: The Annual Darken Building Test and transformer inspections forms part of our stringent maintenance regime and this is to ensure all infrastructure works at its optimum but also to ensure it correctly operates in the event of a full loss of mains power to site. To do this we will be working with our 3rd party contractors and the electricity company to isolate the mains power to the site and perform essential testing of our HV and LV infrastructure and then to carry out the transformer inspection work. During this period, it will be necessary to run the site on generators. Pulsant will follow our Darken Building Test procedure, which will isolate the incoming utility supply to site The testing will be completed by Pulsant staff and a number of specialised 3rd party contractors.

The following works will be carried out; -
1. Isolation of utility mains incomer to site.
2. Monitor failover to Generator and ensure supporting site (Power to Racks will remain UPS protected throughout).
3. Isolation of Transformer to site
4. Carrying out associated inspections on the infrastructure
5. Once inspections and servicing has been completed the power will be restored in a controlled manner.
6. Reinstatement and check that all services back to a normal state.

Monitoring will continue for a further 40 minutes post restoration of utility power to ensure there are no outstanding issues.

All diesel supplies to the site have been checked and confirmed adequate supplies, reserve supplies are on standby. Once works have been completed, the fuel supplies will be replenished accordingly. We have provisioned a maintenance window of up to 10 hours to allow us an adequate time window to complete all the necessary inspections, tests and servicing. All personnel will remain on-site during this maintenance window and for a period of up to 1-hour post the maintenance being completed window. Works are expected to be non-service affecting as customers will remain on UPS protected supplies throughout until all works have been completed. This maintenance does involve changes to power source to the site, hence an At-Risk notification is being sent out.

Pulsant has been preparing extensively for this scheduled Darken Building Test and worked with the associated external parties to align these works.

We will have a number of different contractors and Pulsant personnel on site including expert contractors to support our Generators, UPS devices, electrical infrastructure and most importantly Pulsant network and site support staff. We have worked with the electricity utility supplier to mitigate risk as far as possible, below is the planned approach with regards to completing all necessary works and undertaking this Annual Darken Building Test

SERVICE IMPACT: NONE - No interruption or disruption to service is expected during these essential works, however due to the level of switching that will be undertaken during executing our Darken Building Test procedure and isolation of power so that the mains incomer and transformer can undergo their inspection, we are issuing a courtesy At Risk notification to all our customers.

WHEN: Thursday 5th March 2020, commencing at 08.00 GMT and completed by Thursday 5th March 2020 by 17.00 GMT. To mitigate risk and impact, Pulsant have scheduled for these works to be carried out during the working day. The At Risk window will be kept to an absolute minimum and the site will have utility mains power restored and resuming full site protection at the earliest opportunity.


MAINTENANCE Completed Email services
AFFECTING
Email services
STARTED
Jan 30, 06:00 AM (23½ days ago)
CLOSED
Jan 30, 08:00 AM (23½ days ago)
DESCRIPTION
We will be carrying out tests of some internal systems on Thursday morning from 6AM through to 7AM. These tests may have a small impact of connectivity to other services, namely POP3 and IMAP for email. Any disruption should be short lived and not noticeable.
Resolution:

MAINTENANCE Completed Broadband
AFFECTING
Broadband
STARTED
Jan 29, 06:00 AM (24½ days ago)
CLOSED
Jan 29, 07:28 AM (24½ days ago)
DESCRIPTION
We will be carrying out some work Wednesday morning from 6AM through to 8AM as part of bringing up new hostlinks to BT. We do not expect this to have an impact on services, but it is a period where work is being carried out on our core network and so is considered an 'at risk' period of time.
Resolution:

MAJOR Closed Broadband
AFFECTED
Broadband
STARTED
Jan 22, 11:10 PM (30¾ days ago)
CLOSED
Jan 23, 09:05 AM (30½ days ago)
DESCRIPTION
We're investigating a problem with some broadband lines showing 50% packetloss from just after 11PM.
Resolution: The problem was resolved just after 9AM. the reason for this outage is in https://aastatus.net/33308 "Networks have investigated and advised they found a number of errors on a card in LOH which was erroring and causing back plain errors."

MAINTENANCE Completed Extra Capacity
AFFECTING
Extra Capacity
STARTED
Jan 21, 09:00 AM (1 month ago)
CLOSED
Jan 21, 04:30 PM (1 month ago)
DESCRIPTION
We have upgraded our LINX LON1 peering port in Telehouse to 10G. This increases our capacity and more importantly our resilience. We now have 10G ports to LINX in two separate locations.
Resolution:

MINOR Closed Data SIMs
AFFECTED
Data SIMs
STARTED
Jan 13, 09:00 AM (1¼ months ago)
CLOSED
Jan 15, 04:30 PM (1¼ months ago)
DESCRIPTION
Some of our data SIMs are failing to connect. The problem actually started on Friday and has been confirmed today. This is affecting our data-only SIMs, and not our sip2sim's.
Resolution:

MAINTENANCE Completed Broadband
AFFECTING
Broadband
STARTED
Jan 19, 07:00 AM (1 month ago)
CLOSED
Jan 19, 07:31 AM (1 month ago)
DESCRIPTION
We will be carrying out some work Sunday morning from 7AM through to 9AM as part of bringing up new hostlinks to BT. We do not expect this to have an impact on services, but it is a period where work is being carried out on our core network and so is considered an 'at risk' period of time.
Resolution:

MINOR Closed VoIP
AFFECTED
VoIP
STARTED
Jan 14, 01:00 PM (1¼ months ago)
CLOSED
Jan 14, 05:30 PM (1¼ months ago)
DESCRIPTION
We're investigating problems with some incoming calls on our VoIP service.
Resolution:

This was quite a difficult problem to diagnose and we apologise to those affected. The problem affected, intermittently, about a quarter of phones that register to our VoIP service. Customers who we send calls to your own server or we redirect to other numbers were not affected.

The problem was caused by a misconfigured or buggy phone which hit an edge-case bug in our code. This caused registration updates of that phone to fail. This only affected one of four back-end servers, but it did mean that further registration attempts from other phones to this back-end server also failed. A short term fix was applied and registration details were then being updated successfully.

We do have already lots of monitoring and logging from our VoIP platform but this fault has shown that we can do better and we will be working to improve this so as to help diagnose faults quicker in future.


MAINTENANCE Completed Email services
AFFECTING
Email services
STARTED
Jan 09, 07:00 AM (1¼ months ago)
CLOSED
Jan 09, 07:27 AM (1¼ months ago)
DESCRIPTION
We will be carrying out tests of some internal systems on Thursday morning from 7AM through to 7:30AM. These tests may have a small impact of connectivity to other services, namely POP3 and IMAP for email. Any disruption should be short lived and not noticeable.
Resolution:

MAJOR Closed General
AFFECTED
General
STARTED
Dec 18, 09:40 AM (2 months ago)
CLOSED
Dec 18, 12:04 PM (2 months ago)
DESCRIPTION
We're investigating a faulty switch in Telehouse. This will have some impact on services, we're reviewing this at the moment.
Resolution: The switch is back in service. We're still investigating the initial cause of the problem.