We're investigating problems with some incoming calls on our VoIP service.
We've taken one of our call servers (a.voiceless) out of service, and devices should re-register against b.voiceless. We'll still investigating the cause of this.
Some calls are still being intermittent, we're still investigating.
This is still affecting some customers and we're still investigating.
The problem has been solved, we're reviewing, please re-register your phone.
This was quite a difficult problem to diagnose and we apologise to those affected. The problem affected, intermittently, about a quarter of phones that register to our VoIP service. Customers who we send calls to your own server or we redirect to other numbers were not affected.
The problem was caused by a misconfigured or buggy phone which hit an edge-case bug in our code. This caused registration updates of that phone to fail. This only affected one of four back-end servers, but it did mean that further registration attempts from other phones to this back-end server also failed. A short term fix was applied and registration details were then being updated successfully.
We do have already lots of monitoring and logging from our VoIP platform but this fault has shown that we can do better and we will be working to improve this so as to help diagnose faults quicker in future.