NGE001.LOH FPC 10 failure. Customers connected via this circuit are experiencing total loss of data services.::WO0000000053631
MINOR Closed TTB-Outages
STATUS
Closed
CREATED
Aug 31, 01:50 PM (4¾ years ago)
TYPE
TalkTalk Outage
STARTED
Aug 31, 12:45 PM (4¾ years ago)
CLOSED
Aug 31, 06:20 PM (4¾ years ago)
REFERENCE
37096 / INC13404650
INFORMATION
  • INITIAL
    4¾ years ago

    Summary
    NOC have identified that that customers connected via NGE001.LOH are experiencing total loss of data services. A list of impacted exchanges will confirmed and provided shortly.
    
     
    N/A
                              

  • UPDATE
    4¾ years ago

    Latest Update Network Support team investigations have identified that the FPC10 card has failed and cannot be recovered remotely. NOC are in the process of engaging an Field Engineer to attend site with a replacement card. The FE’s ETA at site is currently being confirmed. A further update will be provided by 15:30.  

  • UPDATE
    4¾ years ago

    Latest Update Network Support team investigations have identified that the FPC10 card has failed and cannot be recovered remotely. NOC are in the process of engaging an Field Engineer to attend site with a replacement card. The FE’s ETA at site is currently being confirmed. A further update will be provided by 15:30.  

  • UPDATE
    4¾ years ago

    Latest Update The field engineer has attend stores however, there are no replacement cards of the required type available. The Field Engineer will now attend another location to pull a card from a newly commissioned NGE which is not yet carrying traffic. Equinix data centre access has been arranged by NOC and ETA for the FE at the fault location in LOH has been confirmed as 16:30.  

  • UPDATE
    4¾ years ago

    Latest Update The field engineer has attend stores however, there are no replacement cards of the required type available. The Field Engineer will now attend another location to pull a card from a newly commissioned NGE which is not yet carrying traffic. Equinix data centre access has been arranged by NOC and ETA for the FE at the fault location in LOH has been confirmed as 16:30.  

  • UPDATE
    4¾ years ago

    Latest Update The field engineer has now arrived at fault location and will commence replacement of the faulty FPC10 with assistance from the Network Support team. A further update will be provided by 18:00.  

  • RESOLUTION
    4¾ years ago

    Technical / Suspected Root Cause NOC monitoring identified an FPC10 (Flexible PIC Concentrator) failure at NGE001.LOH. This caused a total loss of service to approx. 6k B2B circuits from approx. 12:47 (31/08). The Core Network Ops team were engaged and their investigations found that the FPC10 had failed and could not be restored remotely. To restore service as of approx. 17:23 a field engineer attended site and replaced the faulty FTP10 with support from the core network ops team. This incident will now be closed with any further root cause analysis being completed via the problem management process.  

  • Closed