Internet problem
MAJOR Closed Internet
STATUS
Closed
CREATED
Jul 06, 02:56 PM (1¾ years ago)
AFFECTED
Internet
STARTED
Jul 06, 02:45 PM (1¾ years ago)
CLOSED
Jul 06, 03:08 PM (1¾ years ago)
REFERENCE
42442 / AA42442
INFORMATION
  • INITIAL
    1¾ years ago by Andrew

    at around 14:45 there was some sort of problem that affected traffic, we're investigating the cause.

  • UPDATE
    1¾ years ago by Andrew

    Things were back a few minutes later.

  • RESOLUTION
    1¾ years ago by Andrew

    We caused a network loop between two of our switches in one of our London datacentres. This was caused by loading wrong software variant on to one of our test FireBrick FB9000 routers; the software variant that was loaded didn't have the LACP feature and when the FireBrick booted it caused a network loop. This caused a broadcast storm and general chaos until the switch disabled the ports. We do apologise for the inconvenience this caused customers this afternoon, lesson learnt and software builds have been changed to include LACP in all variants.

  • Closed