Internet problem
MAJOR Closed Internet
POST CREATED
Jul 06, 02:56 PM (1 month ago)
STATUS
Closed
AFFECTED
Internet
STARTED
Jul 06, 02:45 PM (1 month ago)
CLOSED
Jul 06, 03:08 PM (1 month ago)
REFERENCE
42442 / AA42442
INFORMATION
  • INITIAL
    Post created: 1 month ago by Andrew

    at around 14:45 there was some sort of problem that affected traffic, we're investigating the cause.

  • UPDATE
    1 month ago by Andrew

    Things were back a few minutes later.

  • RESOLUTION
    1 month ago by Andrew

    We caused a network loop between two of our switches in one of our London datacentres. This was caused by loading wrong software variant on to one of our test FireBrick FB9000 routers; the software variant that was loaded didn't have the LACP feature and when the FireBrick booted it caused a network loop. This caused a broadcast storm and general chaos until the switch disabled the ports. We do apologise for the inconvenience this caused customers this afternoon, lesson learnt and software builds have been changed to include LACP in all variants.

  • Closed