12 Oct 2011 23:59:59
[Broadband] - Something is up - Closed
Details
12 Oct 2011 20:48:25

We see usage have occasional spikes, and they often have a reason, but this evening we have seen silly high usage since around 18:40, not only on 21CN but also 20CN. Something is clearly "up" and there is some "internet event" happening. No idea what yet, but has caught us unawares.

The system does the usual to manange the traffic - give more to the "premium" paying customers, and to generally give small packets for VoIP, DNS, and interactive traffic priority.

But we'll try and work out why and if necessary get more capacity in shortly.

P.S. it is not just us cuaght out by this - logs show some lines seeing loss and latency which is within BT's network (as we ensure the LCP echos get sent anyway).

Update
12 Oct 2011 20:53:51

This is worse than the world cup traffic!

Update
12 Oct 2011 20:55:08

Only clue is new Apple IOS5 stuff - if that is the cause I am impressed.

Update
12 Oct 2011 20:57:04

Usage has just reached unprecidented levels - we have not seen anothing like this...

Update
12 Oct 2011 20:59:52

Customers are even reporting issues on Be lines, what is this?

Update
12 Oct 2011 21:02:30

We have core links hitting 1Gb/s - something we have been planning for in 6 months time - this is really unprecidented usage levels.

Update
12 Oct 2011 21:34:07

One good thing from this is that it has provided the FireBrick team with a good benchmark of what happens when you push both the LNS and BGP routers to the full gigabit throughput. The answer is that it copes really well and is using CPU levels that are actaully very good. But what on earth is causing this unprecidented levels of usage is not entirely clear.

So far we thing there is a windows update and apple IOS5 release, but is that all it is?

Update
13 Oct 2011 08:45:44

We are guessing this was IOS5 release.

Update
14 Oct 2011 18:15:07

Whilst Thursday night was also somewhat excessively busy it was nothing like the previous nigh, and caused some congestion mostly on 21CN.

Closed 12 Oct 2011 23:59:59