We have not done an LNS switch for some time, and it is usually a routine thing when we do s/w upgrades so not even posted on planned works. However, after the issues with a BT fibre yesterday resulted in some unexpected effects, we are planning some updates.
As some of you will know, the same effect, only lasting a few minutes, happened today as well. This was caused by a minor configuration change, and was unexpected.
It seems that we have found what triggers these problems. We need to investigate in more detail, and we will be doing tests on the test LNS. When we have a proper fix we will update the live LNSs. In the mean time we will avoid the config change that caused this.
However, right now, one of the LNSs is "stuck' in this broken state. It is only affecting some lines, and they have been moved to another LNS already. We will however be clearing the remaining lines (around 10% of customers) this evening. We can then restart the LNS in the morning and be back to normal.
The effect will be a PPP restart and should only be a few seconds downtime for those affected (but depends on your router).