16 jul

High Volume Router Issues [Update]

Time: July 16th 2012 14:31:00
Dear Customers,

We are having some high volume router issues in our Haarlem Data Center on the following ranges:

62.212.72.0/23
62.212.82.0/24
62.212.85.0/24
62.212.88.0/23
82.192.94.0/23
82.192.84.0/23
83.149.70.0/23
85.17.24.0/21
83.149.124.0/22
95.211.136.0/21
95.211.144.0/20

We will update this page when more information is available.

[Update 14:41]
The initial cause of the problem was the routing process on the active routing engine.
The process became unresponsive and got killed by the Juniper operating system.
A newly spawned routing process also encountered issues.
We then decided to do a routing engine switchover.
This initially seemed to resolve the issue, but we are finding there are still routing issues.

[Update 14:49]
We are unable to fix the routing process issues on the second routing engine.
The primary routing engine will now be rebooted, once that is back online we will issue another switchover of active routing engine.

[Update 15:04]
We were unable to restore operations on either of the routing engine.
We have decided to initiate a hard reboot of all hardware in the chassis.

[Update 15:12]
Apparently the hard reboot fixed some issues, however the initial issue is still not solved. We are therefore still continue troubleshooting.

[Update 15:31]
Traffic is rebuilding and seems to be restoring. We are investigating the situation and will update this page as soon as there is more information.

[Update 15:42]
Traffic is almost at the same level as before the outage.
If you are still experiencing connectivity issues please open a ticket with support, and clearly mention that you are still experiencing issues.