04 jan

Darkfiber supplier maintenance [Update]

Startdate: January 8th 2010 22:00:00(CEST)
Enddate: January 9th 2010 06:00:00(CEST)

Dear customer,

One of our darkfiber suppliers announced a maintenance which is affecting our rings.
During this maintenance traffic will be rerouted, no impact is expected.

There could be some short periods of packetloss while OSPF reconverges over the alternative path.

Best regards,

LeaseWeb – NOC

02 jan

ddos attack

Time: January 2nd 2010 03:11:00(CEST)
Dear Customer,

We suffered a ddos attack which caused some bgp sessions to flap. This may have caused an interruption to some of our customer. At the moment everything is stable again and we will keep monitoring the situation.

Regards,
LeaseWeb BV

25 dec

EvoSwitch router problem [Update]

Time: December 25th 2009 08:00:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Since this router is a standby router most customers will not experience any problems.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 8:30] We have rebooted this router, and all traffic has been restored. We will investigate the cause of this issue.

[UPDATE 10:10] The router experienced the same problems again, we have reloaded the router again so the the router will stay online till our engineer arrives. We are preparing a module replacement, and will do this when the engineer arrives.

[UPDATE 11:20] Our engineer arrived at EvoSwitch and we now continue with replacing the faulty module.

[UPDATE 12:00] Our engineer replaced the faulty module successfully and the router is online again.

[UPDATE 13:00] The router is still experiencing issues, we are still investigating. When we have an update we will update the NOC page.

[UPDATE 14:30] One module in this router was ‘poisoning’ the other modules, after we replaced this module the problems dissapeared. Our apologies for any caused inconvenience.

25 dec

EvoSwitch router problem [Update]

Time: December 25th 2009 08:00:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Since this router is a standby router most customers will not experience any problems.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 8:30] We have rebooted this router, and all traffic has been restored. We will investigate the cause of this issue.

[UPDATE 10:10] The router experienced the same problems again, we have reloaded the router again so the the router will stay online till our engineer arrives. We are preparing a module replacement, and will do this when the engineer arrives.

[UPDATE 11:20] Our engineer arrived at EvoSwitch and we now continue with replacing the faulty module.

[UPDATE 12:00] Our engineer replaced the faulty module successfully and the router is online again.

[UPDATE 13:00] The router is still experiencing issues, we are still investigating. When we have an update we will update the NOC page.

25 dec

EvoSwitch router problem [Update]

Time: December 25th 2009 08:00:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Since this router is a standby router most customers will not experience any problems.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 8:30] We have rebooted this router, and all traffic has been restored. We will investigate the cause of this issue.

[UPDATE 10:10] The router experienced the same problems again, we have reloaded the router again so the the router will stay online till our engineer arrives. We are preparing a module replacement, and will do this when the engineer arrives.

[UPDATE 11:20] Our engineer arrived at EvoSwitch and we now continue with replacing the faulty module.

[UPDATE 12:00] Our engineer replaced the faulty module successfully and the router is online again.

25 dec

EvoSwitch router problem [Update]

Time: December 25th 2009 08:00:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Since this router is a standby router most customers will not experience any problems.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 8:30] We have rebooted this router, and all traffic has been restored. We will investigate the cause of this issue.

[UPDATE 10:10] The router experienced the same problems again, we have reloaded the router again so the the router will stay online till our engineer arrives. We are preparing a module replacement, and will do this when the engineer arrives.

[UPDATE 11:20] Our engineer arrived at EvoSwitch and we now continue with replacing the faulty module.

25 dec

EvoSwitch router problem [Update]

Time: December 25th 2009 08:00:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Since this router is a standby router most customers will not experience any problems.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 8:30] We have rebooted this router, and all traffic has been restored. We will investigate the cause of this issue.

[UPDATE 10:10] The router experienced the same problems again, we have reloaded the router again so the the router will stay online till our engineer arrives. We are preparing a module replacement, and will do this when the engineer arrives.

25 dec

EvoSwitch router problem [Update]

Time: December 25th 2009 08:00:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Since this router is a standby router most customers will not experience any problems.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 8:30] We have rebooted this router, and all traffic has been restored. We will investigate the cause of this issue.

24 dec

router problem [Update]

Time: December 24th 2009 08:10:00(CEST)
Expected Downtime:Unknown
Location: EvoSwitch
Affected ranges : 94.75.218.0/23
94.75.232.0/24
94.75.250.0/24

*******************************

Dear Customer,

At the moment we experience some troubles with one of routers. Engineer onsite is checking the router at the moment.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 08:23] We had to reboot this router as it was crashed. Currently all connections are online again and traffic is restoring. We are still investigating the cause of this issue.

24 dec

Defective blade [Update]

Time: December 18th 2009 13:35:00(CEST)
Expected Downtime:Unknown
Location: EvoSwitch
Affected ranges : 94.75.218.0/23
94.75.232.0/24
94.75.250.0/24

Reason: Hardware failure
*******************************

Dear Customer,

At the moment we experience some troubles with one of our modules in one of our routers.

We are currently replacing this module.
Customers in the mentioned ranges may experience network issues. This will be restored ASAP.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 13.50]

The module has been replaced by our engineers.
All services should be restored. We will closely monitor the blade, for any issues the upcoming hour.

If you still experience downtime of your services, please email support so we can bring your service back online.

Regards,
LeaseWeb BV

[UPDATE 14.50]
All services have been restored.

[UPDATE 17.30]
We are still seeing strange behaviour on this router, we are currently busy to stabalize this issue. When we are not able to stabalize this issue we need to do a reload. If we have any news we will update the NOC page.

[update 17.38]
We are unable to stabalize the issue, we are going to do an emergency reload with a new software version.

[update 18.00]
The router is online again after the reload, we will investigate if all traffic will restore correctly.

[update 18.15]
The router is up for 15 minutes, and we see the traffic restoring to previous values. Please open a ticket if you still have a problem with reaching your server.

Regards,
LeaseWeb BV