Time: January 29th 2009 19:00:00(CEST)

Dear Customer,

At the moment one of our carriers – Telia – experiences some troubles in their network in Netherlands. Customer may see higher network latencies and/or packet losses.

We reported the problem to Telia. The reason of the outage is fiber cut, which is currently being repaired. The estimated time of solving the problem is 20:00 UTC (21:00 CET).

We will keep you updated if the situation changes.

Regards,
LeaseWeb BV

Time: January 25th 2009 21:50:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Affected IP ranges: DMZ/Firewall IP ranges
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.
We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 22:20] This issue was caused by a really high load on our Firewall, due to an attack. This attack has been filtered and the issue has been resolved.

Startdate: January 23rd 2009 01:00:00(CEST)
Enddate: January 23rd 2009 01:20:00(CEST)

Dear Customer,

For cleanup purposes we need to reroute an uplink fiber for one of our SBP routers.
During this period we will reroute traffic to remaining uplinks.

Kind regards,

LeaseWeb NOC

Time: January 15th 2009 20:00:00(CEST)
Dear Customer,

Today we discovered Windows servers with a worm infection, called Win32/Conficker.A
The symptoms of the infection are:
– any *.microsoft.com domain can no longer be reached.
– Windows update does not work anymore.
To resolve this issue you will need to run the Malicious Software Removal tool from 13th of January 2009.
Since this file cannot be downloaded from after being infected, we provided a downloadlink for that tool here:

Startdate: January 15th 2009 04:30:00(CEST)
Enddate: January 15th 2009 08:00:00(CEST)

Dear Customer,

During this maintenance window we will migrate the current connections we have with AMS-IX at Nikhef to the CRS router at EvoSwitch.

All peering traffic will be rerouted to out AMS-IX router at Telecity-2

Kind regards,

LeaseWeb NOC

[UPDATE 08:30]
The migration was successful.
We have monitored traffic for some time and everything reports ok.

Startdate: January 15th 2009 04:30:00(CEST)
Enddate: January 15th 2009 08:00:00(CEST)

Dear Customer,

During this maintenance window we will migrate the current connections we have with AMS-IX at Nikhef to the CRS router at EvoSwitch.

All peering traffic will be rerouted to out AMS-IX router at Telecity-2

Kind regards,

LeaseWeb NOC

Time: January 7th 2009 15:52:00(CEST)
Dear Customer,

AMS-IX have reported a problem which can cause problems to our network. We are waiting on additional information.

Regards,
LeaseWeb BV

[UPDATE 17:35]
AMS-IX reported the problems were caused by a crash of their active coreswitch, after they switched over to the backup coreswitch we noticed strange issues in one of the four 10GE ports.

Troubleshooting brought to light the port we were connected to on the backup platform was defective. After this an AMS-IX engineer moved the defective connection to a different port on their switch.



************************************
Update 16:21

AMS-IX had closed the incident and reported the problem as solved. If you still have problems please open a ticket with traceroute at [email protected].
************************************
Update 16:36

Some customers are still having problems connecting to our network. We are investigating the problem.

email from AMS-IX:

Ticket: 22567
Summary: Service interruption
Status: OPEN
Opened: 2009-01-07 15:52 +0100
Type: unscheduled
Scope: AMS-IX
Start: 2009-01-07 15:52 +0100

DESCRIPTION:

We are currently experiencing problems on the AMS-IX platform.

DETAILS:

We have no detailed explanation available yet. We will update
this ticket as we investigate.

ACTIONS:

The AMS-IX NOC is investigating.

Time: January 7th 2009 15:52:00(CEST)
Dear Customer,

AMS-IX have reported a problem which can cause problems to our network. We are waiting on additional information.

Regards,
LeaseWeb BV
************************************
Update 16:21

AMS-IX had closed the incident and reported the problem as solved. If you still have problems please open a ticket with traceroute at [email protected].
************************************
Update 16:36

Some customers are still having problems connecting to our network. We are investigating the problem.

email from AMS-IX:

Ticket: 22567
Summary: Service interruption
Status: OPEN
Opened: 2009-01-07 15:52 +0100
Type: unscheduled
Scope: AMS-IX
Start: 2009-01-07 15:52 +0100

DESCRIPTION:

We are currently experiencing problems on the AMS-IX platform.

DETAILS:

We have no detailed explanation available yet. We will update
this ticket as we investigate.

ACTIONS:

The AMS-IX NOC is investigating.

Time: January 7th 2009 15:52:00(CEST)
Dear Customer,

AMS-IX have reported a problem which can cause problems to our network. We are waiting on additional information.

Regards,
LeaseWeb BV

email from AMS-IX:

Ticket: 22567
Summary: Service interruption
Status: OPEN
Opened: 2009-01-07 15:52 +0100
Type: unscheduled
Scope: AMS-IX
Start: 2009-01-07 15:52 +0100

DESCRIPTION:

We are currently experiencing problems on the AMS-IX platform.

DETAILS:

We have no detailed explanation available yet. We will update
this ticket as we investigate.

ACTIONS:

The AMS-IX NOC is investigating.