25 feb

EVO-HV4 router crash [Update]

Time: February 25th 2010 14:05:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch
Affected IP ranges: 85.17.78.0/23 85.17.80.0/22
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[update 14:15]
The router has been reloaded, servers should come online in about 5 minutes.

[update 14:25]
The router is back online. Cause of the crash is being investigated.

19 feb

[20100219] Traffic Graphs showing incorrect values [Update]

Time: February 19th 2010 17:30:00(CEST)
Expected Downtime: not applicable
Location: All
Affected IP ranges: All
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in displaying your Traffic Graphs.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

19 feb

[20100210] Traffic Graphs showing incorrect values

Time: February 19th 2010 17:30:00(CEST)
Expected Downtime: not applicable
Location: All
Affected IP ranges: All
Reason: Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in displaying your Traffic Graphs.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

10 feb

[20100210] EvoSwitch hall 2 network issue [Update]

Time: February 10th 2010 18:30:00(CEST)
Dear Customer,

At the moment we experience some troubles in (part of) our network.

One of our routers located in EvoSwitch has a failing module, since this setup is redundant all traffic switched to the standby router.

Currently this module is being replaced.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Affected IP-ranges:
62.212.66.192/26
62.212.66.192/26
62.212.67.0/26
62.212.67.0/26
62.212.67.128/26
62.212.67.128/26
62.212.67.192/26
62.212.67.192/26
62.212.67.64/26
62.212.67.64/26
62.212.74.0/26
62.212.74.0/26
62.212.74.128/26
62.212.74.128/26
62.212.74.192/26
62.212.74.192/26
62.212.74.64/26
62.212.74.64/26
62.212.91.0/26
62.212.91.0/26
82.192.71.0/26
82.192.71.0/26
82.192.71.128/26
82.192.71.128/26
82.192.71.192/26
82.192.71.192/26
82.192.71.64/26
82.192.71.64/26
85.17.150.128/26
85.17.150.128/26
85.17.152.64/27
85.17.152.64/27
85.17.158.224/27
85.17.158.224/27
85.17.169.64/27
85.17.169.64/27
85.17.171.0/26
85.17.171.0/26
85.17.197.192/26
85.17.197.192/26
85.17.198.32/28
85.17.198.32/28
85.17.199.0/25
85.17.199.0/25
85.17.209.128/26
85.17.209.128/26
85.17.238.128/26
85.17.238.128/26
94.75.198.160/27
94.75.198.160/27
94.75.198.32/27
94.75.198.32/27
94.75.198.64/27
94.75.198.64/27
94.75.202.160/27
94.75.202.160/27
94.75.202.192/27
94.75.202.192/27
94.75.204.192/27
94.75.204.192/27
94.75.213.0/25
94.75.213.0/25
94.75.226.192/27
94.75.226.192/27
94.75.239.32/27
95.211.64.128/25
95.211.64.128/25
95.211.64.32/27
95.211.64.32/27
95.211.65.0/25
95.211.65.0/25
95.211.65.128/26
95.211.65.128/26
95.211.65.192/26
95.211.65.192/26
95.211.66.128/26
95.211.66.128/26
95.211.67.0/26
95.211.67.0/26
95.211.67.128/27
95.211.67.128/27
95.211.67.192/27
95.211.67.192/27
95.211.67.64/27
95.211.67.64/27
95.211.68.0/26
95.211.68.0/26

Regards,
LeaseWeb BV

[UPDATE 18:45]
The failing module has been replaced and all traffic switched back to the active router.

02 feb

[20100303] BBNed maintenance rescheduled [Update]

Startdate: March 3rd 2010 00:00:00(CEST)
Enddate: March 3rd 2010 06:00:00(CEST)

BBNed maintenance

Dear customer,

Our DSL partner BBNed have the following maintenance planned:

_____________________________________________
Activity description : BBned needs to perform maintenance in the core network. These maintenance activities are necessary to assure the stability of our network. During this maintenance we are affecting all our core switches, meaning that all orders will experience a maximum of 5 short outages during this maintenance.

Because the automatic routing process is dynamic we cannot tell which individual orders will be affected at what particular time in the process.

Affected services : Data and voice

Impact : multiple interruptions

Reason : regular maintenance

Date of work : 03-03-2010

Timeframe (CET) : 00:00 – 06:00

Duration : There will be a maximum of 5 interruptions of a maximum of two minutes eac
_____________________________________________

All DSL orders are affected.

[update] The maintenance has been rescheduled from 17/02/2010 to 03/03/2010.

Best regards,

LeaseWeb – NOC

27 jan

[20100227] Storage maintenance [important]

Startdate: February 27th 2010 22:00:00(CEST)
Enddate: February 28th 2010 08:00:00(CEST)

Date: Saturday February 27th 22h00 CET – Sunday February 28th 08h00

Impact: Complete disruption of storage availability
Duration: Approx. 6 hours
Reason: Disruptive upgrade of firmware

Dear Valued Customer,

From Saturday 27th 22h00 to Sunday February 28th 08h00, LeaseWeb has scheduled a large maintenance window on the storage platform. During this maintenance we will upgrade all storage devices to the latest stable releases of the NetApp firmwares and redo some wiring.
The current storage solution doesn’t support non-disruptive upgrades. To ensure the upgrades proceed without any errors, we need to disconnect all connected customers from the storage platform. This will result in a downtime between 6-8 hours.

The upgrade is necessary to improve stability and performance, and have some important bug fixes.


IMPACTED SERVICES
———————–
Virtual Servers (shared/private): Before starting disconnecting the storage solution we will cleanly shutdown all Virtual Machines on our shared/private platforms. After the upgrade we will start all Virtual Machines.

Webhosting: part of our webhosting cluster is using the storage. Websites will be unavailable and a temporary maintenance page will be shown.

Affected IP’s:

85.17.150.98
85.17.150.100
85.17.150.101
85.17.150.102
85.17.150.103


Private connections: Customers having their own virtual platforms or storage connections to our storage infrastructure, are requested to shutdown all connections before 00h00. From 00h00 we cannot guarantee availability during the remaining of the maintenance window. We will inform all customers having such connections.

Please contact [email protected] in case of any questions.

Kind regards,

LeaseWeb Support

27 jan

[20100207] Stockholm connection maintenance

Startdate: February 7th 2010 01:00:00(CEST)
Enddate: February 7th 2010 05:00:00(CEST)

Dear customer,

One of our circuits to Stockholm which we use to get to Netnod (Swedish Internet Exchange), NIX1 (Norwegian Internet Exchange) and DIX (Danish Internet Exchange) will be maintained.

No impact is expected as we will reroute our traffic to other links. During traffic switching and tuning there may be small temporary packet loss or latency increase.

The timezone is CET (GMT+1).

Kind regards,

Leaseweb NOC

20 jan

Stockholm connection maintenance

Startdate: February 7th 2010 01:00:00(CEST)
Enddate: February 7th 2010 05:00:00(CEST)

Dear customer,

One of our circuits to Stockholm which we use to get to Netnod (Swedish Internet Exchange), NIX1 (Norwegian Internet Exchange) and DIX (Danish Internet Exchange) will be maintained.

No impact is expected as we will reroute our traffic to other links. During traffic switching and tuning there may be small temporary packet loss or latency increase.

The timezone is CET (GMT+1).

Kind regards,

Leaseweb NOC

18 jan

DoS EvoSwitch [Update]

Time: January 18th 2010 11:00:00(CEST)
Location: EvoSwitch
Reason: DOS
*******************************

Dear Customer,

We experienced a DoS on one of our routers located in EvoSwitch. We have stopped this DoS, but we are still investigating.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 13:00] The DDoS was attacking one of our customers first, and after that the attacker changed the destination IP to an IP of our router. This caused high CPU load on our routers in Hall 3. We have decided to implement more strict control plane policing to prevent from such attacks in the future.

18 jan

Internal DoS EvoSwitch [Update]

Time: January 18th 2010 11:00:00(CEST)
Location: EvoSwitch
Reason: DOS
*******************************

Dear Customer,

We experienced an internal DoS on one of our routers located in EvoSwitch. We have stopped this DoS, but we are still investigating.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 13:00] The DDoS was attacking one of our customers first, and after that the attacker changed the destination IP to an IP of our router. This caused high CPU load on our routers in Hall 3. We have decided to implement more strict control plane policing to prevent from such attacks in the future.