24 jul

Emergency EvoSwitch high volume router maintenance [Update]

Startdate: July 11th 2013 07:00:00 (CEST)
Enddate: July 11th 2013 07:30:00 (CEST)

Dear LeaseWeb Customer,

We will perform emergency maintenance on one of our high volume routers at the EvoSwitch data center in Amsterdam to replace one of its route-processors. This has been advised by the vendor in order to resolve the instability issue faced with one of the line cards.

Date: July 11, 2013
Time: 7:00 – 7:30 CEST (GMT+2)

This maintenance will have no impact on traffic. The affected router has the following ranges of IP addresses:

95.211.208.0/21
95.211.60.0/23
95.211.104.0/23
94.75.218.0/23
94.75.232.0/24
94.75.250.0/24
2001:1AF8:4050::/44
2001:1af8:4070::/44

82.150.152.0/25
212.7.206.0/23
212.7.210.0/24

For further information, please contact [email protected]

[UPDATE 07:15]
Unfortunately one of the blades crashed during the maintenance. The blade is now booting.

[UPDATE 07:20]
The maintenance is now completed and everything is working fine. We will analyze logs from the blade that crashed with the vendor. We are sorry for any inconvenience.

24 jul

[20130718] AMS Telecity 2 core router maintenance [Update]

Startdate: July 18th 2013 06:00:00 (CEST)
Enddate: July 18th 2013 07:00:00 (CEST)

Dear LeaseWeb Customer,

We will perform maintenance on our core router at Telecity 2 in Amsterdam. This will be done to upgrade the software to the version recommended by the vendor, that will guarantee continuity of our network.

Date: July 18, 2013
Time: 06:00 – 07:00 CEST (GMT+2)

During this maintenance, our second core router at EvoSwitch in Amsterdam will be fully operational and will handle the whole traffic. This maintenance will have no impact on traffic, only small latency changes while alternate routing paths are selected. The changes in network paths may be experienced across the entire LeaseWeb network.

For further information, please contact [email protected].

[UPDATE 6:25]
The maintenance has been completed successfully

24 jul

Emergency maintenance of the premium traffic router at the EvoSwitch [Update]

Startdate: July 11th 2013 06:30:00 (CEST)
Enddate: July 11th 2013 07:00:00 (CEST)

Dear LeaseWeb Customer,

We will perform emergency maintenance on one of our premium traffic routers at the EvoSwitch data center in Amsterdam to replace one its line cards, which we suspect may fail soon.

Date: July 11, 2013
Time: 6:30 – 7:00 CEST (GMT+2)

Because of redundant setup in the premium network, this maintenance will have no impact on traffic, only small latency changes while alternate routing paths are selected. The changes in network paths may be experienced for the following ranges of IP addresses:

94.75.192.0/18
95.211.0.0/19
62.212.75.0/24
82.192.87.0/24
2001:1AF8:4400::/40

5.61.248.0/24
46.182.176.0/22
77.81.240.0/21
87.236.96.0/21
90.156.224.0/20
94.124.56.0/21
145.255.128.0/24
193.107.18.0/24
212.7.193.128/25
212.7.192.128/25
2A01:9400:2::/48
2A02:CC8::/32
2A03:E280:0:1::/64

For further information, please contact [email protected]

[UPDATE 07:00]
The maintenance completed successfully.

24 jul

[20130724] Outage WDC1 [Update]

Time: July 24th 2013 10:57:00 (EDT)
Dear Customer,

After the maintenance from this morning one of the core switches lost all connectivity.
We are currently still investigating.

Affected IP ranges:
108.59.0.0/20
199.58.84.0/22
199.115.112.0/21
198.7.56.0/21
192.96.200.0/21
91.242.214.0/23
162.210.192.0/21
185.28.70.0/24
207.244.118.0/24
209.192.182.0/24
2604:9a00::/32

[UPDATE 11:23 EDT] As both of the core switches are reloading, our vendor advised us to downgrade the software version. We are currently in the downgrade process.

[UPDATE 11:41 EDT] One of the coreswitches has been reloaded with the other software version. The other coreswitch is still having troubles booting the older software version.

[UPDATE 12:15 EDT] Both core switches are back online on the previous software version and stable. We are still investigating intensively with our vendor. A preliminary RFO will be available from the account managers later today. Our apologies for any inconvenience caused

17 jul

[20130724] Network maintenance Virginia datacenter WDC1

Startdate: July 24th 2013 06:00:00 (EDT)
Enddate: July 24th 2013 08:00:00 (EDT)

Dear Customer,

As part of our commitment to continually improve the level of service you receive from LeaseWeb we are undertaking this maintenance.

Due to the outage we have experienced on 7/15/2013 we are planning this maintenance window.
During the maintenance window the software of both our core switches in the Virginia location will be upgraded as advised by the vendor.

We have scheduled a 2 hour maintenance window, however, the actual downtime will be minimal as we will upgrade the redundant core switches one by one.

Affected IP ranges:
108.59.0.0/20
199.58.84.0/22
199.115.112.0/21
198.7.56.0/21
192.96.200.0/21
91.242.214.0/23
162.210.192.0/21
185.28.70.0/24
207.244.118.0/24
209.192.182.0/24
2604:9a00::/32

16 jul

[20130715] Virginia network outage *SOLVED*

Time: July 15th 2013 11:10:00 (UTC)
Dear customer,

Earlier today we have experienced a 5 minute outage in our Virginia datacenter which caused connectivity loss for most of the equipment located in the LeaseWeb facility.

We could not find a root cause which caused the redundancy mechanism on our switch core to fail and cause downtime, but we got an upgrade advise from our vendor to upgrade to a newer software version which has a lot of fixes to improve the redundancy mechanism process, and will improve the re-convergence time a lot.

The maintenance to have this software installed will be announced soon, and will be updated on this page.

Please contact [email protected] if you would require a detailed RFO.

Our apologies for any inconvenience caused.

Affected IP ranges:
108.59.0.0/20
199.58.84.0/22
199.115.112.0/21
198.7.56.0/21
192.96.200.0/21
91.242.214.0/23
162.210.192.0/21
185.28.70.0/24
207.244.118.0/24
209.192.182.0/24
2604:9a00::/32

Timeframe outage
Start: 11:11 UTC
End: 11:16 UTC

Please note that this is a history outage announcement, there is no outage going on at this moment.

15 jul

SSC reinstall environment maintenance [Update]

Startdate: July 23rd 2013 07:00:00 (CEST)
Enddate: July 23rd 2013 09:00:00 (CEST)

Dear LeaseWeb Customer,

We will perform maintenance on our SSC reinstall environment.

Date: 23 July 2013
Time: 07:00-09:00 CEST GMT+2

This maintenance will impact the SSC reinstall environment which will be unavailable during the maintenance for all LeaseWeb customers.

This maintenance will not have an impact on your servers. However, you will not be able to perform the following tasks using the Self Service Center:

• Reinstall your server.

This maintenance affects reinstallations in all countries: The Netherlands, Germany and the United States of America. We will monitor our platform to ensure that everything is running properly after the maintenance.

10 jul

[Leaseweb NL] Packet loss on section of our network due to dDoS attack.

Time: July 9th 2013 17:19:00 (CEST)
Dear Sir/Madam,

We have experiencing packet loss on a small section of our network.

This was due to a dDoS attack.

We successfully blocked the attack and all website are responsive again.
If you still experience issues, please contact us at [email protected].

Affected ip ranges are as follows:

94.75.192.0/18
95.211.0.0/19
62.212.75.0/24
82.192.87.0/24
2001:1AF8:4400::/40

…………

5.61.248.0/24
46.182.176.0/22
77.81.240.0/21
87.236.96.0/21
90.156.224.0/20
94.124.56.0/21
145.255.128.0/24
193.107.18.0/24
212.7.193.128/25
212.7.192.128/25
2A01:9400:2::/48
2A02:CC8::/32
2A03:E280:0:1::/64

04 jul

[20130718] AMS Telecity 2 core router maintenance

Startdate: July 18th 2013 06:00:00 (CEST)
Enddate: July 18th 2013 07:00:00 (CEST)

Dear LeaseWeb Customer,

We will perform maintenance on our core router at Telecity 2 in Amsterdam. This will be done to upgrade the software to the version recommended by the vendor, that will guarantee continuity of our network.

Date: July 18, 2013
Time: 06:00 – 07:00 CEST (GMT+2)

During this maintenance, our second core router at EvoSwitch in Amsterdam will be fully operational and will handle the whole traffic. This maintenance will have no impact on traffic, only small latency changes while alternate routing paths are selected. The changes in network paths may be experienced across the entire LeaseWeb network.

For further information, please contact [email protected].

04 jul

100G AMS-IX optic swap @ Evoswitch [Update]

Startdate: July 4th 2013 06:00:00 (CEST)
Enddate: July 4th 2013 07:00:00 (CEST)

We will perform maintenance on our EvoSwich AMS-IX router to swap the interface module.

During this maintenance, the BGP sessions at EvoSwitch towards AMS-IX will be disabled. Traffic will be rerouted to our AMS-IX interface at TeleCity-2.

[UPDATE 04-07-2013 06:44 CEST]
Maintenance is completed successfully.