Time: December 18th 2012 12:20:00
Dear Customer,

Please be informed that we are experiencing issues in one of our routers in Wuppertal.

We are still investigating the cause of the issue. Please stand-by for updates.

Kind regards,

LeaseWeb NOC

[Update @ 12:30]
Router is back on-line and was caused by a fiber outage. This fiber was down on one end and therefore black-holing traffic.

Startdate: December 17th 2012 23:00:00 (CET)
Enddate: December 20th 2012 07:00:00 (CET)

[UPDATE 12-18-2012 04H00]
Maintenance has been finished for the first nightly move. In case your servers is not reachable, please contact support via [email protected]

[ORIGINAL MAINTENANCE ANNOUNCEMENT]

Dear LeaseWeb customer,

To ensure we can provide you with the highest quality infrastructure and services, we recently announced the migration of all U.S. racks and servers to a new state-of-the-art facility within our U.S. datacenter in Virginia (please go to <a href="http://leasewebnoc.com/en/maintenance/us-important-maintenance-announcement">HERE</a> for the full announcement). During this relocation, we will perform the following activities:
• Disconnect all racks from power sources and network uplinks
• Move complete racks from their current location to their new location
• Reconnect the racks to new power and network infrastructure
• Bring all servers back online

The relocation will take place between 17 – 20 December 2012, from 11:00 P.M. – 7:00 A.M. EST.
Below you will find the schedule for the relocation, based on IP ranges. We kindly request all customers to turn off their affected servers before the relocation. <b><font color=red>We will start the relocation at 12:00A.M. and expect to be finished around 03:00AM. </font></b>

<u>17 – 18 December (11:00 PM 7:00 AM)</u>

108.59.0.0 – 108.59.0.255
108.59.1.0 – 108.59.1.63
108.59.1.128 – 108.59.1.159
108.59.1.176 – 108.59.1.255
108.59.10.0 – 108.59.10.63
108.59.10.112 – 108.59.10.127
108.59.12.128 – 108.59.12.255
108.59.2.192 – 108.59.2.255
108.59.2.32 – 108.59.2.127
108.59.3.0 – 108.59.3.127
108.59.3.192 – 108.59.3.223
108.59.4.0 – 108.59.4.31
108.59.4.64 – 108.59.4.127
108.59.5.0 – 108.59.5.255
108.59.6.128 – 108.59.6.159
108.59.6.192 – 108.59.6.255
108.59.6.64 – 108.59.6.95
108.59.7.240 – 108.59.7.255
108.59.8.128 – 108.59.8.191
108.59.9.0 – 108.59.9.159
108.59.9.224 – 108.59.9.255
198.7.56.0 – 198.7.56.63
<b>198.7.58.0 – 198.7.58.63 (NEW)</b>
198.7.60.64 – 198.7.60.159
199.115.112.192 – 199.115.112.255
199.115.112.64 – 199.115.112.127
199.115.113.160 – 199.115.113.191
199.115.114.160 – 199.115.114.255
199.115.115.128 – 199.115.115.255
199.115.116.0 – 199.115.116.31
199.115.118.0 – 199.115.118.63
199.115.118.128 – 199.115.118.159
199.115.119.128 – 199.115.119.191
199.115.119.64 – 199.115.119.95
199.58.84.64 – 199.58.84.127
199.58.85.0 – 199.58.85.31
199.58.86.192 – 199.58.86.255
199.58.87.0 – 199.58.87.63
199.58.87.160 – 199.58.87.255

<u>18 – 19 December (11:00 PM 7:00 AM)</u>

108.59.1.64 – 108.59.1.95
108.59.10.128 – 108.59.10.255
108.59.10.64 – 108.59.10.103
108.59.11.192 – 108.59.11.255
108.59.11.64 – 108.59.11.127
108.59.13.0 – 108.59.13.63
108.59.13.192 – 108.59.13.255
108.59.14.64 – 108.59.14.255
108.59.2.0 – 108.59.2.31
108.59.2.128 – 108.59.2.191
108.59.3.128 – 108.59.3.191
108.59.3.224 – 108.59.3.255
108.59.4.160 – 108.59.4.223
108.59.4.32 – 108.59.4.63
108.59.7.32 – 108.59.7.223
108.59.8.192 – 108.59.8.255
108.59.8.64 – 108.59.8.127
108.59.9.160 – 108.59.9.223
198.7.56.192 – 198.7.56.207
198.7.60.0 – 198.7.60.31
<b>198.7.60.150 -198.7.60.191 (NEW)</b>
198.7.60.192 – 198.7.60.255
198.7.60.48 – 198.7.60.63
198.7.61.0 – 198.7.61.63
199.115.112.32 – 199.115.112.47
199.115.113.128 – 199.115.113.159
199.115.113.64 – 199.115.113.95
199.115.114.0 – 199.115.114.31
199.115.114.128 – 199.115.114.143
199.115.115.0 – 199.115.115.63
199.115.116.192 – 199.115.116.255
199.115.116.32 – 199.115.116.127
199.115.117.0 – 199.115.117.63
199.115.117.128 – 199.115.117.191
199.115.118.160 – 199.115.118.191
199.115.118.80 – 199.115.118.83
199.115.119.0 – 199.115.119.63
199.115.119.224 – 199.115.119.255
199.115.119.96 – 199.115.119.127
199.58.85.32 – 199.58.85.127
199.58.87.64 – 199.58.87.159

<u>20 – 21 December (11:00 PM 7:00 AM)</u>

108.59.1.96 – 108.59.1.127
108.59.11.0 – 108.59.11.63
108.59.11.128 – 108.59.11.191
108.59.12.0 – 108.59.12.127
108.59.13.64 – 108.59.13.191
108.59.14.0 – 108.59.14.63
108.59.4.128 – 108.59.4.159
108.59.6.0 – 108.59.6.63
108.59.6.160 – 108.59.6.191
108.59.6.96 – 108.59.6.127
108.59.7.0 – 108.59.7.31
108.59.8.0 – 108.59.8.63
<b>198.7.56.96 – 198.7.56.127 (NEW)</b>
198.7.56.128 – 198.7.56.191
198.7.56.224 – 198.7.56.255
<b>198.7.61.96 – 198.7.61.111 (NEW)</b>
199.115.112.0 – 199.115.112.31
199.115.112.128 – 199.115.112.191
199.115.113.0 – 199.115.113.63
199.115.113.192 – 199.115.113.255
199.115.113.96 – 199.115.113.127
199.115.114.32 – 199.115.114.127
199.115.115.64 – 199.115.115.127
199.115.116.128 – 199.115.116.191
199.115.117.192 – 199.115.117.255
199.115.117.64 – 199.115.117.127
199.115.118.192 – 199.115.118.255
199.115.118.64 – 199.115.118.79
199.115.118.96 – 199.115.118.127
199.115.119.192 – 199.115.119.223
199.58.84.0 – 199.58.84.63
199.58.84.128 – 199.58.84.255
199.58.85.128 – 199.58.85.255
199.58.86.0 – 199.58.86.127

<b>Expected downtime</b>
While we will minimize any downtime as much as possible, we expect affected servers to be offline for about 30 – 60 minutes.

<b>After the maintenance</b>
After a server has been relocated, on-site engineers will check if it restarted correctly. An End-of-Maintenance Notification will be sent once a server is back online.

<b>Please contact LeaseWeb support in case of any issues</b>
LeaseWeb will of course perform the relocation with the greatest care. However, should you receive an End-of-Maintenance Notification while your server is not yet online, please notify LeaseWeb Support by replying to the notification. Alternatively, you can email any issues to [email protected].

We apologize for any inconvenience caused and thank you for your understanding.

Startdate: December 17th 2012 06:00:00 (CET)
Enddate: December 17th 2012 08:00:00 (CET)

Dear Sir/Madam,

In order to further increase the stability of the network in our Wuppertal datacenter, we will migrate some hosts from router 1 to router 2. This will be done in order to better balance the load on each router. The maintenance will involve only migrating the switch uplinks/rerouting and will not involve any physical movement/power cycle of your hosts.

In case the maintenance does not complete on Dec 17, we will continue with the move on Dec 18 between 06:00-08:00 GMT+1.

Should you experience any connectivity issues once the maintenance is over, please contact support.

Time: December 17th 2012 16:16:00
Dear Sir/Madam,

We experienced a line-card crash on one of our routers. As one of the router uplinks was connected to this blade, other uplinks are now congested and customers will experience degraded performance.
Following ip space if unreachable
95.211.90.192/26
82.192.78.0/27
95.211.90.128/26
Rest of the ip space will experience degraded performance.
We will update the announcement with the rest of the affected ip space shortly.

[Update @16:30]
Blade was replaced and connectivity has been restored.

[Final update @16:45]

Incident is now closed.
Degraded performance could have been experienced by hosts in the below ranges
62.212.71.0/24
82.192.78.0/24
85.17.72.0/23
85.17.90.0/23
85.17.132.0/24
85.17.136.0/24
85.17.147.0/24
85.17.153.0/24
85.17.172.0/24
85.17.190.0/23
85.17.230.0/24
95.211.88.0/23
95.211.90.0/23
146.185.210.0/23
190.105.193.0/24

Time: November 28th 2012 17:18:00
Dear Sir/Madam,

We are seeing degraded performance most probably caused by a failing optic for servers using ip ranges
95.211.136.64/26
95.211.156.0/27

We will proceed to changing optics and testing the connection. Up to 15 min outage could be experienced.

[Update 18:15]
Maintenance completed successfully.
Should you experience any issues, please let us know.

Time: November 25th 2012 17:14:00
Dear Customers,

We have had a router reload today at Sunday, 25 November 2012, 10:40EST

Region: Washington Data Center

We are investigating the issue and will keep you updated.

Affected IP address ranges are:
108.59.3.0/23
108.59.7.0/24
108.59.8.0/22
108.59.13.0/23
198.7.60.0/23
199.58.84.0/22
199.115.114.0/23
199.115.116.0/22
2604:9a00:2010::/44

Update: Sunday, 25 November 2012, 11:35EST
Router is stable, all connection are up.
We do not have an reason why the router rebooted twice after in such a short time.
We will further our investigation and give an update later

Update: Sunday, 25 November 2012, 12:34:29 EST
Router rebooted again, we suspect software issue and will upgrade the IOS software to a newer version.
This will be done within the next hour.

Update: Sunday, 25 November 2012, 13:11EST
Router rebooted during the upload of the software, it is back online again. We are continuing to upload the new software. When this is completed we will reload the router with new version.
We still have no known reason for the reloads.

Update: Sunday, 25 November 2012, 15:33 EST
We have loaded the new software and it is ready for the next reload, we do not want to reload it at this moment as it has been stable for the last 2 1/2 hours. We will take action to replace the Supervisor Engine next time if the router reboots again on its self, hopefully not.

Update: Sunday, 25 November 2012, 22:40 EST
Router reloaded it's self again, now the router is using the newest version IOS, we will closely monitor the router over the next hour before we take action to replace the Supervisor Engine. As of to now, we do not have the cause of the issue for the reloading.

Sunday, 25 November 2012, 23:28 EST
We have decided to replace the Supervisor Engine because the router is not stable, a Technical Support Engineer is on his way to the data center and we will commence to change the blade within the next hour. We will make regular updates.

Sorry for any inconvenience caused.

Monday, 26 November 2012, 00:02 EST
Engineer is on-site and we are preparing the new Supervisor Engine for the replacement.

Monday, 26 November 2012, 00:21 EST
We are replacing the Supervisor Engine blade now at this moment.

Update: Monday, 26 November 2012, 00:39 EST
Router Supervisor Engine has been replaced and is back up and running.
We will keep you updated and we will closely monitor the router in the next few hours.
Sorry for any inconvenience caused.

Time: November 25th 2012 17:14:00
Dear Customers,

We have had a router reload today at Sunday, 25 November 2012, 10:40EST

Region: Washington Data Center

We are investigating the issue and will keep you updated.

Affected IP address ranges are:
108.59.3.0/23
108.59.7.0/24
108.59.8.0/22
108.59.13.0/23
198.7.60.0/23
199.58.84.0/22
199.115.114.0/23
199.115.116.0/22
2604:9a00:2010::/44

Update: Sunday, 25 November 2012, 11:35EST
Router is stable, all connection are up.
We do not have an reason why the router rebooted twice after in such a short time.
We will further our investigation and give an update later

Update: Sunday, 25 November 2012, 12:34:29 EST
Router rebooted again, we suspect software issue and will upgrade the IOS software to a newer version.
This will be done within the next hour.

Update: Sunday, 25 November 2012, 13:11EST
Router rebooted during the upload of the software, it is back online again. We are continuing to upload the new software. When this is completed we will reload the router with new version.
We still have no known reason for the reloads.

Update: Sunday, 25 November 2012, 15:33 EST
We have loaded the new software and it is ready for the next reload, we do not want to reload it at this moment as it has been stable for the last 2 1/2 hours. We will take action to replace the Supervisor Engine next time if the router reboots again on its self, hopefully not.

Update: Sunday, 25 November 2012, 22:40 EST
Router reloaded it's self again, now the router is using the newest version IOS, we will closely monitor the router over the next hour before we take action to replace the Supervisor Engine. As of to now, we do not have the cause of the issue for the reloading.

Sorry for any inconvenience caused.

Time: November 25th 2012 17:14:00
Dear Customers,

We have had a router reload today at Sunday, 25 November 2012, 10:40EST

Region: Washington Data Center

We are investigating the issue and will keep you updated.

Affected IP address ranges are:
108.59.3.0/23
108.59.7.0/24
108.59.8.0/22
108.59.13.0/23
198.7.60.0/23
199.58.84.0/22
199.115.114.0/23
199.115.116.0/22
2604:9a00:2010::/44

Update: Sunday, 25 November 2012, 11:35EST
Router is stable, all connection are up.
We do not have an reason why the router rebooted twice after in such a short time.
We will further our investigation and give an update later

Sorry for any inconvenience caused.

Time: November 24th 2012 01:13:00
Dear Customers,

We are having issues with a blade in our standby router.

One of the router blade is defect and we will replace it now.

There should be no impact , but because of activities there might may be a brief inconvenience.
We expect the maintenance to take up to 30 minutes.

If you have any issues, please contact us at [email protected]

Update: 03:00
Blade has been replaced and ports are been reconnected at this moment.
Next update will confirm all aplinks connected.

Update: 03:20
All uplinks have been reconnected and all is connections are back to normal.
Sorry for any inconvenience caused.