Time: November 16, 2015 06:18:00 (CET)
Dear Leaseweb Customer,

[UPDATE 07:00 CET]
We had an unexpected linecard issue in one of our core routers in our AMS-01 data center and we were forced to reload the linecard. After reloading connectivity seems to have been restored. We are still working and monitoring the behavior in order to ensure that the network is stable again.

At the moment we are experiencing connectivity issues in part of our Network. We are currently working on the issue.

Updates will follow via this NOC posting.

Time: November 16, 2015 06:18:00 (CET)
Dear Leaseweb Customer,

At the moment we are experiencing connectivity issues in part of our Network in our AMS-12 data center. We are currently working on the issue. Updates will follow via this NOC posting.

Time: November 15, 2015 01:00:00 (CET)
Dear LeaseWeb Customer,

We have experienced connectivity issues on a section of our network due to a DDoS attack in our FRA-10 data center. Between approximately 00:40 and 1:20 CET you might have experienced packet loss or/and higher latency. The attack has been mitigated and connectivity is now stable.

The following network prefixes might have been affected:

46.165.220.0/22
46.165.228.0/22
46.165.240.0/21
46.165.254.0/24
37.58.56.0/22
37.58.60.0/24
185.17.120.0/22
185.49.68.0/23

2A00:C98:2030::/44
2A01:9260:100::/40

Should you still experience issues, please open a ticket with support.

Time: November 8, 2015 16:24:00 (CET)
Dear Leaseweb Customer,

Today, between approximately 16:10 and 16:15 CEST, you might have experienced short periods of service unavailability,packet loss or/and higher latency due to a DDoS attack in part of our premium network in AMS-01. The DDoS attack has been mitigated and connectivity is stable.

Affected ranges:
31.186.168.0/23
37.75.152.0/21
41.191.215.0/24
62.212.64.0/22
62.212.67.0/24
62.212.68.0/22
62.212.74.0/24
62.212.91.0/24
62.212.92.0/24
80.65.32.0/20
82.192.67.0/24
82.192.69.0/24
82.192.76.0/23
82.192.79.0/24
82.192.86.0/24
82.192.92.0/23
85.17.68.0/22
85.17.87.0/24
85.17.92.0/22
85.17.112.0/23
85.17.128.0/17
85.234.224.0/19
91.224.38.0/23
93.190.220.0/22
94.75.226.0/26
95.211.164.0/22
95.211.32.0/20
95.211.48.0/21
95.211.56.0/23
95.211.58.0/23
95.211.62.0/23
95.211.64.0/21
109.120.175.0/24
159.255.192.0/22
167.88.48.0/20
178.18.20.0/23
178.18.22.0/24
178.250.16.0/24
178.250.18.0/24
185.16.84.0/24
185.24.240.0/24
185.38.185.0/24
185.46.232.0/24
185.51.192.0/22
185.63.152.0/22
191.101.48.0/22
191.96.72.0/21
193.27.72.0/23
194.0.197.0/24
195.14.21.0/24
195.140.240.0/22
212.7.204.0/23
2001:1AF8:4100::/40
2001:1AF8:4200::/40
2001:1AF8:4300::/40
2A02:BC0::/32
2A03:7480::/44
2A02:40C0:1::/48
2a01:b2e0::/32

Time: November 3, 2015 11:37:00 (CET)
Dear customer,

@Update 17:45
Connectivity is restored and setup is back to being redundant. No further work will be performed on the setup.

@Update 17:25
While trying to restore the redundant device in an operation that should not have been impacting, the setup has become unreachable. We are actively troubleshooting to restore connectivity.

@Update 13:25
Connectivity has been restored at 13:00. An RFO will follow.

The shared firewall setup is currently down and remote access is also unavailable. Engineers have been dispatched to go on site and evaluate the situation.
Once more information is available, you will be updated.

Affected ip ranges

83.149.76.0/22
83.149.80.128/25

Time: October 13, 2015 09:00:00 (CEST)
Dear LeaseWeb customer,

[Final UPDATE 14 Oct. 18:00 CEST]

LeaseWeb engineers solved the issues and performance is now back to normal levels. Together with our storage provider we are implementing additional measures to prevent this incident from happening again.

We are also still working on adding more capacity to the platform.

If you need further assistance, please contact [email protected] and we will give you the help you need.

Our apologies for any inconvenience caused.

[UPDATE 14 Oct. 17:25 CEST]

Together with our storage vendor we did a forced failover to the second storage head.
We currently see the platform load going down to normal levels, if this continues we expect the platform to stabilise in the next 30 minutes.

We will update this post as soon as new information becomes available.

[UPDATE 14 Oct. 16:40 CEST]

Unfortunately the hardware expansion did not yield the expected results. LeaseWeb engineers are working closely with our storage vendor to resolve this issue.

Please be assured that we are doing our utmost to fix this problem as soon as possible, but despite our best efforts we have not been able to get performance back on the level that we normally offer.

We will update you again once we have more information.

[UPDATE 14 Oct. 11:10 CEST]
LeaseWeb engineers located the apparent cause of the performance issues and a hardware expansion is now being prepared in an attempt to restore the performance of your Public Virtual Server back to acceptable levels. We expect this hardware expansion to be completed in the next hours, so we can start optimizing the load on the platform. Another update of this NOC post can be expected this afternoon.

[ORIGINAL MESSAGE]
Due to high load on our Public Virtual Server platform in Germany, you might experience degraded performance of your Virtual Server. Together with the supplier, LeaseWeb engineers are investigating the cause of this issue.

Please be assured that we will do our utmost to keep the impact as low as possible, but despite our efforts you might experience performance issues on your services.

We will update you again once we have more information.

Time: October 15, 2015 09:00:00 (CEST)
Dear LeaseWeb customer,

[FINAL UPDATE]
LeaseWeb engineers worked throughout the day to restore the datagraph functionality in the LeaseWeb Customer Portal. Since 17:30 CEST all datagraphs should be available again, but a gap in the graphs between 15.00 and 17.15 CEST might be noticeable for customers affected by this issue.

[ORIGINAL MESSAGE]
We are currently experiencing a failure on one of our datagraph servers. Our engineers are investigating the cause of this issue.

As result of this failure, a part of the datagraphs is not correctly displayed in the LeaseWeb Customer Portal. Please be assured that we will do our utmost to resolve this issue as soon as possible.

We will update you again once we have more information.

Time: October 22, 2015 04:55:00 (UTC)
Dear customer,

During the upgrade project of our HKG-10 location we suffered from a power outage in our core infrastructure.

Impacted ranges:
43.249.36.0/23
37.218.240.0/24

[05:07 UTC] All connectivity has been restored, at this moment we are still investigating the cause of this outage.

Time: October 16, 2015 17:30:00 (CEST)
Dear Leaseweb Customer,

[UPDATE 17:50 CEST]
We discovered two line cards crashed and we were forced to reboot them. After we successfully rebooted them all traffic has been restored. We will monitor the situation further and will update you if the issue reoccurs.

If you notice any issues, please open a ticket via the LeaseWeb Self Service Center, or contact us directly at [email protected].

——————-

We are currently observing an issue in our AMS-01 network. We are investigating the cause of the issue. More updates will follow soon through this NOC posting.

Below you will find a detailed breakdown of the IP ranges affected:

82.150.152.0/25
82.150.152.192/26
95.211.60.0/23
95.211.105.0/24
94.75.218.0/23
94.75.232.0/24
94.75.250.0/24
179.43.173.0/25
188.0.224.0/26
188.0.224.64/27
194.0.5.0/24
194.0.28.0/22
212.7.205.0/24
212.7.207.0/24
212.7.210.0/24

2001:1AF8:4050::/44