09 dec

[NL, AMS-01] Connectivity issues in part of our Network

Time: December 9, 2015 11:30:00 (CET)
Dear Customer,

Today between 11:15 and 11:35 CET we experienced connectivity issues in part of our network due to a DDoS attack. During this interval you might have noticed increased latency or/and packet loss. The attack has been mitigated and connectivity is again stable.

The following ranges might have been affected:

85.17.243.192/27
82.192.76.64/26
95.211.42.128/27
95.211.68.0/26
95.211.65.0/25
95.211.69.0/27
95.211.65.128/26
95.211.67.192/27

If you still encounter issues please contact us at [email protected].

09 dec

[NL, AMS-01] Connectivity issues [Update]

Time: December 9, 2015 06:20:00 (CET)
Dear LeaseWeb Customer,

[Update 07:05]
Due to the behaviour of the card and the impact have we decided to replace the card.
Our engineers are currently preparing a replacement card and will proceed with the replacement as soon as possible.

[Original]
Today between 06:20 and 06:25 we experienced connectivity issues in part of our network due to a linc ecard crash. Connectivity was restored, however while investigating and determining the impact we encountered another crash between 06:50 and 06:55 CET.

Affected IP ranges:

94.75.236.0/23
95.211.80.0/21
95.211.96.0/22
95.211.100.0/23
85.17.74.0/23
2001:1AF8:4060::/44
46.243.6.0/23
109.69.56.0/22
179.43.174.0/25
188.0.225.128/25
192.36.61.0/24
212.7.196.0/23
212.7.199.64/26
212.7.199.128/25
212.7.209.0
2A01:9400:1::/48
2A03:F85:2::/48

At this moment is an investigation ongoing, new information will be posed via this NOC posting.

06 dec

[NL, AMS-01] Connectivity issues in part of our network

Time: December 5, 2015 20:33:00 (CET)
Dear Customer,

Between 20:10 and 20:20 CET we experienced connectivity issues in part of our network due to a large DDoS attack. During this interval you might have noticed increased latency or packet loss. Issue was mitigated.

Possibly affected ip ranges:
37.48.98.0/24
37.48.99.0/24
37.48.100.0/23
89.144.18.0/24
95.211.94.0/23
95.211.168.0/21
95.211.184.0/21
95.211.192.0/21
95.211.224.0/21
95.211.240.0/22
103.41.176.0/23
179.43.174.128/26
185.17.184.0/23
185.17.186.0/23
191.101.16.0/24
193.151.91.0/24
2001:1af8:4020::/44

04 dec

[DE, FRA-10] Connectivity issues in part of our Network [Update]

Time: November 23, 2015 15:04:00 (CET)
Dear Leaseweb Customer,

[Final UPDATE 17:25]
Our engineers have investigated the issue further and can confirm everything is stable again.
The outage was caused due to an unexpected routing protocol drop on the router, which caused traffic to be dropped for a short while.
For the next 48 hours will we closely monitor the behaviour of the router to guarantee the stability and reliability of the network.

[UPDATE 15:20]
Currently connectivity has been restored and traffic is back to normal levels. Our engineers are working to identify the root cause of this issue.

[UPDATE 14:50]
At the moment we are experiencing connectivity issues in part of our Network. We are currently working on the issue.

The following ranges are impacted:

46.165.220.0/22
46.165.228.0/22
46.165.240.0/21
46.165.254.0/24
37.58.56.0/22
37.58.60.0/24
185.17.120.0/22
185.49.68.0/23

2A00:C98:2030::/44
2A01:9260:100::/40

Updates will follow via this NOC posting.

04 dec

[NL, AMS-01] Connectivity issues in part of our network [Update]

Time: December 4, 2015 01:47:00 (CET)
Dear Customer,

We are experiencing connectivity issues in part of our network in AMS-01. You might notice intermittent packet loss because of this.We are still investigating the cause and we will update the NOC posting as soon as we have more details.

[UPDATE 02:11]
We have found the issue, it was a memory problem on one of the distribution routers. We have fixed it and connectivity should be fully restored.

Affected ranges:

5.79.80.0/21
46.243.0.0/22
46.243.4.0/23
62.212.72.0/23
62.212.82.0/23
62.212.85.0/24
62.212.88.0/23
82.192.78.0/24
82.192.80.0/23
82.192.84.0/23
82.192.90.0/23
82.192.94.0/23
83.149.70.0/23
83.149.124.0/22
85.17.24.0/21
85.17.72.0/23
85.17.90.0/24
85.17.132.0/24
85.17.136.0/24
85.17.147.0/24
85.17.172.0/24
85.17.190.0/23
95.211.88.0/22
95.211.128.0/22
95.211.132.0/23
95.211.136.0/21
95.211.144.0/20
95.211.200.0/21
146.185.0.0/23
179.43.173.128/25
190.105.0.0/24
212.7.204.192/26
212.7.206.128/27
212.7.212.0/23
216.59.55.0/24
216.59.59.0/24
216.59.60.0/24
185.72.178.0/23
185.108.84.0/23
2001:1af8:4010::/44
2001:1AF8:4600::/40
2A04:88C0::/48

If you still encounter issues please contact us at [email protected].

29 nov

[NL, AMS-01] Connectivity issues in part of our network

Time: November 29, 2015 16:49:00 (CET)
Dear Customer,

Between 16:10 and 16:20 CET we experienced connectivity issues in part of our network due to a large DDoS attack. During this interval you might have noticed increased latency and packet loss.

Affected ranges:

46.243.6.0/23
85.17.74.0/23
94.75.236.0/23
95.211.80.0/21
95.211.96.0/22
95.211.100.0/23
109.69.56.0/22
179.43.174.0/25
188.0.225.128/25
192.36.61.0/24
212.7.196.0/23
212.7.199.64/26
212.7.199.128/25
212.7.209.0/24
2A01:9400:1::/48
2A03:F85:2::/48
2001:1AF8:4060::/44

If you still encounter issues please contact us at [email protected].

29 nov

[NL, AMS-01] Connectivity issues in part of our Volume Network [Update]

Time: November 27, 2015 20:53:00 (CET)
Dear Customer,

[Update 21:05]
Connectivity has been restored.
We will investigate further to determine root cause analysis and appropriate action will be planned.

We are experiencing issues with the stability of one of our routers.
We are investigating and will take appropriate action as soon as possible. All customers behind the specific network segment are currently down.

Affected ip ranges:

95.211.60.0/23
95.211.105.0/24
94.75.218.0/23
94.75.232.0/24
94.75.250.0/24
82.150.152.0/25
82.150.152.192/26
179.43.173.0/25
188.0.224.0/26
188.0.224.64/27
194.0.5.0/24
194.0.28.0/22
212.7.205.0/24
212.7.207.0/24
212.7.210.0/24
2001:1AF8:4050::/44

23 nov

[DE, FRA-10] Connectivity issues in part of our Network

Time: November 23, 2015 15:04:00 (CET)
Dear Leaseweb Customer,

At the moment we are experiencing connectivity issues in part of our Network. We are currently working on the issue.

The following ranges are impacted:

46.165.220.0/22
46.165.228.0/22
46.165.240.0/21
46.165.254.0/24
37.58.56.0/22
37.58.60.0/24
185.17.120.0/22
185.49.68.0/23

2A00:C98:2030::/44
2A01:9260:100::/40

Updates will follow via this NOC posting.

20 nov

[NL, AMS-01] Connectivity issues in part of our Network [Update]

Time: November 19, 2015 16:22:00 (CET)
Dear Leaseweb Customer,

At the moment we are experiencing connectivity issues in part of our Network. We are currently working on the issue. Updates will follow via this NOC posting.

[UPDATE 17:15]
We have found the issue, one of the core routers was black-holing traffic. We restored traffic levels to normal and we are investigating further.

If you still experience issues, please don't hesitate to contact us.

[UPDATE 20th Nov 11:30]
After investigating further, we found that a change we made on one of the core routers triggered the traffic to switch from the PNI sessions to carrier links. The amount of traffic that was switched caused a few links to be congested, so customers could have experienced latency and packet loss.
We are in the process of implementing egress prefix filters to avoid incidents like this.