Time: May 6, 2014 10:04:00 (CEST)
Dear LeaseWeb Customer,

We received a high number of tickets related to network connectivity. While we do not detect any problems inside our network we are investigating. We will update you once we have more information.

[Update 11:00]
We identified issues with our AMS-IX connection at AMS1 data center. We are working with them in solving the issue as soon as possible.

[Update 13:00]
We asked Ams-IX to force a switch-over and this seems to have solved the issue. We are jointly investigating the cause of the issue. An emergency maintenance will be scheduled for tomorrow morning for further troubleshooting.

If you still notice any issues, please open a ticket via the LeaseWeb Self Service Center, or contact us directly at [email protected].

Time: May 3, 2014 23:32:00 (CEST)
Dear Customer,

At the moment we are experiencing issues with the active router within the FRA-10 data center. During this the standby router is fully active and taking over the connectivity.

Affected prefixes:
46.165.204.0/23
46.165.216.0/22
46.165.224.0/22
46.165.236.0/22
91.109.24.0/22
2A00:C98:2200::/40
…………

185.23.16.0/23
193.104.219.0/24

Update at 23:52
The router is reloaded. After this reload services recovered and remain stable. We will monitor it for unexpected behavior. And update this post if needed. In case you have issues, please contact us via the SSC or [email protected].

Time: May 3, 2014 11:03:00 (CEST)
Dear Customer,

At the moment we are experiencing issues at our AMS-12 location. The secondary/standby switch is having hardware issues. We have taken the device out of production and are troubleshooting what is causing this. Customer might see their secondary uplink going down/up during this process, this is without impact since routing is handled by the active router.

Possible affected prefixes:
85.17.32.0/19
85.17.96.0/20
85.17.114.0/24
85.17.117.0/24
2001:1AF8:2100::/40

…………..

46.17.0.0/21
46.102.240.0/21
62.84.240.0/22
77.235.32.0/19
83.223.32.0/20
91.195.118.0/23
93.188.64.0/21
109.70.0.0/21
128.140.216.0/22
185.27.140.0/22
193.43.92.0/24
193.242.108.0/24
195.200.82.0/23
195.242.98.0/23
2A02:2968:20::/48

Update at 11:30
We have decided to reload the device. After this reload the situation returned back to normal. We will monitor it for an hour and if it remains stable start putting back production traffic. For now the situation remains the same as before therefore, only the active router is available during this. An update will be give once further steps are taken.

Update at 13:25
We have re-enabled the interfaces on the secondary/standby router and connectivity remains stable.

In case you still experience issues, please contact us via creating a ticket through the SSC or [email protected].

Time: May 2, 2014 21:21:00 (CEST)
Dear Customer,

We experienced connectivity issues via one of our providers between 18:20 and 21:00 CEST. The cause of this was an incoming attack.

Experienced issues could have been higher latency and/or packet-loss for customers located in the Netherlands using the path via Telia.

Time: April 30, 2014 13:24:00 (CEST)
Dear LeaseWeb Customer,

At this moment we are experiencing a network issue in part of our network due to an MPLS outage on our Ring-1. Updates will follow via this NOC post. We are very sorry for the inconvenience caused.

The following network prefixes have been affected:

5.61.248.0/24
37.1.144.0/24
46.182.176.0/22
62.212.75.0/24
77.81.240.0/21
82.192.87.0/24
87.236.96.0/21
90.156.224.0/20
94.75.192.0/18
94.124.56.0/21
95.211.0.0/19
145.255.128.0/21
193.107.18.0/24
212.7.193.128/25
212.7.192.128/25

2001:1AF8:4400::/40
2A01:9400:2::/48
2A02:CC8::/32
2A03:E280:0:1::/64

[UPDATE 13:45]
The faulty linecard was replaced and connectivity has been restored.
If you notice any issues, please open a ticket via the LeaseWeb Self Service Center, or contact us directly at [email protected].

Time: April 29, 2014 10:35:00 (CEST)
Dear Customer,

At the moment we are having an issue with the line-card on for the active router on the premium network. We will re-seat the line-card to resolve this. Before this we will do a fail-over to the other standby router. There could be packet-loss for a short period of time during the fail-over.

Affected prefixes:
46.165.204.0/23
46.165.216.0/22
46.165.224.0/22
46.165.236.0/22
91.109.24.0/22
2A00:C98:2200::/40
…………
185.23.16.0/23
193.104.219.0/24

Update at 11:10
The blade have been re-inserted and is back online. We are monitoring the status for some time and will put back traffic once it remains stable. Another update will follow once this is done.

Time: April 27, 2014 04:30:00 (CEST)
Dear LeaseWeb Customers,

At this moment we have a major power outage in part of our Frankfurt data center due to bad weather.

We will update you once we have more information. If you notice any issues, please open a ticket via the LeaseWeb Self Service Center, or contact us directly at [email protected].

Update 05:20
We are still waiting on the DataCenter [DC] to give us a update.
Only part of the DC is affected and the the following IP rages are affected:
78.159.126.0/25
84.16.232.0/24
178.162.171.128/25
178.162.192.0/19
212.95.36.0/24
2A00:C98:2050::/44

…………

77.223.144.0/23
185.28.69.0/24

Update 06:00
DataCenter Engineers are still inspecting the building, we cannot enter the building yet.
We have a full team waiting to turn all services up when we get the go ahead to enter the datacenter.

Update 07:00
Engineers have been granted entrance to the building, we are still waiting for the power to be turned back on.

We also have cooling issue in another part of the DC, the following IP addresses are affected:
46.165.204.0/23
46.165.216.0/22
46.165.224.0/22
46.165.236.0/22
91.109.24.0/22
2A00:C98:2200::/40

…………

185.23.16.0/23
193.104.219.0/24

Update: 08:45

Power has been returned on to the DC and we are monitoring all services as they start up.

Update: 08:50

Air-cooling has been switched back on and temperature is dropping back to normal.

Update 09:10

We have all network connectivity back to normal. If you still have issues, please open a ticket with [email protected]