30 nov

[201111230] US DC High Volume Network Router issue [UPDATE] [Update]

Time: November 30th 2011 22:26:00(CEST)
Dear Customer,



At the moment we experience some troubles in (part of) our US network.



We are doing our utmost to solve this issue a.s.a.p.

As soon as we have more information available, we will publish it here.



Please be patient while we are working on this problem.



Regards,

LeaseWeb BV



Affected IP ranges are:



108.59.3.0/24

108.59.4.0/24

108.59.7.0/24

108.59.8.0/22

108.59.13.0/24

108.59.14.0/24



We will update the status regulary, you cand find the updates at the below mentioned URL:



http://leasewebnoc.com/status.php?i=710



[UPDATE 22:45] The issue was caused by a DoS, the attack is filtered. Router is back in normal state

28 nov

DE-CIX connection maintenance [emergency]

Startdate: November 29th 2011 07:00:00(CEST)
Enddate: November 29th 2011 08:30:00(CEST)

Dear Customer,



As a part of our network infrastructure improvement plan we are moving

one of our DE-CIX (Internet exchange in Frankfurt, Germany) connections

to another router and increasing its capacity to 50 Gbps.

We had to coordinate several third parties to do this maintenance thus

we couldn’t announce it earlier, we are sorry for any inconvenience.



This maintenance will enable our servers in German datacenters to use

the DE-CIX connection and will improve the connectivity between our

Dutch and German networks increasing available bandwidth and redundancy.

Our total capacity to DE-CIX will also increase to 80 Gbps.



There is no impact expected, however temporarily you may see different

network paths for your packets which might result in slightly higher

latency.



The maintenance will take place at Tuesday 29th Nov 2011 between 7:00

and 8:30 CET (UTC 1).



Kind regards,



LeaseWeb NOC

26 nov

[20111126] Evoswitch Router outage [Update]

Time: November 26th 2011 19:28:00(CEST)
Dear Customer,



At the moment we experience some troubles in (part of) our network.



We are doing our utmost to solve this issue a.s.a.p.

As soon as we have more information available, we will publish it here.



Please be patient while we are working on this problem.



Regards,

LeaseWeb BV





Affected IP Ranges are:



62.212.64.0/22

62.212.68.0/23

62.212.70.0/24

85.17.128.0/17

95.211.32.0/21

82.192.79.0/24

85.17.87.0/24

85.17.92.0/22



[Update]

The issue is resolved. We are still investigating by which cause the outage has happend



21 nov

[20111121] Session flap core router [Update]

Time: November 21st 2011 14:45:00(CEST)
Dear Customer,

At the moment we experience some troubles in (part of) our network.
All sessions flapped on one of our core routers. Our routers changed their routing paths to our other core router. During switchover there might have been temporarily issues, like packet loss or sub optimal paths.

We are still investigating what caused the sessions to flap.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 15:00] The situation is stable again, however we are still investigating the root cause of the session flap.

Sehr geehrte Damen und Herren,

zur Zeit haben wir Probleme in Teilen unseres Netzwerkes.
Alle BGP Sessions auf einem Core Router sind ausgefallen. Waehrend dieses Ausfalls wurde der Datenverkehr von einem weiteren Core Router verarbeitet. Zu diesem Zeitpunkt gab es ggf. kurzzeitige Packetverluste oder erhoehte Latenzen.

Zur Zeit suchen wir nach der Ursache des Problems.

[UPDATE 15:00] Alle BGP Sessions laufen wieder, die Ursachen Analyse lauft weiterhin

18 nov

[20111117] Peering and high volume routers upgrade [Update]

Startdate: November 17th 2011 06:00:00(CEST)
Enddate: November 17th 2011 07:00:00(CEST)

Dear customer,



During this maintenance window we will upgrade the software on our high volume routers at Evoswitch in Haarlem. The maintenance will take place between 6:00 and 7:00 CET (UTC+1) on the 17th of November 2011. You may see up to 15-20 minutes downtime.



Within the same maintenance window we will also upgrade the software of our peering router in Frankfurt, Germany. There should be no serious impact, however it is possible you may notice temporarily higher latency or small packet loss when traffic is shifted towards alternate paths.



Affected IP ranges for high volume routers upgrade:

46.243.0.0/22

62.212.82.0/24

62.212.85.0/24

62.212.88.0/23

82.192.84.0/24

82.192.94.0/23

83.149.70.0/23

83.149.124.0/22

85.17.24.0/21

88.208.52.0/24

88.208.57.0/24

88.208.58.0/23

95.211.136.0/21

95.211.144.0/20

212.7.212.0/23

2001:1AF8:4000::/40





Kind regards,



Leaseweb NOC





[UPDATE 6:25]

The maintenance has been completed successfully, all systems are back again.

18 nov

[20111118] Powerloft fiber maintenance [Update]

Startdate: November 18th 2011 03:00:00(CEST)
Enddate: November 18th 2011 08:00:00(CEST)

Dear customer,



One of our dark fiber suppliers at location Powerloft in Manassas, VA. has announced a maintenance on several dark fiber paths. During this maintenance we will reroute traffic to different links.



No customer impact is expected, however you may see short latency increase or small packet loss.



The timezone for given timeframe is CET.



LeaseWeb – NOC



[UPDATE 06:40] The maintenance has been successfully performed.

18 nov

CL-WWW load balancer maintenance [Update]

Startdate: November 14th 2011 07:00:00(CEST)
Enddate: November 14th 2011 07:15:00(CEST)

Dear customer,



During this maintenance we will upgrade the firmware image of the current load balancer used for the CL-WWW platform.



Expected downtime for this maintenance is 10 minutes



The affected IP addresses are below:

– 85.17.150.98

– 85.17.150.100



LeaseWeb – NOC



[UPDATE] @ 07:15



Maintenance completed without any problems.

18 nov

Maintenance telephone exchange

Startdate: November 14th 2011 20:00:00(CEST)
Enddate: November 14th 2011 20:30:00(CEST)

Dear customer,



This evening we will perform maintenance on our main telephone exchange in the Netherlands.

The telephone service will be interrupted between 20:00 and 20:30 CEST.



For emergency you can contact us on [email protected]





Kind regards,



Leaseweb – NOC

18 nov

[20111116] EvoSwitch Router component replacement [Update]

Startdate: November 16th 2011 06:00:00(CEST)
Enddate: November 16th 2011 07:00:00(CEST)

Dear customer,



We will replace a hardware component of the high volume router that had issues on the 7th (http://leasewebnoc.com/status.php?i=698) of november. We will perform this replacement to maintain overall connectivity performance. With this maintenance the router will be reloaded and customers will notice downtime.



The following ranges are impacted:

– 94.75.220.0/22

– 94.75.216.0/23

– 94.75.229.0/24

– 94.75.240.0/24

– 94.75.253.0/24

– 94.75.234.0/24

– 62.212.84.0/24

– 95.211.92.0/23

– 85.17.84.0/24

– 212.7.192.0/22

– 2001:1AF8:4030::/44

– 2A01:9400:3::/48



The maintenance will take place on 16th of november between 06:00 CET and 07:00 CET (UTC +1)



Kind regards,



LeaseWeb NOC



[UPDATE @ 06:33]

Replacement went successfull and router is booted up. At the moment all ports are being checked for status.



[UPDATE @ 6:48]

Seems like there is an issue with the software. Therefore a reload is requested to downgrade back to the previous IOS.



[UPDATE @ 7:00]

Everything is recovered and back to normal. If you encounter any issues, please contact technical support trough [email protected].

18 nov

[level 3] emergency maintenance

Startdate: November 11th 2011 00:00:00(CEST)
Enddate: November 11th 2011 06:00:00(CEST)

Dear Customer,



Because of this issue: http://leasewebnoc.com/status.php?i=699



Our supplier announced an emergency maintenance to upgrade equipment.



During this maintenance there can be a brief temporarily increase in latency or minor packet-loss while traffic uses other paths.



This maintenace Will start at:

11-11-2011 0:00 CET



This maintenance ends at:

11-11-2011 06:00 CET





Kind regards,



LeaseWeb NOC