12 aug

Storing sites en e-mail bij Mijn Domein (opgelost)

Hosting provider Mijn Domein kampt momenteel met een storing op haar netwerk, waardoor mail en websites niet bereikbaar zijn. Sites zijn vanaf sommige providers wel bereikbaar en andere verbindingen weer niet.

Update 10:32
Sites zijn vanaf UPC en KPN verbindingen grotendeels wel bereikbaar. Lijkt op peering issues tussen Mijn Domein en verschillende backbones. Vanaf Ziggo is Mijn Domein compleet niet bereikbaar volgens meerdere bevestigingen.

Update 10:48
Mijn Domein bevestigt routing issue. Grote up-link provider ervaart problemen.

Update 11:00
De verbindingen lijken hersteld. Mail en sites moeten weer bereikbaar zijn. De storing werd veroorzaakt door een storing bij de netwerkleverancier Proserve, welke op haar beurt de storing had te danken aan één van haar netwerkproviders. Er is spoedonderhoud gepleegd aan hun Cisco 7609 routers in het EUnetworks datacentrum in Amsterdam, welke is afgerond. Er wordt nog spoedonderhoud gepleegd aan apparatuur op de andere locatie Dataplace.

cisco 7609 router

de Cisco 7609 router

13 apr

Landelijk DSL storing (opgelost)

Momenteel is het niet mogelijk om internet verkeer te gebruiken. De netwerk leverancier is hard aan het werk om het op te lossen.

Major Network Outage: Consumer Internet (DSL) services have been interrupted around the POP location: Telecity.
AXSM 9 card in the Cisco MGX (ASDTEL1MGX02)

(met dank aan Joeri voor de melding)

15 nov

Core router replacement (Telecity 2)

Startdate: December 1st 2010 03:30:00(CEST)
Enddate: December 1st 2010 09:00:00(CEST)

Dear Customer,

During this maintenance window we will replace our current Cisco core router with a Juniper core router.

At 03:00 we will reroute all traffic from the Cisco at Telecity to the core router at EvoSwitch.

Around 08:00 the new router should be up and running, and traffic will be routed back to Telecity.

The should be no traffic interruption, in case you do see issues please contact our support email.

Kind regards,

LeaseWeb NOC

28 aug

Core router software update (CRS-EVO)

Startdate: September 1st 2010 06:00:00(CEST)
Enddate: September 1st 2010 07:00:00(CEST)

Dear Customer,

The outage our network experienced Friday 27-08-2010 (http://noc.leaseweb.com/status.php?i=557) was caused by a bug in the IOS-XR software which runs on our Cisco CRS-1 core routers.

More information about the bug can be read on Cisco’s page concerning this issue.

During this maintenance window we will install the update on our CRS-1 router located in EvoSwitch.
The router will be rebooted.
Before the maintenance starts we will reroute all traffic to the core router located in Telecity-2.
Customer traffic loss is not expected.

Kind regards,

LeaseWeb NOC

21 jun

Nieuwe software voor Cisco EPC3000 modems

Vanaf 21 juni is er nieuwe software beschikbaar voor de Cisco modems van het type EPC3000 (het typenummer staat op de voorkant van de modem).

De software wordt automatisch door de modem gedownload en geïnstalleerd. Nadat de nieuwe software geïnstalleerd is moet de modem gereset worden voordat het weer zal werken. Resetten doet u door de modem minimaal 30 seconden van de stroom te halen.

14 apr

AMS-IX connection maintenance

Startdate: April 22nd 2010 07:00:00(CEST)
Enddate: April 22nd 2010 08:00:00(CEST)

Dear Customer,

During this maintenance we will migrate to a bigger chassis the second side of the DWDM system which we use to connect our Cisco CRS-1 router at Evoswitch to AMS-IX.
This is one of two connections we have to AMS-IX, so no impact is expected as our whole AMS-IX traffic will be rerouted to our second Cisco CRS-1 router at Telecity 2. The expected downtime for our AMS-IX connection at Evoswitch is about 20 minutes within the maintenance window.
This is the last of three needed maintenances. After this one we will be able to finally increase our AMS-IX capacity from 120 Gbps to 140 Gbps.

Kind regards,
Leaseweb NOC

07 apr

AMS-IX connection maintenance

Startdate: April 20th 2010 07:00:00(CEST)
Enddate: April 20th 2010 08:00:00(CEST)

Dear Customer,

Due to our increasing traffic we have recently ordered two additional connections from our Cisco CRS-1 router at Evoswitch to the AMS-IX at the Nikhef datacenter in Amsterdam. Unfortunately AMS-IX switch we are connected to has no more free ports and all our AMS-IX ports at Nikhef have to be migrated to a new bigger switch.
The maintenance will affect one of two connections we have to AMS-IX, so no impact is expected as our whole AMS-IX traffic will be rerouted to our Cisco CRS-1 router at Telecity 2. The expected downtime for our AMS-IX connection at Evoswitch is about 20 minutes within the maintenance window.

The timezone is CEST (UTC+2).

Kind regards,
Leaseweb NOC

02 apr

AMS-IX connection maintenance

Startdate: April 13th 2010 07:00:00(CEST)
Enddate: April 13th 2010 08:00:00(CEST)

Dear Customer,

During this maintenance we will migrate to a bigger chassis the DWDM system which we use to connect our Cisco CRS-1 router at Evoswitch to AMS-IX.
This is one of two connections we have to AMS-IX, so no impact is expected as our whole AMS-IX traffic will be rerouted to our second Cisco CRS-1 router at Telecity 2. The expected downtime for our AMS-IX connection at Evoswitch is about 20 minutes within the maintenance window.

Kind regards,
Leaseweb NOC

06 mrt

core router crs-evo problems. [Update]

Time: March 6th 2010 22:20:00(CEST)
Expected Downtime: Unknown
Location: TeleCity / Redbus / InterXion / Schuberg Philis / Evoswitch
Reason:Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[update 22:38]
The router experienced several problems, and issued a redundancy failover to the backup route processor. The failover caused all BGP sessions to flap. Traffic flow has been restored on the router ahter the failover.
Error messages indicate Cisco support is required. We have opened a TAC case with Cisco.