08 jul

AMS-IX Telecity ports down [Update]

Time: July 8th 2009 16:23:00(CEST)
Expected Downtime: Unknown
Reason: Under investigation / Unknown
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 16:35]
The ports to the AMS-IX switches came up after 7 minutes downtime.
We are investigating the cause.
The AMS-IX NOC has also been contacted, they are investigating their side.

When we receive an update we will update this article.

05 jul

SSC not accessible [Update]

Time: July 3rd 2009 16:30:00(CEST)
*******************************

Dear Customer,

At this moment the SSC (secure.leaseweb.com) is down for maintenance

We will update this post when we have an ETR.

Regards,
LeaseWeb BV


[UPDATE]

Dear customer,

Due to a vulnerability attack and as precautionary followup measure, we will not power up the Self Service Center (SSC) for the time being. We kindly ask you to contact our support at [email protected] if you need to manage your services during that time. We will keep you posted on this site.

[UPDATE Sun Jul 5th 21:20]

On Friday we sent you a security notice on a breach of our systems. We are happy to inform you that we could contain the situation and that everything is under full control at Leaseweb.

Our precautionary measures have been lifted, so you can use the Self Service Center (SSC) again as usual. To be safe, we still advise you to change the passwords to your services as soon as possible. Events have shown that no one is safe from hacker.s attacks!

Assisted by the renowned IT Security firm FOX-IT, we have been implementing further security measures and are now investigating the complete backgrounds of the attack. Minor Leaseweb systems or parts thereof are still offline to this end but very shortly all systems will be completely back to normal at Leaseweb.

We apologize again for the inconvenience our stringent precautions may have been causing.

Best regards,
Your Leaseweb Team

04 jul

SSC not accessible [Update]

Time: July 3rd 2009 16:30:00(CEST)
*******************************

Dear Customer,

At this moment the SSC (secure.leaseweb.com) is down for maintenance

We will update this post when we have an ETR.

Regards,
LeaseWeb BV


[UPDATE]

Dear customer,

Due to a vulnerability attack and as precautionary followup measure, we will not power up the Self Service Center (SSC) for the time being. We kindly ask you to contact our support at [email protected] if you need to manage your services during that time. We will keep you posted on this site.

03 jul

SSC down for maintenance

Time: July 3rd 2009 16:30:00(CEST)
*******************************

Dear Customer,

At this moment the SSC (secure.leaseweb.com) is down for maintenance

We will update this post when we have an ETR.

Regards,
LeaseWeb BV

02 jul

Power issue Evoswitch hall3 [Update]

Time: July 2nd 2009 21:40:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch Hall3
Reason: Power failure / Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 21:55]
The power issue has (partially) affected three rows of racks.
Datacenter engineers are currently troubleshooting the problem.

[UPDATE 22:25]
The datacenter engineers reported they expect to restore power within 10 minutes.

[UPDATE 22:41]
The datacenter engineers reported a defect in a power circuit.

[UPDATE 22:46]
New ETR is 10 minutes

[UPDATE 22:58]
There are some difficulties repairing the issue
New ETR will be posted as soon as we get confirmation.

[UPDATE 23:15]
Power has been restored.
Engineers are checking all affected racks for servers that do not come back online.

[UPDATE]
Some previous incorrect posts have been editted.

02 jul

Power issue Evoswitch hall3 [Update]

Time: July 2nd 2009 21:40:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch Hall3
Reason: Power failure / Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 21:55]
The power issue has (partially) affected three rows of racks.
Datacenter engineers are currently troubleshooting the problem.

[UPDATE 22:25]
The datacenter engineers reported they expect to restore power within 10 minutes.

[UPDATE 22:41]
The datacenter engineers reported a defective power switch.
They are currently replacing it with a new one.

[UPDATE 22:46]
New ETR is 10 minutes

[UPDATE 22:58]
There are some difficulties installing the new switch.
New ETR will be posted as soon as we get confirmation.

02 jul

Power issue Evoswitch hall3 [Update]

Time: July 2nd 2009 21:40:00(CEST)
Expected Downtime: Unknown
Location: EvoSwitch Hall3
Reason: Power failure / Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV

[UPDATE 21:55]
The power issue has (partially) affected three rows of racks.
Datacenter engineers are currently troubleshooting the problem.

01 jul

Power maintenance Schuberg Philis Routers [Update]

Startdate: June 30th 2009 23:00:00(CEST)
Enddate: June 30th 2009 23:59:00(CEST)

Dear customer,

This evening we have planned power maintenance at Schuberg Philis.
Since the power feeds are redundant we don’t expect any downtime, however in case of issues a router may power cycle.

This power maintenance does not affect customer racks, only our ring devices.

Kind regards,

LeaseWeb B.V.


UPDATE: PWR Maintenance went as planned without any interuption.

30 jun

Power maintenance Schuberg Philis Routers

Startdate: June 30th 2009 23:00:00(CEST)
Enddate: June 30th 2009 23:59:00(CEST)

Dear customer,

This evening we have planned power maintenance at Schuberg Philis.
Since the power feeds are redundant we don’t expect any downtime, however in case of issues a router may power cycle.

This power maintenance does not affect customer racks, only our ring devices.

Kind regards,

LeaseWeb B.V.

27 jun

link down [Update]

Time: June 27th 2009 11:45:00(CEST)
Expected Downtime: Unknown
Reason: Hardware failure / Under investigation
*******************************

Dear Customer,

At the moment we experience some troubles in (part of) our network.
A link has failed between one of our routers and an MPLS device.
Because of this most of our DSL customers are no longer able to reach the internet.

We are doing our utmost to solve this issue a.s.a.p.
As soon as we have more information available, we will publish it here.

Please be patient while we are working on this problem.

Regards,
LeaseWeb BV


[UPDATE 12:30]
The engineer is on his way to the datacenter with all spare parts he may need.

[UPDATE 13:15]
Our engineer replaced broken optic module at our MPLS device and the failed connection went up. We believe the issue is solved, however we are doing some tests to fully confirm it. If we find anything wrong we will keep you informed.