Node 13 France – Hard Drive Failure

Thank you to the those of you who reported the slow downs on Node 13 France. We have determined the cause of the issues to be a failing primary hard drive, which will be replaced shortly.

We will update this post as we receive more information / replace the drive.

Update: Our datacenter in France will be replacing the drive shortly.

Update 2: The system has been taken offline. Once it’s back online, we will perform some diagnostic tests and rebuild the array. Once the tests are complete (approximately 15-20 minutes), we will bring the system back online.

Update 3: The system is now back online, and all Minecraft servers are starting up.

Connection Issues between USA to France and overall Canada

It appears that a fire that took place in New York has damaged some fiber optics that controlled a majority of the traffic between the United States and France.

Repair crews are enroute/working on the problem now, but there are over 800 fiber lines sitting in that New York location. This may result in nearly 12-24 hours of spotty connectivity or even complete apparent downtime.

For the latest status, please visit our datacenter’s status page regarding the matter.

PayPal Payment Notification Issues

We have just been made aware of PayPal payment issues starting at midnight CST. This outage directly involves instant payment notifications to our website, and does not stop payments between you and our website. Based on some tweets by other companies, and direct confirmation from a few of our customers, this outage is not just with us.

Until further notice, please be sure to keep your transaction ID from your payment ready. You can find this in an email usually titled “You sent a payment to CubedHost, LLC.” The transaction ID can be found in the top right of the email, labelled “Transaction ID.”

We apologize for the inconvenience, and we are doing everything we can to process your payments as quickly as possible.

Update: PayPal has just notified us that any transactions that didn’t go through will be posted at the end of the night once they’ve fixed their systems.

CubedHost Website & USA Panel Downtime

We’re aware of an issue with our main website and USA Multicraft panel – it turns out our current hardware’s limits have been exceeded by our websites hosted on the box. Therefore, we are performing a transfer of all services hosted on our webserver to a new box.

We may shut down some services temporarily for 10-30 minutes for transfer, so that MySQL and other important files can be transferred without data loss.

Thank you for your patience and understanding.

Update: If you’re seeing this post, our websites have been moved over.

Unscheduled Maintenance – Node 1 & 5 Canada – Hard Drive Replacements

After some routine checks, and reported issues on the systems in question, we have been made aware of some failing hard drives in node 1 and node 5 Canada.

Within the next couple of hours, we will be having the NOC technicians replace these drives. This may mean up to 30 minutes of downtime per system.

Just to note, because we use a RAID1 setup in all of our systems, no data has been lost, and we do not anticipate any data loss on the systems in question.

Thank you for your continued support and patience.

Update: Unfortunately, our datacenter in Canada was unable to comply with our request in a timely manner, which resulted in a couple of hours of downtime without the replacement. We will be rescheduling this period to have the drives replaced.

Planned Network Maintenance – Jan 31st, 2013

Our USA datacenter (PhoenixNAP) has just informed us of a planned network maintenance to replace some bad networking hardware on January 31st, 2013 at 2:00 AM until 5:00 AM CST.

During this time period, an outage of up to 45 minutes is to be expected. The Network Engineers will be taking as many precautions as they can to prevent any extreme downtime.

Node 23 France Downtime

Earlier this morning we became aware of a restart on Node 23 France. We are currently diagnosing any possible hardware-related issues, and will be restarting the Multicraft daemon moments after determining that the node is safe for use.

Update as of 13:26 CST: The Multicraft daemon has been started, and Minecraft servers are now starting.

France to USA Network Connectivity Issues

Around 17:15 CST, we became aware of a network backbone connectivity problem between our Roubaix, France datacenter and Newark, New Jersey, USA. This problem will most likely affect only players from USA who are connecting to France-based servers. We apologize for the inconvenience and will update this post when more information becomes available.

As of 20:45 CST, all issues seem to be resolved. If you still have connectivity problems, please open a support ticket.

France Multicraft Inaccessibility

It appears our France Multicraft is getting hit with a DDoS, large enough to mess with our virtualization software. Generally, this is a bad idea, and should’ve been replaced a long time ago, but we had not gotten to it yet due to many other projects being worked on.

We are currently working to get a new Multicraft panel online for the France location while this DDoS is ongoing.

As we receive and setup a new box for the panel, we will update this posting.

Update: We’ve received the new box, and we’re setting up the HTTP software on it.

Update 2: We’ll be adding another IP to this box in order to prevent Hamachi conflicts, however the panel will now be accessible from http://5.135.188.182/

Update 3: New IP assigned, DNS record changed. The Multicraft link for France should be working in the next 5-10 minutes.

Node 21 France Downtime

Earlier this morning we learned of a possible hardware failure on Node 21 France. After diagnosing the issue and determining that the on-board RAM had gone bad, we put in a request to swap it out and get the server back online as soon as possible.

All went as planned and at this time all services have been restored.

Thank you for your patience!