July 26, 2017

Disaster Diaries #6 - Virtual Network Misfortune

Disaster Diaries 

Disaster Diaries are true stories taken from our support logs that show how we worked with our IT partners to enhance their profitability while helping their customers.

SANs are resilient...right?

GreatHolidays was a successful online travel company with a growing customer base and a 24/7 global operation. The international nature of the business required an ‘always on’ approach so with the help of their support company SafeashousesIT they had just completed the installation of a centralised SAN to run the company’s dynamic, high transaction environment.

 For a relatively small company, SafeashousesIT was excited by this project which was the largest they had undertaken to that point and they needed to secure a bank overdraft in order to cash flow the project.

SafeashousesIT had raised the issue of disaster recovery several times with GreatHolidays management but the company perceived the main threat to their business was server failure and not the unlikely chance of a fire or flood.
“Look, there’s nothing left in the capital budget” complained GreatHolidays CFO Edna Sand. This will just have to be addressed in the next budgetary cycle.

Worst rains hit the southeast in 50 years...

Within 3 months, the winter of 2014 left the southeast of England under water including the offices and computer room at GreatHolidays. The whole of the IT function was knocked out, so there was no website, no call centre and no back office function.

Did that mean there was now no company?

Following the flood, it soon dawned on GreatHolidays management that just because something had never happened before was no proof that it would not happen in the future. The entire office was under 3 feet of water, all power systems blown, the server room in an unknown condition.

GreatHolidays CFO Edna Sand called Safeashouses MD on her mobile and asked what could be done?  Safeashouses MD explained that there was nothing that could be done until they gained access to the server room to assess the damage. Early inspection of the SAN revealed that everything was lost. As it was a fault tolerant system nobody had thought of taking a backup so all the data was gone. GreatHolidays undertook a valiant effort to recover booking information from their industry partners, but word soon got out and customers started cancelling their bookings until the temporary phone system crashed from the volume of calls. By that time, the story was all over Facebook and Twitter. Assurances were made in the industry press but they were perceived as a desperate attempt at damage control.

It was all over and the company folded shortly after leaving SafeashousesIT holding a worthless contract and no way to reclaim the money they had fronted into the project. Within the year, SafeashousesIT had no option but to go into administration.

Whilst there was no capital budget left in the pot, we at DATAFORT worked with SafeashousesIT to propose a DR solution that needed no capital investment.

DATAFORT’s VNCare is a disaster recovery solution designed for Virtual Networks and was supplied as a managed service; so no infrastructure investment, service payments that can be offset against profit and no capital costs. Through VNCare DATAFORT installed an onsite appliance that managed the transmission of data to DATAFORT’s remote datacentre where it was stored as a frequently updated virtual backup image that could be brought online within a couple of hours if the primary network failed. 

When the flood hit, SafeashouseIT called DATAFORT. We immediately invoked the failover servers run from our data centre. GreatHoliday employees connected via VPN to our data centre. Feeds from the various booking services were redirected and the business was back online well within their stated SLA while GreatHoliday employees spent their time speaking to people who had noted a lapse in service.

GreatHolidays was back online selling and booking holidays – and very few people ever knew what happened. So GreatHolidays survived a huge disaster. SafeashousesIT was then able to use them as a reference site to up their game and acquire new customers. Turnover for both companies on the rise, instead of two companies in administration. 

 

The preceding is a description of an actual scenario that took place with a DATAFORT partner. The alternative solution of not having a DR solution did not happen, but is a fairly accurate representation of what does happen to IT support companies when they do not allow for DR within Virtual Networks; as reported to us on a frequent basis. All characters, companies and suggested revenues appearing in this work are fictitious. Any resemblance to real persons or companies, living or dead, is purely coincidental.

What do you want for your business and customers?

If your customers need a DR solution for their Virtual Network we may have the answer with VN Care

Contact Business Development Director Marcie Terman to develop a relationship with DATAFORT. Learn more about how we will work with you to protect your customer’s while enhancing your business efficiency and reputation.