November 19, 2017

Disaster Diaries #5 - A Problem Shared, Is A Problem Halved

Disaster Diaries 5: A Problem Shared is a problem halved

Disaster Diaries are true stories taken from our channel partner support logs. They tell a terrible story that might have destroyed one of our partner's customers and then we tell what actually happened because of high quality backup and recovery services provided by DATAFORT. Later we’ll discuss how we can work with you to provide the same level of care for your precious customers.

SITUATION:

OnesizeIT a successful IT Managed Service provider had a prize client, Assetfund21; a mid-sized asset management firm who they had supported for several years.

One morning OnesizeIT received a call from the compliance officer at Assetfund21. They were in the final stages of negotiating a deal with a much larger fund manager who was considering putting several million pounds under management with Assetfund21. However, due diligence by the large fund revealed that Assetfund21’s business continuity, data archival and disaster recovery plan didn’t provide sufficient resilience. Whilst Assetfund21 were FCA compliant in their own right for the size and complexity of business they normally undertook, their new corporate client needed them to meet a much higher standard to match their own.

Assetfund21 accounted for 31% of OneSizeIT’s monthly billings: They needed to keep them happy!

CHALLENGE:

Assetfund21 now needed a more sophisticated recovery and data retention service with guaranteed recovery times. The solution needed to: 

  1. As nearly as possible, eliminate downtime for the company’s trading and back office systems.
  2. Improve upon overnight backup. If backups completed each night at 6PM, but an accident happened at 5PM, 23 hours later, then a whole day’s worth of transaction data would be lost! Unacceptable.
  3. Create the ability to recover files not only from a limited number of ‘versions’ but as they existed on specific dates and even specific times!
  4. Make it possible to recover individual emails and mailboxes that had been purged from the Exchange Server buffer. And to be able to do it simply and quickly.

Create a disaster recovery strategy that could recover their critical systems remotely within 4 hours. Get the office working in five. And they needed to be able to actually demonstrate that this worked!

Existing backup and recovery

Assetfund21 now needed a more sophisticated recovery and data retention service with guaranteed recovery times. The solution needed to: 

  1. As nearly as possible, eliminate downtime for the company’s trading and back office systems.
  2. Improve upon overnight backup. If backups completed each night at 6PM, but an accident happened at 5PM, 23 hours later, then a whole day’s worth of transaction data would be lost! Unacceptable.
  3. Create the ability to recover files not only from a limited number of ‘versions’ but as they existed on specific dates and even specific times!
  4. Make it possible to recover individual emails and mailboxes that had been purged from the Exchange Server buffer. And to be able to do it simply and quickly.
  5. Create a disaster recovery strategy that could recover their critical systems remotely within 4 hours. Get the office working in five. And they needed to be able to actually demonstrate that this worked!

OneSizeIT’s MD had a hastily arranged management meeting with the Technical Director, Adrain Pipe. Adrain was in no doubt as to which direction to follow! “There’s a lot of software out there that will do what they need. I’ll do some research and then we can put together a solution that makes us a good return and will protect Assetfund21. Doesn’t make sense to lose margin to an outside provider.” 

OnesizeIT implemented an internally managed service based on their infrastructure and a licensed third party software application. 

The wheels started coming off from day one. There wasn’t time to test the newly acquired software and customer support for the package was less attentive than desired. One week quickly turned into 3 before OnesizeIT could setup a proof of concept test to show to Assetfund21. But sadly the POC failed to impress. 

As a consequence Assetfund21 was left making excuses to the 100 million sterling feeder fund. They could not demonstrate the asked for business continuity and lost the fund’s business. This in turn led them to end their long standing service contract with OnesizeIT.

RESULT

In effect OnesizeIT had sacrificed a £5000 a month contract for the sake of gaining a few hundred pounds extra margin and their insistence on keeping all technology in-house. A bad commercial decision. 

The more sophisticated solution was a lot of work and only appropriate for Assetfund21 and maybe one other client. Had they weighed the potential commercial loss against the benefit of a proven outsourced solution it would have been clear how to proceed. They had focused on going down a route they were comfortable with and because of that ended up ignoring the real risks.

Bill Dahous[O1] , the Managing Director of OnesizeIT, saw immediately that trying to create a specific in-house solution for one client was not cost effective and they would struggle to meet the delivery schedule. He therefore instructed Adrain Pipe, the Technical Director, to look for a suitable partner who could deliver the needed service. 

Adrain provided Bill with a short list of potential companies. Bill contacted and met with the short list and selected DATAFORT to undertake a pilot of their Critical Care solution at Assetfund21. DATAFORT worked alongside them to present to Assetfund21. They were able to offer reference sites that had their service in place for a number of years. During a meeting at DATAFORT’s offices, everyone agreed a process whereby OnesizeIT would remain customer facing in most instances, but AssetFund21 could always come to DATAFORT directly if Onesize’s team was unavailable. In essence it would be a ‘no waiting’ service.

Within a week the proof of concept system was up and running and a week later during a conference call between DATAFORT’s Technical Director, Onesize’s MD and Assetfund21’s inhouse counsel, the features of the solution were presented to the prospective client. 

The presentation was given in a way that highlighted the suitability of the service in meeting the compliance requirements of the client. It also demonstrated the minimisation of transaction data loss and how the service improved network continuity whatever happened to the servers or premises! Not to mention an archiving policy that could meet the guidelines provided by the Financial Conduct Authority. 

Assetfund21’s new client felt assured Assetfund21 would not cause risk to their business and signed the new agreement the next day. OnesizeIT received a pat on the back from Assetfund21, secured their future with the company. Thereafter they received quarterly commission deposits from DATAFORT and the team was able to focus on other customer centric activities. [

The preceding is a description of an actual scenario that took place with a DATAFORT partner. The alternative solution of creating an in-house bespoke solution never took place, but is a fairly accurate representation of what does happen to IT support companies when they try to adopt this route; as reported to us on a frequent basis. All characters, companies and suggested revenues appearing in this work are fictitious. Any resemblance to real persons or campanies, living or dead, is purely coincidental.

What do you want for your business and customers?

Contact Business Development Director Marcie Terman to develop a relationship with DATAFORT. Learn more about how we will work with you to protect your customer’s while enhancing your business efficiency and reputation.