Early in my career I was charged with creating a recovery program for my company. The CEO counted on me to find a way to make sure everything was protected and recoverable to meet the needs of the business and in case of a disaster. Of course this new project came about after a MASSIVE outage and millions of dollars were lost. At the time I didn’t realize that this process was actually called Business Continuity (BC). I decided the best course of action would be to meet with all the heads of the business to identify and prioritize the systems and then attach retention times based on the “interview” that I did with them. During this process and execution I learned a few lessons that I think will help you build a successful BC program at your company.
Lesson One: CEO sponsorship is CRUCIAL!
At the end of the day without sponsorship from the CEO a BC program will go nowhere. The CEO heads the business so therefore they are ultimately responsible for it staying running during an outage or disaster. Heads of business will be more willing to spend the time it takes to go through the interview process if the CEO is pushing the project. Early on when I kicked off my BC program I had a hard time getting meetings with heads of business. Once the CEO sponsored the program people made time to work with me.
Lesson Two: Business Continuity is its own discipline
When I meet with customers a lot of times it’s solely a backup admin trying to run this program or companies feel they can lump Business Continuity into Risk Management (RM). Business Continuity and Risk Management are similar but they are also worlds apart. To explain it another way, Business Continuity picks up where Risk Management leaves off. So let’s assume that under Risk Management that the probability of a hurricane leveling your Boston office is near zero. So because of the level zero attached to that risk you decided not to spend the money it would take to put in another office location as backup. Under Business Continuity the hurricane has hit…now what are you going to do about it? The solution could be as simple as letting all your employees work from home and replicating the data to the cloud. Both of these programs assessed the same potential issue but dealt with them very differently. So combining a BC and RM program can create conflicting ideologies. I have found the best results by keeping them separate.
Lesson Three: Effective Business Impact Analysis Saves Time and $$$
One of the most crucial steps in a Business Continuity Program is the interview phase. This is where you meet with the business owners and find out what systems they own and need to have running in an outage or disaster. This is also where you apply priorities and set RPO and RTO. One of the crucial questions to ask is “How much time could this system or function be down before we go out of business?” In some instances systems could be down for a month with no real impact but others can’t be down longer than 5 minutes. The needs of a system will dictate what solution you put into place to support it during a disaster.
A trick I learned early was to create different options for the business owners and assign an estimate cost to each one. So for example if they picked a top tier recovery method I would tell them the potential cost of that solution and they would often drop their need down a few levels. Running an IT shop is not cheap and making the business owners aware of the cost could save some money and make your solution easier and most appropriate.
Lesson Four: There is a real cost to doing DR
During the Business Impact Analysis interviews with the heads of business I would always try and associate a cost to the desired DR program they selected. I created tiers of options and tried to keep it as simple as possible. Here is an example chart like the one I use to use.
Backup to Disk (Local)
Offsite to Tape
Replicate Disk to DR
Backup to Tape
$10.76 / GB
$3.65 / GB
Backup to Tape (Local)
$1.33 / GB
$.33 / GB
$.12 / GB
By walking through the DR tier they selected and running a cost estimate the business leader has a better idea of the financial impact of running in DR. Also, by having the CEO sponsor the project there will be no surprises if there is a need to increase IT budget to accommodate the selected DR tier.
Lesson Five: Testing….Testing….and more….TESTING!!!
After spending a lot of time and money on getting CEO buy in, interviewing heads of business, and purchasing solutions the last thing you want to do is be stagnant. Business Continuity should be a daily, monthly, quarterly and yearly focus. From now on all new machines must go through the BIA process to make sure they get added to DR. At least once a year you should run a full blown test of the solution you created. If a system is crucial to the business a quarterly test should be done. The more testing you do the higher the confidence in your solution you will have. If you need to make adjustments or changes it’s better to know ahead of time than when a real disaster comes and you can’t recover your systems.
Now move forward with confidence and build a Business Continuity plan that fits your business needs. If you need a suite of tools to protect your data and hit the RPO and RTO’s your business requires take a look at the Quest Data Protection Suite.