Venyu Blog

On the heels of the recent posting by my colleague JMCSWEET on Testing Your Backups, comes this eye-opening posting from Bruce Hoard at Virtualization Review entitled “4 Ways to make Disaster Recovery Testing Less Painful.” That, and given what’s happened recently in the wake of the earthquake-tsunami-nuclear reactor compromise ravaging Japan, it’s probably a timely column as well.

Citing comments by Jon Toigo, CEO of IT consultancy Toigo Partners International, who likens such testing as “the long tail cost of DR planning where not thinking about testing up front, when selecting strategies, will come back to bite you in the butt down the line,” Hoard lines up the DR pins and, with help from Jon Toigo and others, knocks them down.

Reason 1: It’s a hassle. Many companies don’t test their DR systems because it’s just too cumbersome in complex IT environments where systems must be up and running 24/7, notes Bob Roudebush, VP of marketing at Neverfail Group. “Of course, the negative ramifications are huge,” he adds. “If you don’t test your DR, it might not work in a disaster, or you might not know the right steps to take to get your systems back up and running.”

Solution: Roudebush says companies should look to implement solutions that have easy testing capabilities built in. These solutions should minimize the time it takes to test systems, should provide automatic testing functionality and should not cause disruption to running applications and systems.

Reason 2: Testing is misunderstood. According to Toigo, “People are afraid that they just spent a boatload of money on some data protection scheme, and if they test it, they may fail.”

Solution: In Toigo’s opinion, a primary purpose of testing is generating failures so DR systems can be improved. Adds William Hughes, Director, Consulting services BC/DR Center of Excellence at SunGard Availability Services, organizations need to promote the view that a test doesn’t need to be passed; it needs to promote improvement. Toigo sums up by declaring: “There are no failed tests.”

Reason 3: Resource Restrictions. IT budgets remain tight, and doing more with less is still frequently the corporate mantra. As a result, CIOs or senior IT execs who signed off on purchasing data protection products one day may turn around and say there is no more money to test them the next. “Projects are generally reducing costs or creating opportunities,” says Hughes. “It’s very hard to impact either one of those areas.”

Solution: Hughes says organizations should re-evaluate their validation and testing processes, and determine ways to minimize the impact on them. That could involve using a third party to conduct the exercise or simply using less constrained resources to participate.

Reason 4: What’s it Worth to Me? The lack of perceived value is a major DR testing bugaboo that results when DR testing is viewed as insurance that may never show any kind of ROI.

Solution: According to Hughes, “This is an issue that plays with resource constraints, since it’s certainly harder to extract resources to support something that’s not perceived as having high value.” He goes on to note that this can happen because the organization looks at the implementation of the solution as the end point, rather than an inflection point. Or it may be that the organization has validated the solution a number of times and sees it as routine or easy. By using testing to continuously raise expectations, there is an ongoing state of improvement that reflects the increased value of testing.

To paraphrase my colleague, Don’t wait to test your DR plan because the first time you really need it and it doesn’t work, everything you did to prepare for that singular incident will be absolutely meaningless. And then, even in the wake of a massive flood, for example, you may indeed be left high and dry.