Hidden threats to enterprise: will your business continuity go according to plan?

FST’s Helia Pheonix looks at what plans financial institutions need to put into place to ensure that shareholders can sleep at night.

Recent large-scale disasters like 9/11, the Asian tsunami of 2004 and hurricane Katrina have established the terms business continuity management (BCM) and disaster recovery planning (DRP) as key terms in today’s business vernacular. However, you don’t need a dramatic terrorist attack or natural phenomenon to have an impact on an organization. Enterprises can be vulnerable to a number of more mundane factors: hardware or software failure, data corruption, power failure, fire, sabotage, vandalism, employee theft, strikes, human error, unproven or unauthorized software, viruses, denial of service attacks, hacking and other illicit cyber intrusions. These can be split into the sections of natural disasters, IT or infrastructure failure, terrorism and industrial unrest. All of these events can affect business. Is your financial services backup-plan thorough enough to stand up to all these eventualities?

For financial services organizations, successful contingency plans need to establish how to provide continued access to business processes while maintaining security and confidentiality in the event of an incident. It is important to remember that BCM is concerned with defining a set of business consequences, rather than dealing with the minutiae of an avian flu pandemic. Lyndon Bird, Technical Services Director for the Business Continuity Institute (BCI) explains further. “In the case of a flu pandemic, BCM is interested in how you would continue your worldwide business if up to 50 percent of your global workforce were affected. If you’re not too careful people start looking at everything as a threat that needs business continuity solutions. I am starting to get asked increasingly about climate change, which is of course an important business and social issue, but it isn’t one that I have a business continuity solution to.”

This new attitude towards BCM is a significant change from the way that BCM was perceived in the US financial sector. Increased security risks, new industry regulations and a growing reliance on technology have all helped to push business continuity up the financial services agenda. “Just a few years ago, BCM was still seen as an issue for technology and communications firms, to recover from technical failures and keep business going,” elaborates Bird. “Now it is seen as a senior compliance issue for management to continue to operate their business as according to defined corporate governance requirements. The after-effects of 9/11 profoundly changed enterprises’ thoughts on BCM. In the financial sector there was a realization that incidents can potentially happen on a wider scale than originally thought, which is now a significant part of contingency planning. At the same time there are increasing compliance requirements – SOX and other regulatory regimes are very tight in these areas, which is having an impact in terms of business. Compliance, terrorism, global issues like Avian flu pandemics and a wider appreciation that business continuity is no longer a localized issue are some of the big changes in thinking.”

A notable example of a successful continuity plan implemented before 9/11 belonged to the New York Board of Trade (NYBOT). The commodities exchange had been spending $300,000 annually for a backup facility that sat idle for years. The exchange used the site in the days after 9/11 and continued to use the site as its headquarters for some time afterwards. The success of the NYBOT’s continuity plan not only invalidated the comments of its critics, but also sparked a radical re-evaluation of the business continuity market. It highlighted the vulnerability of computer systems as well as phone, power, and transportation grids. What had been seen as an issue primarily affecting a company's data centre was now framed as a strategic imperative affecting every aspect of infrastructure.

Reluctance to embrace BCM

Despite the evidence to support early implementation of business continuity measures, many enterprises are still reluctant to address the BCM issue. When profits fall, BCM is often the first thing to be cut from the budget. There are still myths and suspicions surrounding it: if an enterprise is small it doesn’t need BCM, that BCM is too expensive, that virtualized data-backups are not safe, that online backup can’t handle data from a Fortune 500 company, that an enterprise’s emergency plan or IT disaster recovery plan is enough, that business interruption insurance will cover all the loss, that it will be cheaper and easier to cover the BCM project in-house, or that BCM is too complicated to implement at all.

This reluctance to introduce sufficient measures is understandable. Business continuity is not a tangible commodity so it can be difficult to understand its benefits, until you consider the consequences of inadequate contingency planning: in the 1993 World Trade Center bombing, out of the 350 enterprises affected 150 enterprises went out of business. Years later, after 9/11 some enterprises – Morgan Stanley, Cantor Fitzgerald and American Express, for example – were able to resume business within several days while other companies suffered severely, many going out of business. In some cases, there were no backup communication systems and a number of companies didn’t even have an accurate count of employees. Some more worrying statistics are that 80 percent of businesses suffering a major disaster go out of business in three years, while 40 percent of businesses that suffer a critical IT failure go out of business within a year. In the case of suffering a fire, 44 percent of enterprises fail to reopen and 33 percent of these failed to survive beyond 3 years (the primary reason being loss of vital records).

As the regulations governing financial services are increasingly tightened, you might think that means watertight recovery plans are in place. But there is always room for error, as Bird points out: “The perception has been that BCM is a technology issue, so if you have the capability of a technology backup plan then that is all you need to do. Technological recovery is certainly a part of it, and financial services usually have the internal skills and technology to do that part well. But there is also the HR and people factor in recovery planning, which is not generally done well in financial companies. There is also crisis management: precisely how do you deal with the company aspects of the incident when it happens? How do you handle your control centers, your management processes, who is in charge, have you looked at issues relating to stress or trauma, issues with not having the right people in the right place, the decision process, the communication process? Generally speaking, financial services need to concentrate more on crisis management and the HR people side of things. They are very strong in the technological side of things, because that’s where the money has been spent.”

Missing maps

As well as potential shortcomings when organizing staff in contingency planning, financial institutions also face the problem of safeguarding their increasingly complex and distributed IT infrastructures. “That is another technology issue that is not often talked about,” confirms Bird. “The sheer level of complexity that people build their technology to can mean there are more weaknesses than before. Years ago organizations had big mainframe computers. The systems were slow to be developed and they weren’t always very responsive to what people wanted, which meant fewer points of weakness but poor service. Today – particularly in a fast-moving industry like finance – people want great service and they want it quickly. What typically happens is that enterprises will hire contractors, buy some servers and build a system that very quickly goes operational. It starts not being mission critical because it’s only just been deployed. When it becomes successful it goes super mission critical, but it is not mapped against the overall technology infrastructure. When something happens to that system then the enterprise will have problems, as fixing it won’t be within their operating knowledge.”

This is a problem that still needs to be addressed by many enterprises: the problem of stretching to embrace new technology too quickly, having too much data, too many disparate systems which are not properly mapped into their IT architecture. This lack of communication is a symptom of business in a fast-growing industry like financial services, and can mean that many of the server-based systems in the financial world are not as resilient as perhaps they should be. Bird elaborates: “As a consultant in this field I would often go to organizations where the people who were supposedly responsible for business continuity would suddenly find hardware, software, complete mini networks and applications. They had been authorized by departments but added externally, so nobody knew they were there. It sounds peculiar, but it does happen. There are great pressures in large institutions to get the most from their assets but in this race to succeed they don’t always realize that they are not mapping everything that they have got. Testing internal systems is all very well, but if a whole set of servers is outside the mapped system there can be untold problems. The lack of mapping and understanding cause-and-effect is a hidden threat to industry.” While lack of mapping in an enterprise might not be as newsworthy as a natural disaster, it certainly causes problems and should be addressed to ensure successful deployment of contingency plans, not to mention adhering to compliance regulations.

In financial services there is a wealth of sensitive information that is almost all IT-based, accessed on a regular basis by staff and customers, which are more potential weaknesses that can be targeted. Most enterprises operate with desktop computers, distributed servers and possibly a sales force that uses wireless laptops, putting the operation at risk not only from outside forces attempting to penetrate network from cyberspace, but also from potentially malicious insiders. Volumes of sensitive customer information are stored on any financial service’s network, including addresses, phone numbers, account and tax ID numbers. Corruption or breach of security in such an environment could have serious consequences for both enterprise and customer. Like with all things, prevention against cyber-terrorism is better than cure. There are a number of solutions available to enterprises: network intrusion detection systems, access control lists, firewalls, antivirus, antispyware and imposing web content browsing limits for employees are just a few.

Data safety

The safety of customer and corporate data should be of the utmost importance in financial services contingency planning. While enterprises may find it relatively easy to source commodity components (file servers, storage units, network switches), ensuring their data is secure is a more difficult task because it is completely unique. Irregular or defective backup files, overwritten data and viruses are all major causes of data loss. While financial services companies need to ensure they have easy access to a real-time copy of their critical data, the costs of this can be difficult to stomach. “Initially regulations tried to specify that key financial institutions operating in New York ought to have recovery centers at least 200 miles away,” explains Bird. “This was pointed out to be totally impractical. To start with, there is the problem of where to locate the centers. Also for the majority of incidents like the 70 percent which are due to internal issues – nothing to do with national-level disasters – does that mean your enterprise must have a real-time recovery center, 200 miles away, which is totally illogical for you? Another problem is some of the larger financial services organizations don’t go to the third party marketplace for BCM as they look after it internally. However they usually work in relatively close areas, so in a wide-scale incident they’d be knocked out of their own recovery centers.” The availability of backup data is a core component of business continuity management – as is having an effective disaster recovery plan – but one that needs to be reconciled against the cost. A company’s revenue stream, however, is not the only victim of poor business continuity – customer loyalty, staff morale and a company’s brand can also suffer, often with far-reaching consequences.

Surprisingly, despite these well-documented external risks to enterprises, the statistics show that the majority of problems spring from another area entirely. Vendors offering disaster recovery solutions have stated that between 60 to 70 percent of all problems that disrupt business are due to internal malfunctions of hardware or software, or human errors that may lead to fraud. “You could argue if organizations spent more time and effort on building a more resilient infrastructure and invested more in security then these problems would happen less, and would not require the implication of plans,” states Bird. “Business continuity ought to be for catastrophic things, very major interruptions to your normal operations. It shouldn’t be necessary to invoke an IT recovery center just because you have some technical problems, but that is still the major reason why people invoke them. Which is disappointing, in a way.”

Consultant Virginia Robbins, who has worked for a number of years advising banks in transitional phases of business, agrees with Bird, although she has different experience. “The majority of small and mid-sized banks I have worked with exist because they provide better service than bigger banks,” she states. “These banks that I’ve worked with don’t even consider mistakes of this kind – that may lead to fraud and so forth – as being a part of business continuity. These operational issues are so embedded in the banks’ reasons for existing that they don’t consider them to be disruptive issues that require plans. Small to mid-sized banks tend to look at the bigger disasters – here in California for example, earthquakes – as things that will disrupt business. Business continuity for them means planning for scenarios like earthquakes, and discussing how they are going to manage.”

Regardless of how they are categorized, potential disruptions to business are in flux as threats and opportunities emerge daily. These are all well documented in the media: power outages and other problems resultant of inclement weather, virus attacks, increasing hacker sophistication, an employee losing a company’s laptop, flu pandemics. Within enterprises themselves there can be changes in the use of various applications, which require reflective changes in continuity measures. As Robbins explains further: “As recently as the late 1990s, most executives regarded email as nice to have but not essential. Today you simply can’t have phone systems and email going down. There is no tolerance for outages in those systems.”

As the enterprises themselves change, as the potential causes for disruption to business change, so too must business continuity measures change accordingly. Annual risk assessment and annual impact analysis must be undertaken to ensure smooth running of the enterprise under differing circumstances. Robbins elaborates on this matter: “There was the case of the credit union in Louisiana that – after hurricane Katrina – had deployed its continuity plan, was ready to open and start supporting its customers but was asked by local authorities to stay closed for an additional three days. There was no electricity, limited services, and local authorities were focused on bare necessities so they didn’t feel they could provide support for a local bank. Eventually the enterprise brought in their own security to guard the premises. This case shows there was an extra dimension to the contingency planning that hadn’t been considered until it was put into practice.”

The resilient enterprise?

Ensuring resilience is no easy task, and can require a range of strategies and technologies from mirroring and replication to clustered environments and load balancing. There are times, however, when – no matter how well an infrastructure is designed and protected – replacement systems will have to be installed unexpectedly. Although implementing an effective business continuity strategy requires a commitment of both funds and resources, the benefits of such an activity far outweigh the initial investment. These processes and plans could make all the difference to your company’s bottom line in the event of a disaster – no matter how big or small it may be.