Home > Blogs > Shon Harris Guest Post: "Risk Management Mess"

Shon Harris Guest Post: "Risk Management Mess"

Hey there! We have something special for you today. Information security guru Shon Harris, a featured Pearson author, has the following to say with regard to risk management. Enjoy!

What do we have in the world of risk management in the IT and security world today, a bit of a mess. Risk management has been a nebulous, pathless utopia that has been just out of our reach because we are randomly wandering around in pseudoscience and non-sensible numbering systems. Risk management in the financial sector is more defined and stable because it is based in numbers and statistics that have a more direct correlation to the components it is describing. In the information security world we have silly things like the Annualized Loss Expectancy (ALE) formula that has variables that are subjectively populated along with other just as ineffective formulas that provide false senses of securities.

Many organizations, corporations and security consultants are comfortable with common balanced scoreboard values, associating color codes to risk values, and/or using arbitrary risk rating systems. Now these can be useful components in risk assessments and potentially useful tools for risk management – but not how they are usually used today. The crux of the questions you should be asking your employees or consultants who hand you a stack of paper covered with pretty graphs, pie charts, and risk ratings used to explain the risk your organization faces is ‘Where’s the beef?’ What is this information based on? Where did these numbers come from? Why does this server or business process fall within the red risk quadrant and the others in the yellow? But here is another important key – will you know if they have to dance to answer these questions? If your consultant starts throwing out acronyms, formulas, and technical mumbo-jumbo will you just figure that he knows what he is talking about since you don’t understand him? This is what most commonly happens. If someone really understands risk management, she can break it down and explain it properly to the necessary audience without going over their heads. Risk management is complicated, but if you are paying someone to perform it for you, you need to know what you are paying for and more importantly the REAL risks your company faces.

One large corporation called me in as a consultant to help them with a wide range of security issues. They had had one of the top four consulting agencies in there carrying out a risk assessment so that the risks could be prioritized and the customer knew what risks had to be mitigated immediately versus ones that could be tackled over time. The consulting company left my customer with a stack of papers that had colorful spreadsheets, pie charts, graphs, and high-level boilerplate explanations of the identified vulnerabilities. To the executives this was very impressive and they had a false sense of security that their real company risks had been identified and now they just had to be dealt with. The top risk that was called out by the large consulting firm was ‘Mainframe Access’. This is a very vague term and basically useless. I knew my customer had retail stores throughout the United States and Canada and that credit card numbers and social security numbers were flying around their network in cleartext. The mainframe did not hold this type of data or data that was more sensitive in nature.

So this top risk in the paperwork I was holding seemed suspicious to me. I wanted to see what supported this finding. I looked at the mathematical formulas used in the Excel spreadsheet that the risk report findings were based upon and I saw that one variable was not included in their simplistic formula. Someone had fat fingered the formula in the consulting company’s boilerplate template. The stack of reports I was looking at was based upon an incorrect formula. Each risk and its associated ranking was based on this formula. So out of curiosity I corrected the formula and found that now the risk of ‘Mainframe Access’ had moved from number one to number five. This means that the company thought a majority of their upfront resources needed to be allocated towards addressing their mainframe access controls. They had already had meetings, assigned resources, and started focusing more on their old and hardly used mainframe and not on all of the insecure PCs that were holding and processing social security numbers and financial information. But hey, the paperwork had pretty pie charts and the consultants were all in nice shiny suits!

What about those automated products that take input information, do all of the risk analysis and computation in the background, and kick out paperwork that dissects your organizational risks with instructions on how you should go about fixing them? I will return to what your core questions should be, which is ‘what are these results based on’? With risk management products all of their computational secret sauce is baked in and works behind the scenes, which means that it is harder for you to view and evaluate. This means that you need to know enough to ask the right questions before purchasing the product and putting stock into its output. Just to drive the point home, I will provide another scary story.

I was invited to sit a four day class on a risk management product that is commonly used in financial institutions and the government sector. The class was for analysts who configure and use the tool to carry out information security risk assessments. We were able to get under the hood and look at many of the formulas that were being used. The formulas were solid and worthy for their application, but the real problem came from the potential input values. Every formula is just an instruction set with some empty variables. The variables are filled with the necessary input and then the computation is carried out on those inputs and a result is provided. But true with almost every equation is ‘garbage in garbage out’. The result of the equation has a direct relationship with the inputs. The inputs for this product, and many risk management products, are extremely subjective thus the results range from analyst to analyst. Many people in the class were very mystified by the mathematics that was used within the product and felt that because they did not understand it – the product had to be doing things correctly.

If you are not careful, the largest risk your company could face is the risk assessment methodology being used. The best case scenario from this would be that a risk assessment was a waste of time and money. The worst case scenario is that your organization is missing some of the largest REAL risks it is being faced with.

Since the term ‘risk management’ is nebulous and not a defined and scientific discipline within the information security realm, anyone and everyone is putting up their shingle pontificating their skills in this field. Make sure to work with true experts not only in the information security field, but those who also specialize in understanding and mitigating risks to meet your organization’s risk appetite.

Author byline:

Shon Harris, CISSP®, MCSE (shonharris@logicalsecurity.com), is the president of Logical Security, a security consultant, a former engineer in the Air Force’s Information Warfare unit, an instructor, and an author. She has authored two best selling CISSP® books, was a contributing author to the book Hacker’s Challenge, and a contributing author to the book Gray Hat Hacking. Shon has taught computer and information security to a wide range of clients, some of which have included RSA, Department of Defense, Department of Energy, National Security Agency (NSA), Bank of America, Defense Information Systems Agency (DISA), BMC, West Point, American Express, and many more.

Shon was recognized as one of the top 25 women in the Information Security field by Information Security Magazine. She writes security articles for Information Security Magazine and Windows 2000 Magazine and other leading industry journals.