Faculty Advisor: Does Benchmarking Really Show How Well Your Security Program Measures Up to Best Practices?

Q: Our new senior management team is requesting a report on how we measure up against security best practices, but we’re struggling to find formal or accurate benchmarks, and the data we do find isn’t often comparable with our organization. I think best practices are important, but they have to be fair and accurate to be useful, and we’re coming up short. What can we do?

A. You’re not alone. What we’ve found over the years regarding security industry research and benchmarking is that everyone is looking for information, but few are willing to share it. In the realm of enterprise security in particular, there is often an understandable hesitancy about revealing details. Security professionals do not want to reveal how their department operates because sharing too much could lead to new vulnerability. Even when they want to share knowledge, busy professionals often don't have the time, or they don’t have the budget, to retain someone else to do the work.

Many security professionals, like you, turn to benchmarking to answer senior management questions such as:

-- Are we on the right track related to compliance?

-- How do we compare to our peer companies as far as budget and services provided?

-- How can we save money for the services we provide?

-- Consider almost any current news story: What if this happened to our company?

-- What are best practices for handling an information breach?

Looking for useable and reliable answers can be frustrating. A lot of security-related research is driven by a commercial agenda. Or it isn't quite specific enough. Or it's in pieces. Or it's created by people who don't really understand security.

The fact is, the practice of security doesn't really know itself, and most people involved are familiar with various pieces of the puzzle but haven’t looked at the big picture. If you talk to 10 different people, you get 10 different concepts of security, although there will be some common threads.

A strategy + business magazine article, “10 Principles of Organization Design” pointed out that organizational benchmarking should be used sparingly and carefully. Briefly, the article argues that there are so many factors influencing business structure and strategy – from location, to customer demographic, to organizational value proposition – that it’s difficult to find organizations that present one-to-one comparisons. And benchmarking with the wrong example, it says, will hurt rather than help. And we agree and have tried to articulate this to our practitioner community.

The article concludes its benchmarking principle this way: “If you feel you must benchmark, focus on a few select elements, rather than trying to be best in class in everything related to your industry.” That is, if senior management is requesting best practices, look for best practices within narrowly defined parameters, and be prepared to explain why broader comparisons are ineffective.

The SEC created the Security Leadership Research Institute (SLRI) to help solve some of these issues. We strive to provide accurate, specific and security-focused benchmarking data. The SLRI is designed to facilitate sharing of practitioner-based research reports and benchmarks. Ultimately, the goal is to create a dynamic collection of research intelligence that will evolve along with ongoing changes to the security function.

The SLRI is currently conducting a comprehensive security and risk management benchmark focusing on security budgets, programs, services, and staffing. There is no cost to participate and the resulting in-depth analysis and report will only be made available to those that participate in the benchmark. For information, visit the SLRI here. 

Answer provided by Kathleen Kotwica, EVP, Security Executive Council, and Principal Analyst, Security Leadership Research Institute, and Greg Kane, Director of I.T. and Product Technology, SEC.