Quantify Your Security Position

One of the easiest things to do is to say that your organization wants to improve security because of x,y,z factors across the industry. A much harder thing to do is to describe what effect this will have on your organization or exactly what the improvement will look like or when it will be done. Furthermore, security isn’t sexy how you articulate both your current position and a future desired position is especially important, if you want funding to enhance your program or make changes to it. It’s not enough to use (what I like to call scalar words) Good, Bad, Excellent, Needs improvement, because at the end of the day, that’s just your opinion, and in the high stakes game of security, opinions can be dangerous and not get funded, what you really need is the ability to quantify your position.

Quantify Security in Context

There are many different ways to quantify a security position, if I was only interested in how secure the code was, I might employ an aggressive threat modeling strategy. However that would only give me 1 dimension into the problem and if that’s a particular problem area perhaps 1 dimension is all I actually need to solve the problem. However, how many threats are in the code is not going to give me an accurate idea of how mature the software security program is.

Often times, organizations implement a software security program, with some activities and artifacts, these activities do actively work to improve the security posture of the organization. They work well measurable metrics are tracked to quantify the effectiveness of the security initiatives, then organizations argue that security has improved therefore morale has improved and we’re done. These organizations keep up their various security activities they might change from time to time, however there is often little desire to improve and to grow, one might say to mature the overall security initiative.

This often raises two very interesting points, how do you know that the security initiative isn’t the very best that there is? and if what we’re doing right now, is working for us as an organization why do we want to become more mature, especially if (as the assumption goes) it might cost more money? An organization might have the very best security initiative there is out there, they might have solved problems that the world is struggling with, I doubt it, but it’s possible. An organization might have developed a very good security program that, where it can see a measurable output in security bug reduction and penetration test findings and to that, I would say very good. However learning, improvement and maturing can happen in isolation, this is the reason that you send developers and architects to conferences, and new folks join the teams, organizations bring specialty trainings…. all these activities improve the maturing of an organization. So how does an organization know where it needs improvement? Well it’s people might tell it, or it can compare itself to the industry and various other players in the industry do thing. Essentially the organization quantifies its position on a topic and then compares that position to the rest of the industry.

Well security is no different! In order to actually improve, an organization needs to quantify its position against a standard set of dimensions that every other organization will quantify its security position on, and then can the two positions be compared in context. An organization wants to mature its security practices because as an organization becomes more secure in practice, it does those various things faster, and quicker more efficiently & effectively, leading to an ultimate cheaper result all the way around.

Quantify using BSIMM

The Building Security In Maturity Model(BSIMM). Is essentially that a model for helping organizations build security into standard development practice and then maturing along that practice. It lists lots of activities around a standard maturity model, so activities for levels, 1 -> 3. The other effect that it does though is the BSIMM Project works on collecting data points, from companies whom have worked to quantify their maturity within the model. The direct result is that the software industry now has 12 standard data points at various levels of maturity to self assess, which gives an effective way to quantify an organizations position, both relative to the industry and relative to maturity, growth, and year over year security initiatives.

The security data points that each organization make a self assessment on are:

1. Strategy & Metrics
2. Compliance & Policy
3. Training
4. Attack Models (effective threat modeling)
5. Security Features & Design
6. Standards & Requirements
7. Architecture Analysis
8. Code Review
9. Security Testing
10. Penetration Testing
11. Software Environment
12. Config & Vuln Mmgt

The latest version of the report and corresponding positional statements look like:

As you can see, the industry standard (of the organizations that took part) doesn’t really reach a level 3 maturity on the model yet, so there is still some work to do and as an industry, this gives a very clear picture on the skill sets an industry needs to develop in to help organizations improve. It also demonstrates that on average while the industry is starting to take some aspects of security seriously, it clearly is not taking all aspects of security seriously and has some work to improve.

A random company that I generated data for, to illustrate comparisons looks like:

As you can see from this graph, by standardizing on 12 key data points where an organization can quantify maturity levels allows ABC company to understand how they compare to the rest of the software industry with regards to their security position.

Organizations can use this tool and methodology to generate a comprehensive set of data points which are easy to quantify which allows the security engineer to be able to both explain the current security position of the organization and desired positions. This also generates good metrics for analyzing the security position of your organization year over year, or on a more frequent basis, it allows an organization to know exactly where they have to improve skill sets through a combination of training, hiring. This can help you also focus the efforts of your security program over the next time frame. I highly recommend you check out the BSIMM, you don’t have to fully adopt the program, but use the parts of it that make the most amount of sense for your program, organization and objectives.

The post Quantify Your Security Position appeared first on Security Synergy.