You are here: Home Contents V7 N1 V7N1_Jansen.html
Personal tools

Research Directions in Security Metrics



Full text

Journal of Information System Security
Volume 7, Number 1 (2011)
Pages 322
ISSN 1551-0123
Wayne Jansen — National Institute of Standards and Technology, USA
Information Institute Publishing, Washington DC, USA




More than 100 years ago, Lord Kelvin observed that measurement is vital to deep knowledge and understanding in physical science. During the last few decades, researchers have made various attempts to develop measures and systems of measurement for computer security with varying degrees of success. This paper provides an overview of the security metrics area and looks at possible avenues of research that could be pursued to advance the state of the art.




Security Metrics, Computer Security, Security Evaluation




Bellovin, S. (2006). On the Brittleness of Software and the Infeasibility of Security Metrics, IEEE Security and Privacy, Volume 4, Issue 4, July-August.

Berinato, S. (2005). A Few Good Information Security Metrics, CSO Magazine , Security_Metrics?contentId=220462&slug=&

Blaze, M. (2004). Safecracking for the Computer Scientist, Draft Document,

Bowring, J., Orso, A., Harrold, M. (2002). Monitoring Deployed Software Using Software Tomography, ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, Charleston, South Carolina

Brenner, B. (2007). Windows Admins Feel Post-Patch Tuesday Pain,, October 19, 2007,,289142,sid14_gci1277683,00.html

Carin, L., Cybenko, G., Hughes, J. (2008). Cybersecurity Strategies: The QuERIES Methodology, IEEE Computer, Vol. 41, No. 8

Center for Internet Security (CIS) (2008). The CIS Security Metrics Service,

Chandra, P., Chess, B., Steven, J. (2006). Putting the Tools to Work: How to Succeed with Source Code Analysis, IEEE Security & Privacy, vol. 4, no. 3, pp. 80-83

Chen, H., Wang, F. Y. (2005). Artificial Intelligence for Homeland Security, IEEE Intelligent Systems, vol. 20, no. 5, pp. 12-16

Commission of the European Communities (CEC) (1991). Information Technology Security Evaluation Criteria (ITSEC), Harmonised Criteria of France - Germany - the Netherlands - the United Kingdom, CEC Directorate XIII/F SOG-IS,

Common Criteria Portal (2006). Common Criteria for Information Technology Security Evaluation Part 1: Introduction and general model, Version 3.1 Revision 1,

Department of Defense (DoD) (1985). Trusted Computer System Evaluation Criteria, DoD 5200.28-STD,

Dondo, M. (2007). A Fuzzy Risk Calculations Approach for a Network Vulnerability Ranking System, Technical Memorandum 2007-090, Defence R&D Canada – Ottawa,

Dougherty, C. (2008a). Debian and Ubuntu OpenSSL Packages Contain a Predictable Random Number Generator, Vulnerability Note VU#925211, U.S. Computer Emergency Readiness Team,

Dougherty, C. (2008b). Multiple DNS Implementations Vulnerable to Cache Poisoning, Vulnerability Note VU#800113, U.S. Computer Emergency Readiness Team,

Dowd, M. (2008). Application-Specific Attacks: Leveraging the ActionScript Virtual Machine, IBM Global Technology Services,

Figueroa, J. (2009). Discovery Systems Check Their Own Facts, In the News, IEEE Intelligent Systems, Vol. 24, No. 3

Garfinkel, S. (2008). Alarming Open-Source Security Holes: How a programming error introduced profound security vulnerabilities in millions of computer systems, MIT Technology Review,

Gray, M. (1999). Applicability of Metrology to Information Technology, Journal of Research of the National Institute of Standards and Technology, Vol. 104, No. 6,

Guelev, D. P., Ryan, M., Schobbens, P. Y. (2004). Model Checking Access Control Policies, Proceedings of the 7th Information Security Conference, Palo Alto, CA

Henning, R., et al. (2001). Proceedings of the Workshop on Information Security System Scoring and Ranking, Applied Computer Security Associates, Williamsburg, Virginia,

INFOSEC Research Council (2005). Hard Problem List,

The Institute for Information Infrastructure Protection (I3P) (2009). National Cyber Security Research and Development Challenges Related to Economics, Physical Infrastructure and Human Behavior: An Industry, Academic and Government Perspective,

International Systems Security Engineering Association (ISSEA) (2008). SSE-CMM: Systems Security Engineering Capability Maturity Model,

Jelen, G. (2000). SSE-CMM Security Metrics, The National Institute of Standards and Technology (NIST) and Computer System Security and Privacy Advisory Board (CSSPAB) Workshop, Washington, D.C.

Juranić, L. (2006). Using fuzzing to Detect Security Vulnerabilities, INFIGO-TD-01-04-2006, Infigo Information Security,

Kaksonen, R. (2001). A Functional Method for Assessing Protocol Implementation Security, VTT Publications 448, Technical Research Centre of Finland,

Keizer, G. (2008). Hackers Attack Newest Windows Patch, PC World,

Kirkland, D., Salem, L. (2006). BogoSec: Source Code Security Quality Calculator, IBM,

Lemos, R. (2008). Patches Pose Significant Risk, Researchers Say, SecurityFocus,

Liblit, B. (2004). Cooperative Bug Isolation, PhD Thesis, University of California, Berkeley,

Littlewood, B. et al. (1993). Towards Operational Measures of Computer Security, Journal of Computer Security, vol. 2, no. 2-3, pp. 211-230

Manadhata, P., Wing, J. M. (2005). An Attack Surface Metric, CMUCS-05-155, Carnegie Mellon University,

Manadhata, P., Tan, K., Maxion, R., Wing, J. (2007). An Approach to Measuring a System’s Attack Surface, CMU-CS-07-146, Carnegie Mellon University,

Marco, L. (1997). Measuring Software Complexity, Enterprise Systems Journal, 

Markoff, J. (2008). Leaks in Patch for Web Security Hole, The New York Times,

McGill, W., Ayyub, B. M. (2007). Multicriteria Security System Performance Assessment Using Fuzzy Logic, The Journal of Defense Modeling and Simulation (JDMS): Applications, Methodology, Technology, Special Issue: Homeland Security, vol. 4, no. 4,

Michael, C., Lavenhar, S. (2006). Source Code Analysis Tools – Overview, Cigital, Inc.,

Nagel, B. (2008). Excel Patch Causes Miscalculations, Government Computer News,

Neuhaus, S., Zimmermann, T., Holler, C., Zeller, A. (2007). Predicting Vulnerable Software Components, ACM Conference on Computer and Communications Security (CCS '07), Alexandria, Virginia,

Ozment, A., Schechter, S. (2006). Milk or Wine: Does Software Security Improve with Age?, 15th USENIX Security Symposium, Vancouver, Canada,

Poulsen, K. (2008). Researchers Use PlayStation Cluster to Forge a Web Skeleton Key, Wi red Magazine,

Reid, G., Mell, P., Scarfone, K. (2007). CVSS-SIG Version 2 History, Forum of Incident Response and Security Teams,

Reith, M., Niu, J., Winsborough, W. (2007). Apply Model Checking To Security Analysis in Trust Management, C107-0030, University of Texas at San Antonio ,

Röning, J., Laakso, M., Takanen, A., Kaksonen, R. (2002). PROTOS - Systematic Approach to Eliminate Software Vulnerabilities, Invited presentation at Microsoft Research, Seattle, Washington,

Savola, R. M. (2007). Towards a Taxonomy for Information Security Metrics, International Conference on Software Engineering Advances (ICSEA 2007), Cap Esterel, France

Schwarz, B., Chen, H., Wagner, D., Morrison, G., West, J. (2005). Model Checking an Entire Linux Distribution for Security Violations, 21st Annual Computer Security Applications Conference, Tucson, Arizona,

Science Applications International Corporation (SIAC) (2007). Microsoft Windows Server 2003, XP Professional and XP Embedded Security Target, Version 3.0, SIAC Common Criteria Testing Laboratory,

Shah, S. (2003). Measuring Operational Risk Using Fuzzy Logic Modeling, International Risk Management Institute, Inc. (IRMI),

Storms, A. (2008). Many Microsoft Bulletins Replaced; Bigger Set of Kill Bits Issued, nCircle,

Torgerson, M. (2007). Security Metrics, 12th International Command and Control Research and Technology Symposium, Newport, Rhode Island,

Torgerson, M. (2007). Security Metrics for Communication Systems, 12th International Command and Control Research and Technology Symposium, Newport, Rhode Island,

Vaughn Jr., R., Henning, R., Siraj, A. (2002). Information Assurance Measures and Metrics – State of Practice and Proposed Taxonomy, 30th Hawaii International Conference on System Sciences, Big Island, Hawaii,