More than 100 years ago, Lord Kelvin observed that measurement is vital to deep knowledge and understanding in physical science. During the last few decades, researchers have made various attempts to develop measures and systems of measurement for computer security with varying degrees of success. This paper provides an overview of the security metrics area and looks at possible avenues of research that could be pursued to advance the state of the art.

Bellovin, S. (2006). On the Brittleness of Software and the Infeasibility of Security Metrics, IEEE Security and Privacy, Volume 4, Issue 4, July-August.

Berinato, S. (2005). A Few Good Information Security Metrics, CSO Magazine , http://www.csoonline.com/article/220462/A_Few_Good_Information Security_Metrics?contentId=220462&slug=&

Blaze, M. (2004). Safecracking for the Computer Scientist, Draft Document, http://www.crypto.com/papers/safelocks.pdf

Bowring, J., Orso, A., Harrold, M. (2002). Monitoring Deployed Software Using Software Tomography, ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, Charleston, South Carolina

Brenner, B. (2007). Windows Admins Feel Post-Patch Tuesday Pain, SearchSecurity.com, October 19, 2007, http://searchsecurity.techtarget.com/news/article/0,289142,sid14_gci1277683,00.html

Carin, L., Cybenko, G., Hughes, J. (2008). Cybersecurity Strategies: The QuERIES Methodology, IEEE Computer, Vol. 41, No. 8

Center for Internet Security (CIS) (2008). The CIS Security Metrics Service, http://securitymetrics.org/content/attach/Metricon3.0/metricon3-kreitner%20handout.pdf

Chandra, P., Chess, B., Steven, J. (2006). Putting the Tools to Work: How to Succeed with Source Code Analysis, IEEE Security & Privacy, vol. 4, no. 3, pp. 80-83

Chen, H., Wang, F. Y. (2005). Artificial Intelligence for Homeland Security, IEEE Intelligent Systems, vol. 20, no. 5, pp. 12-16

Commission of the European Communities (CEC) (1991). Information Technology Security Evaluation Criteria (ITSEC), Harmonised Criteria of France - Germany - the Netherlands - the United Kingdom, CEC Directorate XIII/F SOG-IS, http://www.iwar.org.uk/comsec/resources/standards/itsec.htm

Common Criteria Portal (2006). Common Criteria for Information Technology Security Evaluation Part 1: Introduction and general model, Version 3.1 Revision 1, http://www.commoncriteriaportal.org/files/ccfiles/CCPART1V3.1R1.pdf

Department of Defense (DoD) (1985). Trusted Computer System Evaluation Criteria, DoD 5200.28-STD, http://csrc.nist.gov/publications/history/dod85.pdf

Dondo, M. (2007). A Fuzzy Risk Calculations Approach for a Network Vulnerability Ranking System, Technical Memorandum 2007-090, Defence R&D Canada – Ottawa, http://www.ottawa.drdc-ddc.gc.ca/docs/e/TEOTM-2007-090.pdf

Dougherty, C. (2008a). Debian and Ubuntu OpenSSL Packages Contain a Predictable Random Number Generator, Vulnerability Note VU#925211, U.S. Computer Emergency Readiness Team, https://www.kb.cert.org/vuls/id/925211

Dougherty, C. (2008b). Multiple DNS Implementations Vulnerable to Cache Poisoning, Vulnerability Note VU#800113, U.S. Computer Emergency Readiness Team, http://www.kb.cert.org/vuls/id/800113

Dowd, M. (2008). Application-Specific Attacks: Leveraging the ActionScript Virtual Machine, IBM Global Technology Services, http://documents.iss.net/whitepapers/IBM_X-Force_WP_final.pdf

Figueroa, J. (2009). Discovery Systems Check Their Own Facts, In the News, IEEE Intelligent Systems, Vol. 24, No. 3

Garfinkel, S. (2008). Alarming Open-Source Security Holes: How a programming error introduced profound security vulnerabilities in millions of computer systems, MIT Technology Review, http://www.technologyreview.com/Infotech/20801/?a=f

Gray, M. (1999). Applicability of Metrology to Information Technology, Journal of Research of the National Institute of Standards and Technology, Vol. 104, No. 6, http://nvl.nist.gov/pub/nistpubs/jres/104/6/j46gra.pdf

Guelev, D. P., Ryan, M., Schobbens, P. Y. (2004). Model Checking Access Control Policies, Proceedings of the 7th Information Security Conference, Palo Alto, CA

Henning, R., et al. (2001). Proceedings of the Workshop on Information Security System Scoring and Ranking, Applied Computer Security Associates, Williamsburg, Virginia, http://www.acsac.org/measurement/proceedings/wisssr1-proceedings.pdf

INFOSEC Research Council (2005). Hard Problem List, http://www.cyber.st.dhs.gov/docs/IRC_Hard_Problem_List.pdflivepage.apple.com

The Institute for Information Infrastructure Protection (I3P) (2009). National Cyber Security Research and Development Challenges Related to Economics, Physical Infrastructure and Human Behavior: An Industry, Academic and Government Perspective, http://www.thei3p.org/docs/publications/i3pnationalcybersecurity.pdf

International Systems Security Engineering Association (ISSEA) (2008). SSE-CMM: Systems Security Engineering Capability Maturity Model, http://www.sse-cmm.org/metric/metric.asp

Jelen, G. (2000). SSE-CMM Security Metrics, The National Institute of Standards and Technology (NIST) and Computer System Security and Privacy Advisory Board (CSSPAB) Workshop, Washington, D.C.

Juranić, L. (2006). Using fuzzing to Detect Security Vulnerabilities, INFIGO-TD-01-04-2006, Infigo Information Security, http://www.infigo.hr/files/INFIGO-TD-2006-04-01-Fuzzing-eng.pdf

Kaksonen, R. (2001). A Functional Method for Assessing Protocol Implementation Security, VTT Publications 448, Technical Research Centre of Finland, http://www.vtt.fi/inf/pdf/publications/2001/P448.pdf

Keizer, G. (2008). Hackers Attack Newest Windows Patch, PC World, http://www.pcworld.com/businesscenter/article/144486/hackers_attack_newest_windows_patch.html

Kirkland, D., Salem, L. (2006). BogoSec: Source Code Security Quality Calculator, IBM, http://download.boulder.ibm.com/ibmdl/pub/software/dw/linux/l-bogosec.pdf

Lemos, R. (2008). Patches Pose Significant Risk, Researchers Say, SecurityFocus, http://www.securityfocus.com/news/11514

Liblit, B. (2004). Cooperative Bug Isolation, PhD Thesis, University of California, Berkeley, http://pages.cs.wisc.edu/~liblit/dissertation/dissertation.pdf

Littlewood, B. et al. (1993). Towards Operational Measures of Computer Security, Journal of Computer Security, vol. 2, no. 2-3, pp. 211-230

Manadhata, P., Wing, J. M. (2005). An Attack Surface Metric, CMUCS-05-155, Carnegie Mellon University, http://reports-archive.adm.cs.cmu.edu/anon/2005/CMU-CS-05-155.pdf

Manadhata, P., Tan, K., Maxion, R., Wing, J. (2007). An Approach to Measuring a System’s Attack Surface, CMU-CS-07-146, Carnegie Mellon University, http://reports-rchive.adm.cs.cmu.edu/anon/2007/CMUCS-07-146.pdf

Marco, L. (1997). Measuring Software Complexity, Enterprise Systems Journal, http://cispom.boisestate.edu/cis320emaxson/metrics.htm 

Markoff, J. (2008). Leaks in Patch for Web Security Hole, The New York Times, http://www.nytimes.com/2008/08/09/technology/09flaw.html?_r=1&oref=slogin

McGill, W., Ayyub, B. M. (2007). Multicriteria Security System Performance Assessment Using Fuzzy Logic, The Journal of Defense Modeling and Simulation (JDMS): Applications, Methodology, Technology, Special Issue: Homeland Security, vol. 4, no. 4, http://www.scs.org/pubs/jdms/vol4num4/McGill.pdf

Michael, C., Lavenhar, S. (2006). Source Code Analysis Tools – Overview, Cigital, Inc., https://buildsecurityin.us-cert.gov/daisy/bsi/articles/tools/code/263-BSI.html

Nagel, B. (2008). Excel Patch Causes Miscalculations, Government Computer News, http://www.gcn.com/online/vol1_no1/45992-1.html

Neuhaus, S., Zimmermann, T., Holler, C., Zeller, A. (2007). Predicting Vulnerable Software Components, ACM Conference on Computer and Communications Security (CCS '07), Alexandria, Virginia, http://www.st.cs.uni-sb.de/publications/files/neuhaus-ccs-2007.pdf

Ozment, A., Schechter, S. (2006). Milk or Wine: Does Software Security Improve with Age?, 15th USENIX Security Symposium, Vancouver, Canada, http://www.usenix.org/events/sec06/tech/full_papers/ozment/ozment.pdf

Poulsen, K. (2008). Researchers Use PlayStation Cluster to Forge a Web Skeleton Key, Wi red Magazine, http://blog.wired.com/27bstroke6/2008/12/berlin.html

Reid, G., Mell, P., Scarfone, K. (2007). CVSS-SIG Version 2 History, Forum of Incident Response and Security Teams, http://www.first.org/cvss/history.html

Reith, M., Niu, J., Winsborough, W. (2007). Apply Model Checking To Security Analysis in Trust Management, C107-0030, University of Texas at San Antonio , http://stinet.dtic.mil/cgi-bin/GetTRDoc?AD=ADA462754&Location=U2&doc=GetTRDoc.pdf

Röning, J., Laakso, M., Takanen, A., Kaksonen, R. (2002). PROTOS - Systematic Approach to Eliminate Software Vulnerabilities, Invited presentation at Microsoft Research, Seattle, Washington, http://www.ee.oulu.fi/research/ouspg/protos/sota/MSR2002-protos/index.html

Savola, R. M. (2007). Towards a Taxonomy for Information Security Metrics, International Conference on Software Engineering Advances (ICSEA 2007), Cap Esterel, France

Schwarz, B., Chen, H., Wagner, D., Morrison, G., West, J. (2005). Model Checking an Entire Linux Distribution for Security Violations, 21st Annual Computer Security Applications Conference, Tucson, Arizona, http://www.acsac.org/2005/papers/165.pdf

Science Applications International Corporation (SIAC) (2007). Microsoft Windows Server 2003, XP Professional and XP Embedded Security Target, Version 3.0, SIAC Common Criteria Testing Laboratory, http://www.commoncriteriaportal.org/files/epfiles/20080303_st_vid10184-st.pdf

Shah, S. (2003). Measuring Operational Risk Using Fuzzy Logic Modeling, International Risk Management Institute, Inc. (IRMI), http://www.irmi.com/Expert/Articles/2003/Shah09.aspx

Storms, A. (2008). Many Microsoft Bulletins Replaced; Bigger Set of Kill Bits Issued, nCircle, http://blog.ncircle.com/blogs/sync/archives/2008/08/many_microsoft_bulletins_repla.html

Torgerson, M. (2007). Security Metrics, 12th International Command and Control Research and Technology Symposium, Newport, Rhode Island, http://www.dodccrp.org/events/12th_ICCRTS/CD/html/presentations/108.pdf

Torgerson, M. (2007). Security Metrics for Communication Systems, 12th International Command and Control Research and Technology Symposium, Newport, Rhode Island, http://www.dodccrp.org/events/12th_ICCRTS/CD/html/papers/108.pdf

Vaughn Jr., R., Henning, R., Siraj, A. (2002). Information Assurance Measures and Metrics – State of Practice and Proposed Taxonomy, 30th Hawaii International Conference on System Sciences, Big Island, Hawaii, http://csdl2.computer.org/comp/proceedings/hicss/2003/1874/09/187490331clivepage.apple.com.pdf