fletcher - Measuring the test, not the result... - nCircle Connect

The 2011 FISMA report is out, and I commend it to you ? it?s actually interesting reading.? OMB and DHS are to be commended for their sustained efforts to drive agencies to improve their security performance by pushing them towards continuous monitoring.? After all, it?s a truism in IT security and everywhere else that ?you get what you measure,? and therefore, having agencies demonstrate that they are measuring assets, vulnerabilities and configurations on a monthly basis should be a good thing.? Section 4 of the report shows a dramatic improvement in agencies? adoption of continuous monitoring as measured, in part, by their ability to submit an automated feed to CyberScope.?

?

But, the report raises more questions than it answers, to my mind.? Some of the agencies that have been widely recognized by peers for the effectiveness of their continuous monitoring programs in reducing organizational risk show very poorly in this report.? ?For example the State Department and USAID, both of which have created exemplary, award-winning risk remediation programs on a foundation of strong continuous monitoring, are called out for not submitting Cyberscope data (Figure 4, p.19); and State is near the bottom in terms of their scanning coverage (Figure 5, p.21).? And yet other agencies ? some that are in the headlines all the time with security lapses, and are in the absolute bottom in terms of IT security spending (Figure 16, p.33)-- are posting 95 or 100% coverage.?

?

OMB is measuring continuous monitoring ? but continuous monitoring is a measurement process; in and of itself it does not improve security posture.? Continuous monitoring is only valuable to the degree that it provides the necessary foundation for a program of proactive asset management, risk reduction, and improvement in configuration compliance.? (?check out DHS? CAESARS reference architecture for a great treatise on this topic?)

?

Imagine a program to improve children?s educational performance by conducting a day-long standardized test every month.? Results are immediately posted to the State School Board, but may or may not be available to the teachers and students.? Would you measure the success of such a program by counting how many schools were able to submit 100% of test results?? Of course not ? such a program might be valuable, but only to the degree that the information can be used to improve instruction in the classroom.? ????

?

The 2011 FISMA report shows valuable progress on a number of fronts.? But it doesn?t yet tell us whether continuous monitoring is achieving the desired effects, or even whether the results of the monitoring are being fed back to the people on the ground who can act on the findings.? OMB needs to peel this onion a little further and show us what those scores are based on.? Let?s hope that these 100% implementations of continuous monitoring aren?t like the standardized test example ? using up limited time and resources to achieve compliance to a reporting standard ? but not actually contributing to security.?

Source: http://connect.ncircle.com/t5/Federal-Outlook/Measuring-the-test-not-the-result/ba-p/1962

joseph kony joseph kony higgs boson ipad 3 release date apple store down apple live blog ohio primary

Source: http://okycos.livejournal.com/21828.html

how to carve a turkey how to cook a turkey yorkshire pudding larry the cable guy miracle on 34th street santa tracker monkey bread