Roland Daccache, Senior Regional Sales Engineer MENA, Fidelis Cybersecurity explains its time to rethink how we measure effectiveness & efficiency of security operations
CISOs have become leaders in their businesses rather than just experts in their departments. They need to educate their peers on the scope, scale, severity and solutions for cybersecurity and how emerging threats affect each aspect of the business, elevate the cybersecurity discussion out of the trenches of speeds, feeds and fingerprints and finally report on evolving metrics that impact the bottom line of the business to facilitate rapid decision making by the rest of their executive peers.
A recent report from the SANS institute found that 71% of organizations do not have regular metrics for or even measure incident response performance, process and effectiveness. Without metrics there is no objective way to determine progress. Enter Automated Detection & Response (ADR). A unified, ADR platform that provides its own broad and unique visibility across networks and endpoints, uses a variety of different but coordinated techniques to detect threats at any stage of the attack lifecycle, automatically correlates and validates the impact of the threat, and consolidates redundant or related security events in to a single “conclusion” and gives security operations analysts all the information, context, guidance and tools they need to investigate, contain and remediate the attack.
As such, the new thinking of ADR enables new metrics that drive results, that impact not only security posture, but also the bottom line of the business, as detailed below.
Cost Per Incident (CPI)
CPI can be measured as (the time per incident) x (average hourly rate for a Tier 1 analyst). To get a baseline, run that formula through your IR playbook for each phase of a response from detection, decision to escalation & investigation to response determination to response and remediation execution. Then run it again with an ADR platform in place in a Proof of Concept (POC) or even as a table-top exercise. A further extension of this metric involves the empowerment of tier 1 and 2 analysts. When Tier 1 and 2 analysts are empowered with an ADR Platform to perform or augment the work of a Tier 3 analyst (a very scarce resource!), then substantial effectiveness savings can be quantified.
Cost Per Workflow
Review, investigation and response workflows are both personnel and technology-dependent. Automation reduces personnel and technology dependencies. Reducing technology dependencies decreases personnel maintenance requirements. Thus, automation impacts personnel cost, technology cost, and maintenance cost. Leaders will see that entire steps of their workflows are able to be reduced or eliminated completely; delivering massive acceleration, huge savings and massive efficiency boosts as teams can focus on the validation of real incidents rather than wasting time on a wild goose chase.
Automatic Detection vs Manual Detection
Establish a baseline for determining the ratio of detections your security stack produces vs the combined number of human detections you receive. To figure out the human detections, determine the number of staff detections (e.g. an employee recognizes that their machine is malfunctioning, or an IT Admin recognizes that a system is performing in unusual ways) plus the number of external detections (e.g. the number of times you get a call from the authorities /IT Admins) plus the number of detections your Security Operations staff create by manually synthesizing data from your security stack and Security Event and Incident Management (SEIM). This will give you a sense of the efficiency of your current system. With ADR you can expect the ratio to tilt substantially toward the automation side of the equation which means substantially better security operations efficiency.
Percent Investigation vs Volume
Determine what is slipping through the cracks. By measuring investigations versus alert volume, you can get a sense for what might be slipping through the cracks and creating risk. With the ADR system you should expect to see a shrinking gap and massive improvement. For example, if an organization is typically performing 3 investigations for every 100 alerts (3/100 or 3%) and then implements an ADR which sees a 10% alert-to-conclusion rate and an additional 2 investigations (5/10 or 50%) that can yield a massive 1500% increase to security operations effectiveness.
Ratio of Investigation to Response
This metric shows how many items that were investigated lead to a response workflow going through completion. The ratio indicates where Security Operations teams may be wasting time. If an investigation is started and then abandoned due to lack of context, insight or actionable intelligence, then time and resources are not only wasted, but the result is a huge opportunity cost in lost time and loss of focus on threats and attacks that are actionable. Organizations that implement an ADR platform should expect to see a convergence of “investigations-to-response” since more investigations are against validated conclusions rather than merely suspected attacks.
Rate of Validation
This metric measures the time it takes to make a decision. Analysis paralysis and Security Operations uncertainty increases dwell time and risks the spread of an attack. It also takes time away from investigating and responding to other attacks or compromises that may be happening at the same time. By measuring the decision rate both before and after implementing an ADR platform, the Security Operations team is able to demonstrate agility and increased response capacity without adding scarce people resources.
Remediation Response vs Reimage
This metric measures business disruption. Disrupted business means substantially higher cost from delays, lost productivity or even liability to third parties. The more surgical and remote responses that are enabled by the ADR platform, the fewer “big hammer” fixes of reimaging an end-user’s endpoint have to happen. That means less business disruption and inconvenience for employees. Business disruption can be quantified based on the staff role, affected device role and length of time for a response. Taking someone’s laptop for a day to reimage it is an inconvenience. Taking down a payment processing server is a substantial disruption – even when hot backups and clustered failovers are part of the solution.
The ADR approach thinks differently about security operations. ADR is based on a purpose-built platform designed to deliver validated conclusions about attacks, intrusions and compromises at any stage of the attack lifecycle while also automating the response capability to those attacks. This transformation enables new metrics that impact the organizations’ business and bottom line. Each of these metrics point to the potential and necessity of adopting an ADR approach and making it the cornerstone of your cybersecurity strategy in 2018 and beyond.