Combatting Cyber Alert Fatigue

Distinguishing Important Alerts from Noise

Everyone experiences alert fatigue.

Alert fatigue is such a common problem that it dates as far back as Aesop’s Fable from the 6th century BC:

There once was a shepherd boy who was bored as he sat on the hillside watching the village sheep.
To amuse himself he took a great breath and sang out, ‘Wolf! Wolf! The Wolf is chasing the sheep!’ 
After repeatedly raising the alarm – for his own entertainment – the villagers began to ignore
the boy. So, when a wolf really was chasing the sheep … no one listened.  

Just like the boy who cried wolf, cyber alerts wear down analysts and SecOps teams. The more alerts that go off, especially if they appear to be false, the more likely people are to ignore them.

The story’s message still holds true today: alert fatigue gets to everyone, no matter how diligent the team strives to be.

The Car Alarm Problem

When was the last time you reacted to a car alarm?

Car alarms were designed to deter thieves. In car alarm utopia, the family is quietly eating dinner in their home when a dastardly thief attempts to steal their vehicle. The alarm sounds as the thief attempts to open a door. The head of the family rushes outside to confront the bad guy.

In car alarm reality, the alarm sounds and the entire neighbourhood rolls their eyes, mutters a few choice words about annoying car alarms, and waits for the obnoxious sound to desist.

Your staff treats most cybersecurity tools and their alerts just like car alarms.

No matter what the vendor promises, there is always an explicit trade-off between the number of false alerts received and the number of security breaches your team misses. Without going into minute details about the math behind type I and type II errors and their relationships for cybersecurity monitoring, the general shape of the false positive and false negatives errors resembles the curves in Figure 1.

Alert Desensitization and Error Rate Trade-Off

Increasing sensor sensitivity ensures fewer important alerts will be missed (see red descending line); however, that mean more false negatives. This occurs when a cybersecurity tool indicates an action is not an attack when it actually is.

Reducing sensor sensitivity prompts fewer false positive alerts, and also has a higher rate of missed attacks (see ascending green line). More alerts do not mean better security, after all. An alarm generated when an attack is not taking place simply irritates staff.

Better engineering can alter the slope of these curves, but the trade-off always exists. To catch ALL attacks, a team must investigate many alerts that could be attacks, but actually aren’t.

Tips for Reducing Alert Fatigue

What can we do to deal with alarm fatigue?

Option #1: Act Only on Confirmed Alerts

The first strategy is to only investigate alerts that are 100% confirmed.

If the wolf is chasing the flock every time the boy alerts the village, the villagers will still show up when the boy cries wolf. Even for the Nth time.

A cybersecurity team can follow this approach by relying only on sensors that are highly indicative of malicious activity (like an AV finding malware in the network) or high-quality indicators of compromise.

Unfortunately, this leaves an operation exposed to missed attacks. For example, the attacks that bypass AV or for which no IoCs exist yet. That means sensors will miss the more novel, unknown attacks.

Option #2: Invest in Sensor Tuning

The second strategy is to invest time and effort in tuning sensors.

This generally means constantly tweaking sensor configuration to improve detection and remove sources of false positives. There will still be a trade-off between false positives and false negatives; however, the curves can be “sharpened” to get a better deal on the trade-off.

With highly tuned sensors, a team gets very few false negatives, while dealing with a manageable number of false positives.

Note that this process is extremely resource and time consuming.

Option #3: Outsource Alert Management

The third and final option is to outsource alert management.

By adopting a managed detection service, a 3rd party service attends to the alerts and only signals for their client’s attention when alerts are deemed critical.

Like in option #1, the team will never tire from responding to alerts – because the alert is always signalling a threat is imminent. In this option, your team choses to work with cybersecurity professionals who do the work of constant tuning for significantly lower price than having someone on staff to do it.

Managing Alert Fatigue with SMART-Monitor

CYDEF’s SMART-Monitor service combines the work of processing alerts and tuning the baseline for greater efficiency. Our machine learning emulates operator decisions to filter trustworthy behaviour to reduce future false positives - but only after a live, human expert initially vets the program behaviour. That helps to avoid false negatives.

When outsourcing alert management, ensure that your service provider is truly addressing the volume of alerts and not ignoring them or shipping you all the false positives. Look out for our blog post (coming soon) providing tips on measuring your managed service response to alerts!