Machine learning, advanced search boosts cybersecurity

This content is provided by Elastic

By David Erickson, Director of Solutions Architecture, Elastic

Government cybersecurity teams are outgunned and outmatched by adversaries who have become increasingly sophisticated, aggressive, and disruptive. They’re looking to pivot defensive operations from a reactive, signature-based process to one focused on intelligence gathering and active threat hunting in order to deal with the new wave of sophisticated adversaries.

It’s a large change, and the tools used to collect and analyze data have to change to support it.

Government technologists need to follow the adversary’s trail by asking questions of the massive amounts of data in a fluid and agile way. Finding these well-hidden adversaries and tracking their activity is fundamentally a search problem.

Search engines allow analysts to ask ad hoc questions and cull more relevant answers from structured or unstructured data. Distributed search systems like Elasticsearch allow analysts to move quickly through their investigation, even for incredibly large volumes of information. Adding machine learning capabilities to the mix enables security teams to scale further and do more with the petabytes of data coming from their cybersecurity centers. It helps analysts address the major challenges, like alert fatigue and expanding attack surfaces, that come with dealing with billions of security events..

Alert fatigue and expanded attack surfaces

Security analysts are no longer responding to just five to ten incidents, but to tens or even hundreds of thousands of incidents each day. It is very easy to become overburdened with event waves. Additionally, attackers are innovating, and the most damaging attacks are utilizing never-before-seen exploits of weaknesses in our systems. Finding the “unknown unknown” is not a job for signature-based scanners. It requires detection systems that can learn and adapt.

The emerging migration to cloud architectures only serves to compound these problems as SOC teams must contend with expanded attack surfaces. Deployment options like cloud computing allow organizations to spin up large infrastructure projects in a matter of days. It’s great for the agility and cost effectiveness of government programs, but serving a defensive role poses new challenges in a changing and now often cross-cloud environment.

The holistic view

There is simply too much for human eyes to watch over. Eyes-on-glass approaches fail, and a new generation of operations assisted by artificial intelligence and automated response are needed to support defensive cyber operations. The goal is to look at all of the data in real time without having to leave any on the floor or unevaluated due to cost or time constraints. Enter machine learning.

Machine learning features in the Elastic Stack automatically model the behavior of an organization’s Elasticsearch data in real time to identify issues faster, streamline root cause analysis, and reduce false positives. With machine learning, a small team of operators can do what it would have taken hundreds of people to achieve ten years ago. Combining machine learning-based detection with powerful search capabilities allows analysts to sift through all the data they would have missed and use it to generate insights  that strengthen their cyber defenses.

Elastic’s  machine learning algorithms use unsupervised methods that can detect anomalies in Elasticsearch time series data without requiring a labeled training set. Instead of responding to threats after the fact by creating algorithms to detect a similar type of attack in the future, Elastic machine learning features zero in on what is weird, different, or anomalous — even the first time a new attack pattern occurs. Adversaries know what’s been seen before, so they are trying new ways of attacking. Machine learning zooms in on when a user or software started to behave differently. An unusual pattern of data transfer activity from an employee’s machine might be a sign of an insider threat or a sign of malware moving through the network.

Using advanced search and machine learning, analysts can look at the entire history of an incident when it does happen. Elasticsearch can go back days, weeks, and even months, to see what has occurred in the past. After detecting anomalous behavior, analysts can verify what is good or bad using their expertise or domain of knowledge. They can cut through all of the noise and hundreds of thousands of alerts because the machine learning algorithm prioritizes the incidents that they should address.

What’s more, security operators can gain better visibility into their network and cloud environments using Beats, Elastic’s lightweight open source data shippers. These sensors can be deployed everywhere – virtual servers, servers throughout a network, or on cloud resources. Deployed on cloud resources, Beats gives security operators a lower-level point of view on what is happening across their network. Beats can centralize information per cloud environment and send a data feed back to Elasticsearch, where machine learning algorithms can spot anomalies and search powers the post-detection investigation. The result is increased visibility into what is happening across an ever-expanding attack surface.

The bottom line

IT managers and security operations teams need visibility and situational awareness over their entire infrastructure. They need a holistic view of their data centers and networks that are connecting into multiple clouds. They must correlate threats that attack both on-premises systems and their cloud resources. Defeating that type of advanced, persistent threat requires a comprehensive view – not small siloes where analysts are viewing information across 10 different screens. Elastic’s blend of search and machine learning capabilities along with its suite of analytics, data capture, sensor, and visualization tools give government agencies the arsenal that they need to meet their adversaries head on and stay a step ahead of cyber threats.

About Elastic

Elastic is a search company. As the creators of the Elastic Stack (Elasticsearch, Kibana, Beats, and Logstash), Elastic builds self-managed and SaaS offerings that make data usable in real time and at scale for search, logging, security, and analytics use cases. Since its founding in 2012, there have been more than 250 million cumulative downloads of Elastic software. Elastic is a distributed company with more than 1,000 Elasticians in 30 countries. Learn more at elastic.co.  

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.