Tackling Cybersecurity Data Sprawl Without Normalizing Everything

The Challenge of Cybersecurity Data Management

Cybersecurity teams are facing an unprecedented amount of data, making it increasingly difficult to identify potential threats. The proliferation of cybersecurity tools has led to a surge in data generation, but this has also created a new set of challenges. With so much data at their disposal, teams are struggling to find the needles in the haystack. The introduction of AI-powered tools has attempted to address this issue, but the problem persists.

The idea of normalizing data to a common standard and unifying it into a single repository has been proposed as a solution. This approach suggests that by converting data from different sources into a centralized, standard format, it would be easier to make sense of the data. However, this has proven to be a complex undertaking, and the reality is that most organizations’ data remains unnormalized.

The Limitations of Data Normalization

The concept of data normalization has been around for a while, and it was one of the key selling points of systems like Splunk, Elastic, and MongoDB. These systems allowed users to dump all their data into one big pile and then use advanced algorithms to make sense of it. However, this approach has led to a proliferation of single sources of truth scattered around the enterprise, making it difficult for security analysts to navigate.

The cybersecurity skills shortage has further exacerbated the problem, with newly hired staff struggling to learn which tool does what and how to correlate between different data sources. The effort of maintaining complex extract-transform-load (ETL) toolchains to keep normalized datasets in sync is also taking senior staff away from investigations.

Alternative Approaches

Some vendors, like Komprise, are attempting to automate the process of data normalization and indexing. Microsoft’s Azure Sentinel is also providing multiple views into a centralized data store to suit different ways of working. However, these solutions are not without their limitations, and the reality is that most organizations’ data remains unnormalized.

Cybersecurity startup Crogl has taken a different approach, attempting to mimic the way humans link similar concepts together as they hunt for threats and breaches. By creating a kind of ‘synonym’ index that sits in front of existing systems and datasets, Crogl is able to map similar concepts together and identify where they might be found.

A New Approach to Data Management

Crogl’s approach is centered around the idea of creating a metadata layer that sits on top of existing datasets, allowing users to map back to specific parts of the datasets regardless of where they are resident. This approach makes it easier to find correlations between datasets without having to move or copy the source data first.

The goal is to make finding anomalies within the sea of ordinary and expected data easier, especially when responding to an incident. By providing a layer of assistive tools on top of existing systems, Crogl is able to free up time for security analysts to focus on standardizing where it helps most.

Ultimately, the key to effective cybersecurity data management is to recognize the practical realities that confront us. While standards like the Common Information Model or the Comment Event Format are useful, we should not give up on making progress merely because we do not have a perfect, fully-normalized logging infrastructure. By embracing common standards where we can and using assistive tools to fill the gaps, we can make progress towards more effective cybersecurity data management.