Developing & Testing a New Data Visualization Approach
Too Much Data
The more we use technology the more data we create. In fact, every day we create over 2.5 quintillion bytes of data.
At a glance, this sounds great. The more data we have about anything, the more we can understand and learn. However, industries are reaching a point where there is just too much data for them to analyze. Especially in cybersecurity.
In cyber-security, analysts look at all the different devices connected to their network in order to help identify threats and attacks. They look at when new devices join the network and how they are connected, along with many, many other pieces of data. This can be very difficult to keep track of when you’re watching over a large network, such as a large company.
At the time, there were tools available to help visualize this data to make them more digestible. However, once the network reaches a certain size, the visualizations became unintelligible and ended up looking like a giant hairball. Yuck.
Mapping the Solution
Our client had done a lot of research into the space to identify why the current solutions were not up to par. Most solutions were taking a “force-direct graph” approach. Again, this worked well for visualizing small networks, but as the network gets bigger, the visualization gets more unintelligible.
The better solution was to take a “Circle Packing” approach. This design allowed for a clearer understanding of how data is organized as well as great interaction with thousands of data points through zooming, filtering, and highlighting.
They had built a prototype using this new approach and, even though it was an improvement from other solutions, it still needed to be a turnkey solution for giant data sets. They needed a productized solution and asked Taivara to build it for them.
Building through Experimentation
Using D3.js we experimented with a few different layouts that retained relative node positions for our data sets. After getting close with a few, but not close enough, we built a custom algorithm to do the trick.
The prototype had unknown devices (read: potential threats) orphaned in the diagram. At a large scale, these unknown devices were scattered throughout and unrecognizable. We played with a few options but decided on grouping them together so that defenders can find all threats in one place.
We also tested various different zooming, panning, and filtering interactions to find the right balance of each to make navigating the diagram easy.
To do all this we used a combination of D3.js, React, Redux, RxJS, and Ramda.js to make the UI clean and snappy.
FYI - This product is open source
If you’re interested in creating your own solution, contact us and we can get you started.
After launch, the product received great feedback from the cybersecurity industry including many experts in the field. We continue to update and improve the product as well as apply it to other industries outside of cybersecurity.
More Client Success Stories
Our client had invested millions in developing a product but struggled to get to market. Learn how our innovation coaches led them down the path toward success.
Our client was well-experienced in science and technology R&D but needed help commercializing their intellectual property in new markets. Learn how our approach helped bridge that gap.
Learn how we took a senior thesis project and turned it into an acquired startup.
Beauty stores are always collecting information about their customers to help drive recommendations and sales. Their old process was hindering them from growing until we introduced the solution.
After identifying potential markets to pursue, a client re-engaged Taivara to test their market assumptions and identify the keys to success.
A client wanted to know if they could turn existing intellectual property into a product and sell it. Read how we uncovered their hidden market potential.