“Dark” data–sounds foreboding, doesn’t it? It’s not as mysterious as it sounds. This is actually just an umbrella term for the immense stockpile of untapped, neglected data that exists within an organization’s reach, but hasn’t been explored yet.
Until recently, the large amounts of unstructured data from sources such as audio and video feeds, transactional systems, and the “deep web” were all but impossible to analyze for insights. Now, new opportunities are opening up as advanced analytics technologies allow businesses to unearth critical operational insights and forecasts from a treasure trove of data they once thought of as “dark” and inaccessible.
According to Deloitte’s recently released report, “Tech Trends 2017: The Kinetic Enterprise,” the time for companies to utilize dark analytics is now. Bill Briggs, global chief technology officer of Deloitte Consulting LLP, breaks it down.
When you talk about dark analytics, what exactly does that include?
So, there are three big categories. The first one is probably the highest of importance for a big organization, which is the data that already exists in the transactional systems inside a company. Because it is owned by different divisions, or in different systems, traditionally, it’s been hard to pull together into a complete view of a customer or product or performance and tap its full potential.
The second is unstructured, nontraditional data like audio, images, and video that may live within the organization–or potentially can be accessed by external sources–and we can suddenly do something with it.
The third category is the “deep web,” which allows organizations to tap into academic information or communities of expertise that aren’t indexed and available on the broader public internet. So, by looking at the deep web, finance companies, for example, can get signals on investments and market shifts that might be there. But this is definitely still in the early days.
So why is this relevant to companies now? What has changed?
A big part of it is that we have advances in computer vision, pattern recognition, machine learning, and deep learning that can allow us to do something with these sources at a massive scale that we couldn’t before–that’s exciting.
I think the main point, however, is that it’s not about the techniques or the vastness of the data. It’s about having clarity about what questions we need to ask that we haven’t been able to before, or existing questions we struggle with that suddenly–with all of this new potential insight–we can answer in a much richer way. That defines the data we need. That defines the approaches that are appropriate, the models, the algorithms, the techniques, the technologies.
And then, most important, we need to consider: What would we do with the answer if we got it? Insights are fantastic, but if we want impact, we have to do something with it.
Do you see dark analytics as an opportunity to change the organization, to understand customers at a greater depth?
Yes, all of the above. Could we use video analytics to better understand customer flow, and not just where they’re going in a retail store, but their mood, their emotional state? How do those factors actually influence the way they navigate through the aisles, where they stop and what products they’re attracted to? Can we do something with real-time promotions, pricing, to actually tap into that? But we also have to watch out for the “creepiness factor”–to not cross the security and privacy line. How do you ask customers to opt in with transparency?
What are some other applications of dark analytics you’ve seen?
In the report, we feature an oil and gas company that’s been doing acoustic analytics for a long time. It’s given them an amount of precision in knowing what’s happening deep underground in an oil well. They can use that to guide the drilling pattern to determine if it’s oil flowing, or if it’s a mixture of oil and gas, or if it’s sediment.
So, how do we apply all that to the supply chain? Well, what’s the sound of impending failure? Can I have acoustic analytics that could initiate preventative maintenance and avoid a failure because it can hear that a fan is not operating at the right frequency?
What inhibits companies from embracing dark analytics?
A lot of companies don’t know what’s possible. Data is a very complex puzzle that can be overwhelming. It’s probably the most important asset they have, but they haven’t been able to solve for it. They’re asking themselves, How can I make this attainable without it being a massive eight- or nine-figure investment?
How can companies integrate these practices into existing operations?
How do we create a mind-set of data as a core discipline? Treat it as your most strategic asset. We’re seeing a lot of interest in creating labs that actually take real live data from the organization, apply some of these techniques, and show them how analytics is not this passive rearview mirror of what’s happened, but that it can be used to predict and prescribe any wide range of business decisions. Basically, show what it would be like to be an insights-driven organization–which can inspire a broader investment.
This article was created for and commissioned by Deloitte Consulting LLP.