By focusing on the right areas and using the right technology, businesses can bring light to their data and save their bottom line.
Businesses are always happy to have more data about their operations, customers, and the results of their strategy implementations. The problem is that, once they have it, they may not know exactly what to do with all that information.
Not knowing the best way to interpret and act on the data can literally be costing your business. And those costs could be in the form of lost revenue, missed opportunities, lower efficiency and productivity, quality issues and more. Forrester reports that between 60 percent and 73 percent of all data within a business organization goes unused for analytics.
Even though more businesses are advocating about big data, utilizing technology and systems to collect more data and recognizing the value of data.
This unknown and unused data, known as dark data, comprises more than half the data collected by companies.
Gartner defines dark data as:
“The information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes (for example, analytics, business relationships and direct monetizing).
“Increased data growth over the past decade has created an unstructured data nightmare,” says Alan Dayley, research director at Gartner. “It’s not just the cost to store it. Huge volumes of dark data make it harder to find what is useful and may mean we miss business opportunities.”
IBM estimates that 80 percent of all data collected is dark data. Most of this is generated during regular business activities but isn’t used for any other purpose. Types of dark data include:
- User activity logs
- Server monitoring logs
- Customer conversations
- Emails, meeting minutes and employee created data
Dark data leads to High Costs and Poor Business
Many companies have wasted precious time and resources investing in the wrong systems, trying to enforce new systems or connect them to legacy applications, and then compound the issue by handing employees’ mountains of data they can’t possibly analyze. Because businesses can’t effectively manage their data—i.e. ensuring it’s clean, accurate, governed and accessible—they are unable to discover meaningful insights. Worse of all, businesses may end up making bad decisions based on inaccurate data leading to poor businesses and doomed sales and marketing efforts.
Veritas estimates the global cost of dark data is expected to reach $3.3 trillion USD by 2020.
Companies are in no way intentionally misusing their data. They are simply trying to make sense of the information they have, which is understandable given the high volumes of complex data being collected. Navigating through dark data and trying to uncover patterns and predict trends may feel similar to uncovering the proverbial needle in a haystack.
With businesses generating more and more data, the costs and liabilities associated with dark data continue to grow. Determining what data to look for your business is the first challenge to overcome, and from there, you only need to know where to look within the data.
Don’t be afraid of your dark (data)
The term “dark data” is intimidating. After all, spooky things come out of the dark – vampires, ghouls. But huge amounts of unstructured data don’t have to be scary.
“No matter which types of dark data your organization collects, or how it is stored, the key to keeping data out of the dark is to ensure that you have a means of translating it from one form to another and ingesting it easily into whichever analytics platform you use.” - Alan Dayley.
While the cost and security concerns are valid enough reasons to clean your data, there’s a huge incentive in paying attention to your dark data. It contains valuable insights about what your customers want, and where your business needs optimizing.
The opportunities in dark data
There are economic opportunities in unanalyzed operational data. Businesses can look to utilizing this data to drive new revenues or reduce internal costs.
Writers at Datumize recommends: “By utilizing new technologies around business intelligence and IT tools, companies can join structured and unstructured data sets together
to provide high-value results. When done correctly, the benefits will easily outweigh the costs involved with mining dark data.”
Examples of dark data that are often not utilized include server log files that can provide clues to website visitor behavior, customer call detail records that can reveal consumer sentiment and mobile geolocation data that can uncover traffic patterns and density mapping to aid in business planning.
Managing your dark data
1) Start by analyzing your current data situation – determine what data might be useful, and what needs to be deleted and stop being collected.
2) Classify existing dark data – once you have identified the data you need; the next step is to turn it into structured data. Look to Big data tools to help you with this. Hadoop and similar SQL tools are great at dealing with large amounts of data.
3) Reach out to all stakeholders – once you have classified your data, start creating accessibility to the data.
Nitin Mital, Head of Deloitte Consulting’s Analytics & Information Management practice, believes that bringing in key stakeholders early and from other areas of the business is critical to “illuminating” all that dark data.
“Work with business teams to identify specific questions that need to be answered. Then identify the sources of data that make the most sense for your analytics efforts.” For example, talk to your head of marketing about their goals for the quarter. How can data collected across the business (for example, customer conversations or product usage logs) help marketing understand customers better? - Nitin Mital
Generating and storing large amounts of data that do not serve any purpose is useless. Businesses that unlock the secrets behind their dark data will gain a strategic advantage over those who are still left in the dark.
To receive insights on the latest topics, subscribe to our monthly newsletter or follow us here.
Comments