Dark data, defined as information that organisations collect, process and store during regular business activities, but generally fail to use for other purposes, is set to become an even bigger problem as the Internet of Things takes hold, creating even more excess data.
Now businesses are being urged to have clear information management plans in place to decide what to keep and what to delete.
John Culkin, Director of Information Management at Crown Records Management, said: “Hiding dark data away seems to be not just planned but instinctive, as if it were down to human nature.
“People fill their attics and garages with ‘useful’ things that could be needed in the future. The corporate equivalent is when people say ‘keep the data just in case’. How much money around the world is wasted due to those few words?”
Culkin believes keeping unnecessary data could have a significant impact on IT budgets, from paying for cloud storage to meeting the challenge of future data protection regulations which will require personal data of European citizens to be accessible and deletable.
He said: “Storing data you cannot find or identify in future is going to be an increasing risk for businesses – and the irony is it is often kept for unconvincing reasons.
“Perhaps data is being kept ‘in case it is needed in the future’ – assuming it can be found at the time of need, of course. Or because a company believes some unknown future insight can be found using future analytical technology.
“Perhaps it is stored away out of a mistaken belief that keeping more data means a company is less likely to lose something. Or because a business believes data is like a rare book and its monetary and nostalgic value might increase.
“All of these misconceptions lead to a hoarding of dark data which is unnecessary, not useful and inherently risky. Having a proper information management system in place to prevent the storage of too much dark data is crucial.”
Here are 10 ways to prevent dark data accumulating in your business:
1 Don’t accept that because data storage is cheap everything can be kept.
2 Develop policies and continuous training for all staff about managing data (including data on local drives, laptops, shared drives, removable devices and mobile devices).
3 Understand which regulations require what data to be kept for how long. Don’t keep it longer than necessary unless clear benefit can be derived from it.
4 Don’t believe all data in enough volume is “Big Data” and therefore has value. Much of it will not be useful in big data projects, especially as it’s often unstructured and in various formats.
5 Ensure defensible deletion (a comprehensive policy to reduce both storage costs and the legal risks associated with storing too much data) is used actively.
6 As new systems or applications are adopted, consider what data they accumulate and how it is used. Don’t have debugging log levels run at too high a setting or longer than necessary to find problems.
7 Just because you can get data and store it, don’t do so without a valid reason.
8 Charge cost centres for storage. If it’s free, the data addiction will only grow and there is little incentive to reduce it.
9 Have an email management system that is centralised and not using local ‘.pst’ files, automatically deleting messages as per the organisation’s policy.
10 Remember newer forms of data such as voice, video, mobile text or social media data. Much of this data will be in the cloud – often put there by creative departments without contacting IT first and therefore hidden.
