Dark Data: How Healthcare Organizations Can Make Informed Decisions 

Updated on September 28, 2022
Adobe Stock royalty-free image #200146313, 'Shot of Corridor in Working Data Center Full of Rack Servers and Supercomputers with Internet connection Visualisation Projection.' uploaded by Gorodenkoff, standard license purchased from https://stock.adobe.com/images/download/200146313; file retrieved on May 9th, 2019. License details available at https://stock.adobe.com/license-terms - image is licensed under the Adobe Stock Standard License

Facing ongoing challenges such as inflation, increasing labor expenses, and staff shortages, many healthcare financial leaders are looking for solutions to secure a solid financial future. This is an increasingly important issue, as over half of healthcare executives said financial viability is one of the top concerns in the coming two years. 

As health systems continue to face an uncertain economic climate, many don’t know where to turn when managing their finances. Healthcare leaders may be surprised that a first step in improving financial performance isn’t always examining the company’s balance sheet. It could start with unlocking hidden data, called “dark data.” Collected by an organization during day-to-day operations, dark data remains usually untapped or under-utilized but represents an unseen asset.

By leveraging dark data, healthcare organizations can reveal overlooked insights in budgeting and planning for areas such as patient costs and procedure volumes, labor wages, staffing requirements, and more. Tapping into this data with the promise of artificial intelligence (AI) and hyper-personalization enables better financial, operational, and strategic decisions that lead to improved performance and provider satisfaction. To bring this data to light, leaders must invest in tools that improve data visibility so they can determine where to economize and enhance overall performance.

What is dark data? 

The term dark data may have ominous connotations; however, there’s nothing particularly “dark” about this type of information. Dark data in healthcare is simply untapped data that can range from patient records and clinical trial information to data regarding service line rates and margin conditions. It can also lead to incredible advances in patient outcomes, driving hyper-personalized treatment plans and medication guidance.

However, despite the variety of rich data, dark data is consistently difficult to uncover and can be costly to store in a secure manner. And it’s a widespread issue; nearly three-quarters of all data within an enterprise remains unused for analytics, often because it is multifactorial and requires standardization and structure to gather meaningful insights.  Dark data remains hidden within the depths of massive healthcare databases, with many organizations failing to realize the full potential or extent of its capabilities.

Compliant with HIPPA regulations, much healthcare data has been stored for years, leading to a massive intake of dark data that has only taken up storage space after its initial documentation purpose. This presents an undeniable opportunity to tap into this pool of data, gather insights that can optimize financial performance and employee experience, thus unlocking the value that lies in dark data. 

Applying a technology such as AI can help detect, sort, classify and identify logic and patterns, turning dark data into a data-driven framework to inform insights and manage finances. Analyzing multifactorial data in a standardized, cost-effective, and secure manner is complex and requires technical solutions and back-end support, which can be a strain on operations. 

How to best mine dark data 

Picking and choosing which dark data to mine – and how to access that data – starts with auditing your organization to determine how to best manage and utilize the data. 

For example, organizations should determine if it’s a higher priority to look at their budgeting and labor costs in the wake of ongoing healthcare worker shortages or financial standing as it relates to partners and customers. 

As a second step, organizations need to be cognizant of security and privacy. Healthcare data breaches impacted 45 million individuals in 2021 alone.  Especially in healthcare, dark data contains sensitive information called Protected Health Information (PHI). Healthcare executives need to ensure the pillars of security and privacy are being upheld when mining dark data to avoid accidental exposure. With increasing interconnectedness and interoperability, healthcare organizations and executives must also be conscious of third-party vendors with access to their data and take a proactive approach to cybersecurity.

Thirdly, it’s important for leaders to ensure they have the right infrastructure to manage and handle the massive amounts of dark data that exist within an organization. This includes the normalization and translation of unstructured data from various sources, for example, into the HL7 FHIR standard widely used for data analytics. Technology solutions that implement AI and machine learning can assist to oversee the process, making it more effective and time efficient – processing a 20-year data backlog in just a matter of days – but it requires standardized data of good quality to produce high-quality results.

Finally, picking the right partner to optimize the dark data mining processes for efficiency, accuracy, and long-term fiscal health cannot be understated. Dark data comes with certain challenges that take time and resources – if organizations aren’t equipped with the right tools to streamline the process, they could be left with more work with little reward or even more accumulation of dark data. 

Evaluate your financial performance and cutting costs

Tapping into digital solutions can help provide next-generation financial planning, decision support, and business intelligence capabilities to improve the cost of care, productivity, and operational outcomes for healthcare organizations. 

This will enable healthcare organizations to meet financial benchmarks and goals, execute strategic plans, improve employee rates and hours, and better serve community needs. By evaluating financial performance, healthcare organizations can digest their data in a manner that will glean more actionable insights, guide decisions and improve performance across many metrics.

Taking the next step 

Bringing data out of the dark is one strategy for helping health systems make well-informed financial decisions when there is little room for financial error.  Identifying dark data, recognizing security risks, and having the right infrastructure will help organizations cut costs and improve overall patient care. Organizations should select their partners wisely, invest in the proper tools, and continuously monitor financial performance to adjust plans as needs and challenges arise.

Joerg Schwarz, Ph.D. is Snr. Director of Strategy and Solutions for Healthcare Interoperability at Infor.

Steve Wasson is Executive Vice President & General Manager of Data and Intelligence Solutions at Syntellis.