Data Unreliability in a Metric-Defining Age

Updated on June 11, 2019

By Kyle Monticone

The healthcare industry is infamously reliant on data to define and drive organizational metrics.  Data outlines the quality of patient care, the productivity of employees, and the financial stability of the organization.  However, as organizations center their operations around data, they are woefully treading in the dangers of patient care.

In college, my professors taught my class to support our positions with peer-reviewed articles found in educational databases.  Any individual well-versed in academic research knows that there are often articles supporting both sides of an argument.  To circumvent this, professors taught our class to carefully manipulate the opposing argument to support our position, or to neglect the opposing argument altogether.  This danger carries over to the healthcare sector.

Process improvement (PI) departments and lean six sigma teams are the drivers of healthcare quality.  More organizations are hiring engineers and data analysts to provide reports to track custom metrics.  When used well, these teams can drive an organization forward.  More often than not, these teams are driving administrative decisions based upon arbitrary, unreliable, and manipulated data.  

It is common for administrators to use data to justify departmental productivity, analyze patient satisfaction, and review performance metrics.  Since most departments use unit-specific software, it is hard to build reports that genuinely capture the entire patient experience.  To mitigate this, healthcare organizations contract with outside vendors to custom build reports and extract data.  These vendors do not usually understand the overall operation and will not corroborate any data, but they provide raw data nonetheless.  

I have experienced spreadsheets with inherently wrong formulas.  In one situation, an employee used the wrong formula to justify the closing of an entire department.  Without my meddling, the employee would have reduced the number of full-time employees, eliminated a much-needed service line, and impacted patient care. 

I have also seen analysts work together to create custom yet manipulated reports to support their positions.  In one occurrence, the analysts proposed situational formulations to our executive team.  Our administration at the time, a cautious supporter of data, asked the department to revisit their formulation.  Fair and cognizant executives want best practices; they do not want to choose from a list of situational and arbitrary formulas.  The analysts did not truly want accurate data.  They purposefully neglected numerous contributing factors, and they did not even visit the actual patient floors to review workflows and talk with first-line employees and managers.

On the opposite side, I have observed as management instructed employees to artificially shape data to exceed key performance metrics at an outpatient healthcare facility.  The management knew there was not a system in place to authenticate their data, and they needed a larger budget with additional associates allocated to their facility to advance the patient experience.  They defended their actions by arguing that there was no other way to get the resources they needed.  Management would have realized the facility’s needs with a simple walk-through.

Healthcare is robust for job stability.  It is hard to replace the empathetic human element that patients typically want in their care.  Consequently, a lot of healthcare processes are manual.  Not every system is the same.  There are not automatic trackers for patient location, real-time patient satisfaction, or even financial remittance.  Despite this, many people still use already-compromised data to justify decisions.

I often tell my associates that data is merely a tool.  It can influence a decision, but smart individuals should not solely use data to make a decision.  Data needs to be cautiously reviewed and supported by first-line employees who are involved in the actual processes.  These employees can help ensure the relevancy and accuracy of the data since administrators will use the data to affect the department’s operations.

Leaders must cultivate a culture of trust by fighting for validity and increasing communication with their teams.  Without a system to verify data, the efficacy, safety, integrity, and quality of patient care is negatively impacted and compromised.  

Kyle Monticone is an administrative fellow of the El Paso County Hospital District system.  He received his Master of Healthcare Administration from Texas State University.

The Editorial Team at Healthcare Business Today is made up of skilled healthcare writers and experts, led by our managing editor, Daniel Casciato, who has over 25 years of experience in healthcare writing. Since 1998, we have produced compelling and informative content for numerous publications, establishing ourselves as a trusted resource for health and wellness information. We offer readers access to fresh health, medicine, science, and technology developments and the latest in patient news, emphasizing how these developments affect our lives.