The reliability of the current state of health information is extremely limited.
EDITOR’S NOTE: Dr. Joseph Nichols is producing a four-part series on healthcare data for ICD10monitor. This is the first installment in his exclusive series.
The healthcare industry has spent an extraordinary amount of time, money, and effort to gather data and provide healthcare information for a wide variety of purposes. In theory, this information should help drive clinical, financial, policy, and personnel decisions for payors, patients, providers, government, and all healthcare stakeholders. Without reliable healthcare information, all stakeholders are at significant financial or personal health risks.
The largest breadth and volume of healthcare information is derived from clinical and business transactions, as opposed to structured research. This transactional-based information includes virtually every instance of healthcare delivery wherein there is a claim for some form of reimbursement.
There are significant advantages to this data, since it, by definition, includes nearly every healthcare observation and event that is included in a claim for payment. Population-based information does not, however, replace more controlled research studies. Both information sources play a significant role in understanding the nature of healthcare delivery, in order to help stakeholders make wise choices.
The reliability of the current state of health information is extremely limited, due to a number of information challenges that will be discussed in a series of papers related to the quality of data and the resultant information produced. Many of these challenges are poorly understood by those relying on this information.
The focus of the series is to frame the challenges related to the creation of health information, and to suggest potential solutions to creating an information environment that provides a solid knowledge base for all policy and personal healthcare decisions.
The discussion will be limited to the area of population-based data derived from healthcare transactions, rather than structured research studies.
Key Considerations in Assessing Information Quality
Despite the constant change and contradictions inherent in current health information, there has been a general assumption that the information we receive about healthcare is complete, accurate, reproducible, and based on verifiable facts. These assumptions about the source of data facts, and analysis of those facts, need to be tested to ensure that the information derived actually provides the information needed to make good decisions and drive policy that is in the best interest of those receiving healthcare.
Getting to reliable information involves four high-level information domains:
- Data Acquisition
- Data Management
- Data Analysis
- Controlling Bias
A simple outline of these areas is presented below. Subsequent papers in the series will dive deeper into each of these domains.
Data Acquisition
Data acquisition is the process of collecting observations about objects and events. that process should not be driven by any assumption about outcomes. it should only represent a broad collection and recording of observations in an area of interest.
The following are high-level requirements for data acquisition:
- Complete and accurate observations relevant to the area of study.
- Complete and accurate documentation of those observations.
- Standardization of the observed facts through consistent, comparable coding and terminology.
Data Management
Acquired data has no innate value without properly structured storage and retrieval mechanisms. each data element has a relationship to some other data element or defined concept. for example, a great piece of literature is simply the structuring of a set of words. The literature only gains meaning based on the relationship of each word to other words, as well as to other implied or defined higher-level concepts.
The following are high-level requirements for data management:
- A structured database design that represents accurate relationships of concepts at all levels.
- Governance of data to assure reliability, standardization, and comparability over time and across enterprises.
- Technical and non-technical support to keep data secure and privacy protected.
Data Analysis
The analysis of data assumes that observations, documentation, and standardization provide a comparable universe of facts that can be organized and presented in a way that results in information needed to drive decisions and actions. analysis can be used, however, to create very different results from the same data, depending on the rigor applied in the analytic process.
The following are high-level requirements for data analysis:
- Consistent, clearly defined methods of aggregation or classification.
- Statistical validity.
- Multifactorial testing of reporting results.
- Transparency to analytic methods.
- Safeguards against bias-driven analysis.
Controlling Bias
All of the requirements above are overshadowed by the biggest enemy to quality information: bias. There has been a great deal written about information bias. whether writing about science, psychology, philosophy, politics, economics, sociology, or any number of other knowledge domains, most authors agree that the human tendency toward bias is the biggest impediment to the truth.
It seems that truth that does not support our beliefs is not welcome. Controlling or limiting bias can be the biggest overall challenge, as it may not be embraced by some in the pursuit of accurate information. Unfortunately, many information seekers want to merely confirm their own beliefs, rather than learn new information that may challenge those beliefs, and cause them to change their decisions.
The following are high-level requirements for controlling bias:
- A commitment to data quality, data management, and data analysis, independent of the desired or expected result.
- A commitment from all participants to limit bias throughout the entire process of creating information.
- Adherence to controls in each phase of the process to limit bias, or at least expose potential bias.
- The appropriate use of statistics to counteract bias, rather than support it.
Summary
Improving healthcare information is a monumental task, due to challenges in data acquisition, data management, data analysis, and bias, as well as a general lack of understanding about the requirements for creating reliable information. For most, a critical assessment of information presented to them is a great deal of work. We tend to color our perceptions of information in a manner that is consistent with our preexisting beliefs. the “truth” seems to be malleable, and may not be supported by our desired or expected outcome.
Programming Note: Listen to Dr. Nichols report this story live today during Monitor Mondays, 10-10:30 a.m. EST.
