By Robert L. Moore, MD, MPH, MBA, Chief Medical Officer

“It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”

– Sir Arthur Conan Doyle, physician and author

One of the most under-recognized leadership skills is using data to inform decision-making. We say we believe in data-driven decision-making, but the reality is that we often make decisions without the data needed to support them.

Human beings make small decisions about life every day, such as what to eat for lunch or what show to stream that evening. The answers to these are often gut-level decisions, based on how we are feeling at the moment, as well as our preferences, often based on what we have experienced before. Data is not required, although we may use some data and perhaps some recommendations to make these decisions if it is presented to us.

The quality of decisions we make in other settings depends on the data available, whether in the exam room, the clinic, or the leadership suite.

Data in Clinical Decision Making

As clinicians, we make decisions about the most likely diagnosis a patient has, or which therapeutic modality to use given the individuality of the patient before us. With years of experience, these decisions become increasingly automatic, tempered occasionally by a Continuing Medical Education (CME) article or consulting a research-based medical journal. Whether the answers we choose are automatic or considered, they are based on a body of training and knowledge that is associated with our clinical role, customized by the individual patient data available to us.

While clinical decisions depend on training and patient data, clinical operational decisions rely on a very different type of data.

Data in Clinical Operations

When making decisions about clinic operations, the same clinical decision-making approach is often less effective. Clinic operations decisions can include determining how many patients per hour to assign an experienced physician that is new to a clinic, or whether holding a staff huddle at the start of each shift is cost effective. As a result, we tend to rely on gut-level decision-making, simply because the right data isn’t readily available to make an informed decision.

Even when data is available to inform management decisions, it often consists of isolated observations that are not broadly representative and may have been gathered in ways that make them unreliable or biased. One example is looking at the number of patients an individual provider saw in a given shift and drawing a conclusion about their productivity, without looking at the number of patients scheduled, how they were confirmed, what the no-show rate is, what their productivity is in other clinical settings, or what the front office is telling patients who call in.

A clinical practice can seem to run adequately without data informing the decision-making. Everyone is working, patients are seen and cared for. But to make operations most efficient, effective, and patient-centered, managers need to collect and monitor data. A basic data set for a multi-provider practice is:

• Number of patients seen per shift
• No show rate
• Supply and demand analysis on a day of the week/month of the year basis: demand for same day appointments, number of open appointment slots available when the day begins
• Call drop rate for incoming phone calls
• Patient-PCP continuity rate: rate at which the patient sees their primary care provider and the rate at which the PCP is seeing their assigned patients
• A post-visit patient satisfaction survey
• Percentage of patients discharged from the hospital or the emergency room who have a primary care provider appointment or at least a phone contact within seven days of discharge

In our experience in looking at data for 150 primary care providers, health centers that are gathering and using this data regularly are top performers, with better quality scores, patient satisfaction scores, and financial outcomes. Setting up the systems to gather and regularly review such data is a critical management competency. Senior leaders, including medical directors, need to ensure that excellence in everyday operational decision-making is promoted and maintained. A good way to do this is to have a daily or weekly dashboard of key operational parameters that is generated and reviewed by the leadership team.

Making changes that affect clinician work requires leaders who have earned their clinicians’ trust and can translate operational drivers and goals into terms clinicians understand, often leveraging their own quantitative skills to validate how the data is being interpreted. Trust and interpretation also help clinicians with another type of data, used for larger organizational, strategic, and tactical decision making.

Data in Strategic, Tactical, and Policy Decision Making

“I made the decision based on my best guess, because the data wasn’t available.”

I have heard this many times over the years as a justification for making decisions that ultimately proved harmful or, at best, ineffective.

Healthcare leaders and policymakers often believe they have limited data to inform their decisions and, as a result, default to instinct or rely on isolated snippets of data. What they really lacked was a system for extracting the insights needed to evaluate their options from data already being collected.

This can be present at different levels:

1. The data exists but is not readily available to analysts or decision makers.

For example, the Partnership Quality Dashboard allows clinic leaders to see which patients are driving their ambulatory sensitive hospital admission measure. However, this data needs to be attributed to the PCP that patients are assigned to, for leaders to evaluate strategic options to best address these high-cost patients.

2. Data is analyzed and presented in a way that is not interpretable by leaders.

In this case, the data is available and even integrated with other data sources, but the analysts, report-writers, and data scientists lack the contextual information to extract and present it to answer the logical questions leaders are asking. One example is presenting reports on clinical quality data organization-wide, not drilling down by clinical site or assigned PCP or by insurance type or co-occurring disorders known to affect quality (like unhoused status or substance use disorder).

The most important starting point to data-driven decision-making for leaders is answering the related questions: “What question am I trying to answer?” and “What will I do with the data once I have it?”

Partnership prides itself on being responsive to our communities and our counties, and our staff time is too valuable to spend generating reports that are glanced at briefly and then set aside. The same is true for the time and expertise of analysts. That’s why clearly defining the question you are trying to answer is so important. Too often, the data being requested does not actually provide the information needed to answer the underlying question.

For example, we had a community stakeholder ask us for claims data on rates of substance use disorder in unhoused populations. They wanted to understand the different rates of use of different substances (the question they wanted to answer), to design potential interventions (what they would do with the data). For a variety of reasons, claims data would not have provided an accurate measure of these rates, and any conclusions drawn could have been misleading. In this case, survey data would be most appropriate, which is gathered by other entities, not Partnership. State government agencies collect this data, and post it online, so we redirected them to that website.

How can leaders better access and use the data already available to help with strategic and policy initiatives? There are four parallel activities to gain this expertise:

1. Cultivate a learning mindset, when it comes to data review and analytics.
   • When presented with data or graphs, push to validate the quality of the underlying data and look for ways in which the presentation of the data is unclear or biased. Ask questions and seek clarifications!
   • When trying to understand an issue or look up a particular data point, carve out some extra time to occasionally dive deeply into that data set, to try to really understand it.
2. As part of your leadership self-development, seek some formal education or self-education on the topics of the interpretation of data, the use of Excel to conduct basic data analysis, and the many ways the presentation of data is used to mislead.
3. Hire people with an aptitude for finding hidden gems in data sets to be part of your team. They may be epidemiologists, clinicians with epidemiology or biostatistics background, or individuals with training in applied sciences, like engineering. A helpful exercise during the interview process is to give them a graphic representation of data that you are familiar with and that isn’t very clear and ask them to walk through their thought process as they review it.
4. Take some time to familiarize yourself with online data sources already at your disposal: U.S. Census data, California Department of Health Services data sets, Health Resources and Services Administration data, County Health Ranking data, and Indian Health Services data are good starting places.

Sometimes there really isn’t data available to answer a critical question needed to make an important decision. When this is the case, there are a couple of options:

1. Start with a small-scale investigation: Let the techniques of Quality Improvement methodology kick in. What small sample-size survey (often a sample of 20 to 50 is enough to answer the question) or clinical focus group could you run to quickly get the answer you need?
2. Call a friend: Ask your circle of clinical leaders (including Partnership medical directors) if they know of a data source that will answer the question you have or a way of gathering information that has been done successfully before. Almost always, someone else has faced the issue you are dealing with and developed data sources to help with this. Tap into this prior knowledge!

When tempted to make an operational, strategic, or policy decision without data, keep this saying in mind: “Use the data you have!”

The next time you are considering an important decision, search out the data that is already available to help you make a more informed decision. Additionally, try not to “reinvent the wheel.” If you don’t know where to find the data you need, put out a call for help to ask for advice from leaders who seem to have good systems in place.