Evidence-Based Management: A View from the Jury Box

Evidence-based management is a process that in the end helps to make decisions improve the probability that outcomes will be positive.

Healthcare is a complex industry, and we, as managers, must work within those complexities on a daily basis. The outcomes of our decisions are not always as we would expect. We can, however, improve the odds of having reality match our expectations if we make the decision to rely on evidence rather than anecdote for our decisions. The simple concept is this: we work in a field that is rife with chaos – and contrary to what some may think, chaos is neither random nor unpredictable. 

A few years ago, I had the privilege of serving on a jury. The charge was rape and the trial lasted three full days. It was a tough case, for we the jurors had to hear testimony that was at times emotionally disturbing and for some, quite painful. And while there were many nuances associated with the process, what did not escape me was how our jury system offers a lesson regarding how management should function: on the basis of evidence rather than anecdote or pure gut reactions. In fact, the jury system is the perfect example of evidence-based management, following the concepts and techniques of testing the null hypothesis.

In applied statistics, where I spend much of my time, the null hypothesis is the gold standard for testing any theory. In general, it says that we accept the status quo as being correct unless we have evidence (sometimes overwhelming and sometimes not) to suggest otherwise. In my situation, the null hypothesis is that the person on trial is not guilty, and the alternative hypothesis is that they are guilty. Notice that we don’t say “innocent” or “guilty” but rather “not guilty” or “guilty” – and that is because the purpose of the trial is not to prove innocence, but rather to disprove guilt.

Many times, people do not realize just how their biases can affect their decisions. In this case, there were plenty of sources of bias: race, personal experience, emotion, etc. In the end, we came to a unanimous verdict – not by ignoring our own biases, but by allowing the evidence to speak for itself. We could coopt another quote by saying “the evidence speaks so loud, I can’t hear what you are saying.”

Using the null-hypothesis theory, we started with the assumption of not guilty, so to find the defendant guilty, it was up to the prosecutor to introduce enough evidence to cause us to doubt the null and accept the alternate. And that is how evidence-based management should work. We all have biases and make decisions based on our own experience and heuristics, but instead, we should put more weight on evidence than we do our gut.

So, what is evidence-based management (EBM)? It is a global management concept that employs practical applications such as process modeling, cause-and-effect analyses, decision theory, problem-solving, and other areas of critical thinking. It can expand into data and predictive analytics, business intelligence, and other fields and techniques that both are the antecedent as well as the precedent of EBM. As it applies to us managers and leaders in healthcare, it is about making management decisions in such a way as to ensure that we significantly improve the probability of a positive outcome. In order to achieve this, one needs to get a foundational understanding of, at the least, data analytics and business intelligence. This doesn’t mean that you have to be a statistician or data scientist, although more and more organizations are hiring folks with these skills in order to advance their agenda in EBM. On a regular basis, I am engaged to help clients solve some complex problem within their organization. As a baseline, you should know that I am a terrible decision-maker; just ask my wife! But I have a reputation as a great problem-solver, and the reason is that I have training and experience in most areas of analytics and business intelligence. I am a statistician. I am experienced with software development and programming. When you put all of this together, it means that, just like with our jury example above, while I can’t get rid of my biases, I can overcome them with evidence. Where does that evidence come from? From a heavy reliance on data. And reliance on data is the first step, as it represents the “evidence” portion of “evidence-based management.”

Data come in all forms and sizes and styles, and it is incumbent on those of us who choose to be evidence-based managers to learn more about how to obtain, organize, and utilize the data available to us. As you can imagine, this discussion requires far more time and space than what’s allotted here, but don’t let that stop you from exploring this yourself. I am old. I am so old that I still know how to use a slide rule (and in fact, still do!) It used to be that data was difficult to come by and quite difficult to analyze due to limitations of computer hardware and software (I told you I was old). Now, the problem is that we have so much data coming at us so quickly – and even though we now have the computing power to analyze almost anything, not everything is worth analyzing. Quite the paradox, actually. As evidence-based managers, we have to look at data with a keen eye, separating the “useful” from the “nice to know” and figuring out the best method(s) for analysis. 

The second vector in becoming an evidence-based manager is the ability to make decisions that in the end improve the probability that outcomes will be positive. This is harder than it sounds, as so much of our decision process is based on our life experiences. Sometimes, that’s ok. A mosquito lands on your arm and you swat it. You don’t take the time to Google “mosquitos” to see what species it is first. Your house is on fire, so you run. You don’t take the time to calculate the burn rate of the combustible materials first. First-order decisions are straightforward and easy, but others are much more difficult and require a more disciplined and data-centric approach. Complex issues, such as selecting the right electronic health record (EHR), require a complex approach to decision-making. There are lots of interrelated parts and many interdependent actors, and coming to the most optimal (not necessary the most “right”) decision depends on your ability to acquire the evidence necessary to rule out or support your decision (the hypothesis). Becoming a better decision-maker, by becoming more dependent on data (evidence), will almost guarantee that your decisions will more often than not result in a positive outcome – and everyone appreciates those kinds of results.

The third vector in becoming an evidence-based manager is problem-solving. Everyone loves a problem-solver, and not only is this good for your organization, it is good for your career. Problem-solving comes with practice and results from effort in both analytics and decision-making. A good place to begin is to study Lean Six Sigma (LSS). In my opinion, LSS platforms like PDSA (Plan, Do, Study, Act) are key to solving problems and have the benefit of testing and implementing solutions. Lean Six Sigma helps one go through a problem from definition to cause and effect to solution in a structured and ordered fashion, which is critical to establishing a good workflow. And a good workflow is the basis to building a more efficient organization. Remember, eliminating waste from a process is the best way to improve efficiency, and these types of problem-solving techniques were designed with efficiency in mind. 

What’s the different between the jury system and love? The jury system lasts forever! Juries have been around since the 12^th century (at least), so maybe we should take a look at them more closely as they relate to our own approach to management. 

And that’s the world according to Frank.