"In theory, there is no difference between theory and practice..."

My wife and I watched the Netflix documentary ("It Ain't Over") on the legendary New York Yankees baseball catcher Yogi Berra the other night.  I highly recommend it!  There was a segment of the documentary that questioned why Berra wasn't included among the four greatest living baseball players (Willie Mays, Johnny Bench, Sandy Koufax, and Hank Aaron), as voted by the fans and announced at the 2015 MLB All-Star Game in Cincinnati, which my wife and I were lucky enough to attend!  While I did not participate in that vote, I told my wife that had I voted, I probably wouldn't have thought to vote for Berra either (I regretfully admit that I actually wasn't sure that he was still alive then).   

Today Berra is remembered more for his so-called Yogi-isms than his prowess as a ball player, which is both unfortunate and unfair.  He is arguably one of the greatest catchers to ever play the game, and he should rank right up there with some of the all-time Yankee greats.  He played 19 seasons in total, eighteen of which were with the Yankees.  He was selected to the MLB All-Star game 18 times, won the World Series as both a player and manager/coach 13 times (out of 21 total), won the American League Most Valuable Player three times, and had his number 8 retired by the Yankees.

So I feel a little guilty using one of his Yogi-isms as the title of today's post, but I do so with the greatest of respect for who he was as a player, manager/coach, and person.  While there's no evidence that he actually ever said it, the Yogi-ism goes like this, "In theory, there is no difference between theory and practice, but in practice there is..."  It certainly sounds like something Yogi Berra would say!  Regardless of its origin, it's a great quote!

There is a clear difference between the idealized state, where several different variables can be controlled as much as scientifically possible, and the real state, where we simply cannot control what happens.  We see this over and over again in research, where therapies that appear to be quite promising in early-stage clinical trials, end up falling far short of expectations in later and larger clinical trials.  Here is one of the reasons why I like quality improvement science so much.  Rather than trying to control all of the variables, we observe a system and follow what happens when we make small changes to it, which, in essence, is very similar to what clinical research does, right?  

Rather than trying to control the dependent variable by controlling for all of the independent ones, quality improvement science observes what happens to the whole system over time, using statistical process control.  If you plot the outcome of interest on the y-axis over time on the x-axis, you basically have what is called a run chart.  Even if you've never heard of a run chart, there is a good chance you have seen one before.  Daily fluctuations in stock prices are often depicted as a run chart.  For example, take a look at the run chart below of the S&P 500 price fluctuations during a certain period of time:

Even though there was a clear decrease in the S&P 500 on February 2, 2014, it doesn't appear that it has changed all that much from the beginning of January, 2014 to the beginning of April, 2014.  Traditional methods used in clinical research today would arbitrarily select a certain time period, calculate the mean and standard deviation values, and compare using a specific kind of statistical test.  But what time period do you use?  Let's look at two different ways to analyze the data above - the top graph is the monthly average while the bottom graph is the price at the start of each month:

The two graphs look very different, don't they?  The top graph makes it appear that there was an increase in the S&P 500 from February to March, which was sustained in April.  However, the bottom graph makes it appear that the S&P 500 during February was very different compared to the other three months.  Either conclusion in this case would be wrong - go back and look at the run chart in the original graph!  There really is not that much of a change in S&P 500 over the time-period of interest!

How would we know whether there was a change in the S&P 500 over time?  For this, we would need to plot the data using a control chart.  Control charts are run charts that have a couple of important and additional features - a trend line (usually the median) and upper / lower control limit lines (calculated using specific formulas, depending upon the kind of data that you have) - see the example below:

   

With control charts, there are specific rules (often called "Western Electric Rules", because they were first developed and used by the Western Electric Company in the late 1950's) to determine if there is common cause or special cause variation.  "Common cause variation" suggests that a process is stable and in control - any variation in the data over time represents normal fluctuations that often occur (such as the beat-to-beat variability in your own pulse when you are sitting down while reading blog posts!).  In contrast, "special cause variation" occurs when the process is unstable or out of statistical process control due to a specific or unique circumstance (going back to the previous example, the increase in your pulse when you go out for a run around the block on a hot day). 

If we used statistical process control rules ("Western Electric Rules") in the S&P 500 example above, the change in price over time would be shown to represent common cause variation.  Special cause variation would occur if, say there was an economic recession.  If you are interested in learning more about statistical process control, I highly recommend the following three books:

Understanding Variation: The Key to Managing Chaos by Donald Wheeler (no relation)

The Improvement Guide: A Practical Approach to Improving Organizational Performance by Gerald Langley, Ronald Moen, Kevin Nolan, Thomas Nolan, Clifford Norman, and Lloyd Provost

The Health Care Data Guide: Learning from Data for Improvement by Lloyd Provost and Sandra Murray

Statistical process control allows us to monitor changes in our process over time, i.e. in real world practice, as opposed to what happens in the strict constraints of an experiment.  So, "In theory, there is no difference between theory and practice", except for the fact that there is...