Note to plain text readers: To see the three figures in the body of the newsletter, I suggest you
read it by clicking on the "View as Web Page" link immediately below.
[~ 875 words: take 46 minutes to read over a break or lunch]
View as Web Page
The Alternative to the "Simple, Obvious, and Wrong" SWAG

My last newsletter demonstrated a common incorrect technique  based in "traditional" statistics  for comparing performances based on percentage rates. This newsletter will use the same data to show what should be done instead. To quickly review the scenario: In an effort to reduce unnecessary expensive prescriptions, a pharmacy administrator developed a proposal to
monitor and compare individual physicians' tendency to prescribe the most expensive drug within a class. Data were obtained for each of a peer group of 51 physicians  the total number of prescriptions written and, of that number, how many were for the target drug.
The Correct Alternative  a pchart Analysis of Means

During this time period, these 51 physicians had written 4032 prescriptions, of which 596 were for the target drugs  an overall rate of 14.8 percent. The goal of analysis of means (ANOM) is to
compare a group of physicians who have what should be similar practice types, a relatively homogenous "system," if you will. Each is compared to their system's overall average. Variation is exposed and then there is conversation to discuss the variation, then reduce the inappropriate and unintended variation. For each individual physician's performance, one calculates the commoncause limits of what would be expected due to statistical variation from the system's 14.8
target prescription rate. Based on the appropriate statistical theory for percentage data based on counts (i.e., binomial), a standard deviation must be calculated separately for each physician because each wrote a different number of total prescriptions. Note that this is similar to what was done for the uchart ANOM for rates explained in my 25 August
newsletter.The calculation for the pchart ANOM is as follows: Once again, note its similarity to the uchart calculation, as well as the philosophy of its use. They differ only in this calculation of the common cause band.
As in the uchart, this result of the square root is then multiplied by three (for "three standard deviations"), then added and subtracted to the overall mean
to see whether the actual value for an individual physician is in the range of this expected variation, given an assumed rate of 14.8 percent.
The SWAG graph is shown below and the pchart ANOM is below that.
Note that what many of you would consider conservative three standard deviation limits  calculated correctly  are in this case comparable to approximately 1.5 standard deviation limit of the incorrect analysis. Another difference: the overall system value obtained from the aggregated summed numerators and
denominators of the 51 physicians was 14.8 percent, which differs from merely taking the average of the 51 percentages (15.8).
In ANOM, anyone outside his or her (correctly calculated) unique commoncause band is a probable special cause; these physicians are truly "above average" or "below average." Note that physicians 48 and 49 could still be indicative of a prescribing process at 14.8 percent because of the number of prescriptions written.
Using the previous SWAG analysis,
depending on the analyst's mood and the standard deviation criterion subjectively selected, he or she could claim to statistically find one or 11 upper outliers, using two or one standard deviations, respectively. The ANOM shows eight probable above average outliers, with a lot more certainty than a SWAG.
So, what should we conclude from our correctly plotted graph? Only that these outlier physicians have a different process for prescribing this particular drug than their
colleagues, 36 of whom exhibit average behavior  these physicians between the red lines are indistinguishable, from each other and the system average of 14.8 percent. For some physicians, this outlier variation might be appropriate because of the type of patient they treat ("people" input to their process), and for others it may be inappropriate or unintended due to their "methods" of prescription, but they don't know it. Maybe collegial discussion (including the outliers who are below
average) using this graph as a starting point would be more productive than what has become known as "public blaming and shaming."
Stopping the CrazyMaking

And then there are people obsessed with "standards"...
I cringe as I think of these people creating and enforcing standards via the WAG approach: set a standard above which they "feel" no
one should be and give feedback to those physicians. In this case, if the standard is 15 percent, 27 physicians would get such feedback (19 inappropriately). If the enforcers decide to get "tough" and set a 10 percent goal, then 35 or 36 physicians would get feedback. There is no realization that the current system seems "perfectly designed" to perform at 14.8 percent.
==================================== Integrating ANOM with Standards
Thinking ====================================
If the standard were 15 percent, the first order of business would be to study the eight above average outlier physicians and see whether it is appropriate to bring them into the average red zone with their 36 colleagues. If the standard were 10 percent, there needs to be fundamental change in all physician behavior, but the discussion could begin by dialoguing with the six physicians who are truly below average.
And look
what is lost in all these discussions: linking all this with observed patient outcomes for the individual physicians. Might this put the focus where it should be rather than cost? I would call this a much better analysis for determining where the process should be  which I think would beat a WAG in the patients' eyes.
Until next time...
Kind regards, Davis
======================================================================= P.S. I am in the middle of feverishly editing the new edition of Data Sanity =======================================================================
If you are interested in hearing about or applying the
innovative ideas of my upcoming revised edition of Data Sanity, I can help you with that by...
* ...spicing up your professional or internal conferences as a plenary speaker
*...sharpening your and your staff's skills with a retreat
* ...mentoring you to create awareness that you are surrounded by similar
opportunities. Solve these and watch the resulting breakthrough in your thinking and effectiveness...and respect for your role
Please contact me to discuss these opportunities, ask for the Preface and Introduction of the revised Data Sanity, or just about any other reason! I love corresponding with my readers and answering their questions. [ davis@davisdatasanity.com ]
========================================================= Was this
forwarded to you? Would you like to sign up? ========================================================= If so, please visit my web site  www.davisdatasanity.com  and fill out the box in the left margin on the home page, then click on the link in the confirmation email you will immediately
receive.
========================================================== Want a concise summary of Data Sanity...in my own words? ========================================================== Listen to my 10minute podcast. Go to the bottom left of this page: www.davisdatasanity.com
.

 