Summary and Perspective of Recent Literature
Werneke M, Edmond S, Deutscher D, Ward J, Grigsby D, Young M, McGill T, McClenahan B, Weinberg J, Davidow A. (2016). Effect of Adding McKenzie Syndrome, Centralization, Directional Preference, and Psychosocial Classification Variables to a Risk-Adjusted Model Predicting Functional Status Outcomes for Patients with Lumbar Impairments. JOSPT; 46:726-741.
STUDY’S PUBLISHED CONCLUSION
The small added prognostic capabilities identified when combining McKenzie or pain-pattern classifications with the SCL-BPPM classification did not significantly improve prediction of functional status outcomes in this study.
At first glance, it would appear that MDT classification along with psychosocial classification has no importance when attempting to determine prognosis in patients. However, it is important to first understand the study’s design.
WHAT IS THIS STUDY TRYING TO DO?
This study is attempting to determine what independent factors best explain or predict a patient’s functional outcome at discharge from physical therapy services. Multiple models were developed in a series examining the predictive power of patient characteristics, therapist characteristics and the effect of adding MDT, Pain Pattern, and Psychosocial classification categories, as well as, a combination of the aforementioned classification paradigms. All eight models (Table 5) were compared in a head-to-head manner. These statistical comparisons allowed the determination of which model had the greatest ‘predictive power’ (i.e. R2value) for predicting a patient’s functional outcome following treatment.
The ‘predictive power’ of a model is represented as an R2 value. The greater the R2value, the stronger the predictive ability of a model for the given dependent variable. The dependent variable we are concerned about is the functional status of the patient at discharge. The functional status of the patient is assessed by Focus On Therapeutic Outcome’s (FOTO) lumbar measure. This measure is psychometrically reliable, valid and responsive and has been described in detail elsewhere [1-5]. FOTO uses a 0-100 functional scale to express a patient’s overall level of function (0 = ‘essentially bed ridden’ vs 100 = ‘participating in collegiate sports’).
In TABLE 5, you will see two R2values per model; One that is calculated initially with our available data for the study, and a second that is generated by PRESS (Prediction Error Sum of Squares). PRESS is used to avoid ‘overfitting’. Overfitting is a problem that can occur in complex statistics when you have many variables to assess. As stated earlier, the purpose of the proposed models is to PREDICT functional changes for future patients. The model, however, is using a data set that has already been collected and, in the worst case scenario, the model generated would essentially ‘memorize’ the data points used and thereby have 100% prediction for the available data but have no utility with future data. PRESS is used to cross validate the initial findings of each model. To do this, PRESS uses the model’s prediction equations on a completely separate collection of patient data and shows how similar the two findings are. To demonstrate validity, you want the model’s predictions and PRESS’ predictions to be fairly close, if not ideally identical. The findings demonstrate that the margin for error is small.
Only significant independent variables are included in the model to calculate the overall R2 value. An independent variable’s explained variance is represented as a beta coefficient. Beta coefficients are a way of representing to what extent a variable, such as age, has an ability to influence for better or worse a dependent variable relative to all variables measured. The beta coefficients reported in Table 5 indicate the amount of explained variance that each significant independent variable contributes to the predictive power of the model compared to a reference standard.
EXAMPLES:
Model 2 (FOTO and MDT Classifications) demonstrated an additional 2.8% in predictive ability compared to Model 1 (FOTO). Reducible Derangement is the reference standard for MDT classifications. Compared to a Reducible Derangement, Chronic Pain State in this model predicts that the patient will achieve 14.3 fewer points over the course of care. Mechanically Inconclusive is predicted to achieve 5.1 fewer points of functional gains by discharge compared to an individual classified as Reducible Derangement.
Model 7 included the addition of MDT classification and SCL-BPPM to FOTO’s original model and resulted in an additional 3.6% predictive power. Again, Reducible Derangement is the reference standard for MDT classifications. Compared to a Reducible Derangement, Chronic Pain State in this model now predicts 13.4 fewer points of function at discharge. This highlights that the strength of each beta coefficient is dependent on all the variables calculated in the equation.
MDT clinicians consider classification essential to guiding treatment and setting long-term expectations (prognosis) for our patients. We found that classification categories were significant and generated large beta coefficients within all classification models examined (except for fear avoidance model) yet when comparing models in a head-to-head manner as we did in this study, the conclusion appears to contradict the data reported in Table 5.
IS THE CONCLUSION CORRECT?
We observed that the addition of classification variables added an extra 4% in R2value after controlling for patient and therapist characteristics (i.e. 44% vs 40%), but R2values were not statistically different between models. At first glance, if the reader only read the abstract, they are left with the impression that classification was not only statistically insignificant, but clinically unimportant.
Although the differences in R2value between models were not statistically different, that does not mean that classification was not important!!! The devil is in the details. Understanding the statistical complexity of the study design, knowledge of previous prediction models developed and published in the physical therapy literature, and careful interpretation of the data presented in Table 5 offers a different perspective.
IMPORTANT DATA FINDINGS from TABLE 5 (below)
- Model 7 (addition of MDT and SCL-BPPM) improved the original Model by 3.6%.
- MDT Classification beta coefficients were generally larger values (i.e. -5.0, -13.4) than SCL-BPPM beta coefficient values of -3.3 and -3.2. Therefore, MDT is a greater prognostic variable then SCL-BPPM.
- Model 8 (addition of Pain Pattern and SCL-BPPM) improved the original Model by 3.9%.
- Pain Pattern Classification beta coefficients were generally larger values (i.e. -8.1, -3.2) than SCL- BPPM beta coefficient values of -3.3 and -4.0. Therefore, Pain Pattern Classification is a greater prognostic variable then SCL-BPPM.
- Chronic Pain Syndrome (MDT Classification) had the greatest beta coefficient of all at -13.4 (Model 8).
- FABQ had NO statistical benefit in predicting outcomes.
Due to the original study’s design of comparing models in a head-to-head fashion, it is correct that statistically a 3 - 4 % prediction (achieved by MDT / Pain Pattern / Psychosocial) is insignificant compared to a 40% prediction (achieved by FOTO’s original baseline model).
We recommend for future studies to examine what variables added sequentially in a single model have the best predictive capabilities. If we look at this data from the perspective of what variables explain the largest amount of variance, things appear different. The DISCUSSION section of the article highlights these important facts and expands upon the clinical importance of interpreting classification beta coefficients.
TABLE 5
MODEL 1 2 3 4 5 6 7 8
(permission granted from JOSPT to use this table)
PLACING THE RESULTS IN PERSPECTIVE
Predictive models for patient functional change are seeking the GOLD STANDARD of 50%. The gold standard would be able to explain 50% of the variation in patient outcomes from start to finish of an episode of care. However, this gold standard does not yet exist.
The highest predictive capabilities to date in published literature is FOTO at 35 - 40%. If you remember, combining MDT / Pain Pattern with psychosocial (SCL-BPPM) resulted in a 3 - 4 % prediction of outcomes. When you add MDT / Pain Pattern / SCL-BPPM to FOTO, you have a predictive capability of nearly 44%. That is a TREMENDOUS FEAT!
Considering that the variables used to account for FOTO’s numbers have as little as 1% prediction, a variable that demonstrates 3% is on that scale BIG.
Reality = MDT Classification (~3%) is a BIG / STRONG variable in predicting outcomes.
Reality = Pain Pattern Classification (~3%) is a BIG / STRONG variable in predicting outcomes.
The literature is filled with studies demonstrating the importance of psychosocial variables. FABQ was demonstrated to contribute nothing to the prediction of functional outcomes for patients. The SCL-BPPM was shown to be a significant single variable at 1%. Compared to the single variable of MDT Classification or Pain Pattern Classification, psychosocial variables predictive ability is not nearly as important. Once again, this study supports previous findings that eliciting or failing to elicit Centralization / classifying or failing to classify as Derangement is a stronger predictor of patient outcomes then psychosocial variables.
Secondary findings observed trends in outcomes related to McKenzie level of postgraduate education / training and the treating therapist. Dip.MDT achieved significantly greater functional scale outcomes then those with Cert.MDT. However, the treating therapist was also a greater predictor of functional change then the level of MDT training. Essentially, clinician characteristics that drive them to pursue advanced training may have an increased desire to excel professionally and develop stronger therapeutic alliances with patients.
TAKE AWAY MESSAGE
This and other powerful literature supporting MDT published in peer-reviewed journals is the end result of the hard work and dedication of our MDT research group dedicated to collecting data on a daily basis in the clinic to scientifically expand upon the MDT literature and to report on the merits of what we observe during every day practice.
We, as clinicians, are learning every day a bit more about what is best treatment and why some treatments are more beneficial than others. If you want to be a force in molding where the profession is going, collect data then join your colleagues on FOTO. It will be a humbling experience and one that will challenge you to be the best clinician you can be.
Please feel free to contact me, Brian McClenahan, bmcclen@gmail.com, with any questions or let me know if you are interested in joining our MDT research group. Become active in research driven by clinical practice. Walk the walk. Don’t just talk the talk!
LET THE SYSTEM BE YOUR GUIDE.
References:
- Hart DL, Deutscher D, Werneke MW, Holder J, Wang YC. (2010). Implementing computerized adaptive tests in routine clinical practice: experience implementing CATs. Applied Meas;11(3):288-303.
- Hart DL, Mioduski JE, Werneke MW, Stratford PW. (2006). Simulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function. J Clin Epidemiol; 59(9):947-956.
- Hart DL, Stratford PW, Werneke MW, Deutscher D, Wang Y-C. (2012). Lumbar computerized adaptive test and modified Oswestry Low Back Pain Disability Questionnaire: relative validity and important change. JOSPT; 42(6):541-51.
- Hart DL, Werneke MW, Wang YC, Stratford PW, Mioduski JE. (2010). Computerized adaptive test for patients with lumbar spine impairments produced valid and responsive measures of function. Spine; 35(24):2157-2164.
- Resnik L, Liu D, Hart DL, Mor V. (2008). Benchmarking physical therapy clinic performance: statistical methods to enhance internal validity when using observational data. Phys Ther; 88(9):1078-1087.