Article Text
PDF
Abstract
Background The endurance shuttle walking test (ESWT) has shown good responsiveness to interventions in patients with chronic obstructive pulmonary disease (COPD). However, the minimal important difference (MID) for this test remains unknown, therefore limiting its interpretability.
Methods Patients with COPD who completed two or more ESWTs following pulmonary rehabilitation (n=132; forced expiratory volume in 1 s (FEV1) 48±22%) or bronchodilation (n=69; FEV1 50±12%) rated their performance of the day in comparison with their previous performance on a 7-point scale ranging from −3 (large deterioration) to +3 (large improvement). The relationship between subjective perception of changes and objective changes in performance during the shuttle walk was evaluated.
Results Following pulmonary rehabilitation, the anchor-based approach did not allow a valid estimation of the MID in the ESWT performance to be obtained. After bronchodilation, patient ratings of change correlated significantly with the difference in walking distance (r=0.53, p<0.001) and endurance time (r=0.55, p<0.001). For the pharmacotherapy data, regression analysis indicated that a 65 s (95% CI 45 to 85) change in endurance time and a 95 m (95% CI 60 to 115) change in walking distance were associated with a 1-point change in the rating of change scale. These changes represented 13–15% of the baseline values.
Conclusions A change in endurance shuttle walking performance of 45–85 s (or 60–115 m) after bronchodilation is likely to be perceived by patients. This MID value may be specific to the intervention from which it was derived.
- COPD
- exercise
- shuttle walking test
- minimal important difference
- COPD mechanisms
- exercise
- pulmonary rehabilitation
Statistics from Altmetric.com
- COPD
- exercise
- shuttle walking test
- minimal important difference
- COPD mechanisms
- exercise
- pulmonary rehabilitation
Introduction
Exercise testing has become a key approach to evaluate the functional impact of chronic obstructive pulmonary disease (COPD) treatments because it allows the assessment of exercise tolerance and exertional dyspnoea, two fundamental outcomes in patients with COPD.1 While maximal progressive cycle ergometry remains the most widely used exercise test in clinical practice,1 constant-load tests have gained popularity in recent years because of their enhanced sensitivity to various interventions.2–9
The endurance shuttle walking test (ESWT), an externally paced constant-speed field walking test, has shown high responsiveness to treatment in patients with COPD.10–13 In a randomised, double-blind, placebo-controlled and crossover trial, the ESWT was found to capture larger and more consistent improvements in exercise performance after the administration of ipratropium bromide compared with a constant-load cycling exercise test performed at 80% of peak work rate.11
Based on the evidence accumulated to date, the ESWT seems particularly suited to detect functional improvements after treatment in patients with COPD. Furthermore, like other field tests, the ESWT has the advantage of requiring minimal equipment and less technical expertise than laboratory-based tests, thereby being less expensive and easier to administer.5 It is also perceived as being more reflective of daily living than laboratory-based tests (because it employs a daily living activity; ie, walking on a flat surface) and yet being more standardised than self-paced walking tests (eg, the 6 minute walk test (6-MWT)) because it imposes the walking speed and is thereby less affected by patient motivation and pacing ability.5 14
However, the minimal important difference (MID) for the ESWT has yet to be established, limiting the interpretability of observed changes in performances. The MID corresponds to the smallest change that can be perceived by patients.15 By providing an indicator of the amplitude of change needed for patients to detect a significant difference in a given outcome variable, the MID allows clinicians and researchers to interpret findings from clinical trials beyond their statistical significance by considering clinical meaningfulness. While there are many acknowledged methods to estimate the MID, there is no consensus on which method is to be favoured. However, it is recognised that using multiple methods to triangulate the MID is a valid path.16
The present study was undertaken to determine, in patients with COPD, the MID for the ESWT. Specifically, our objectives were to use data from both pulmonary rehabilitation and bronchodilation studies to: (1) evaluate the degree of association between objective and subjective measures of changes in ESWT performance after these interventions; and (2) provide an estimate of the MID for the ESWT. We hypothesised that we would be able to determine an MID value for the ESWT from bronchodilation and pulmonary rehabilitation studies. Some of the results of this study have been reported previously in the form of an abstract.17
Methods
Settings
Data were obtained from two distinct settings: pulmonary rehabilitation studies from Leicester and Nottinghamshire, UK; and bronchodilation studies from the Institut Universitaire de cardiologie et de pneumologie de Québec, Canada.
Rehabilitation data were collected from two separate cohorts of patients with COPD that underwent pulmonary rehabilitation. Participants from both cohorts participated in a 7 week programme which included aerobic and resistance exercises, three times per week.18 Participants completed two ESWTs, one at baseline and one after the rehabilitation programme, amounting to one comparison point per participant. A total of 132 patients were included in the two programmes, resulting in 132 comparison points.
Bronchodilation studies were conducted as part of a clinical research programme seeking to evaluate the responsiveness of the ESWT to detect improvements in exercise capacity following bronchodilation in patients with COPD. This programme was also initially set up to estimate the MID of the ESWT. While the data pertaining to the ESWT's responsiveness to bronchodilation have been published,11 19 20 none of the MID data has been reported, except in the form of an abstract.17 Study participants completed between two and four ESWTs as part of these trials, such that the number of comparison points obtained per participant varied from one to three. From a total of 69 patients, 146 comparison points were thus obtained. These studies used a randomised double-blind controlled crossover methodology. Because the order of drug administration was randomised, the second ESWT of any comparison pair could be done under placebo or bronchodilation conditions. As a result of this study design, deterioration, status quo or improvement could be perceived after the second test of a pair in comparison with the first test. The time between two ESWTs of the same pair varied from 2 to 14 days. The detailed methodology for these trials has been reported previously.11 19 20
Endurance shuttle walking test
The ESWT was performed in an enclosed corridor on a flat 10 m long course, following the procedure described by Revill and colleagues.10 After a 90 s warm-up, walking speed was set at the pace corresponding to 80–85% of Vo2 peak, which was estimated from the distance walked during a previously completed incremental shuttle walking test (ISWT).21 Before each test, participants were instructed to walk for as long as possible at the speed dictated by the auditory signal. The endurance time as well as the distance covered in that time were recorded. The warm-up period was excluded from this analysis.
Global rating of change
Immediately after each ESWT (with the exception of the initial one) and before any feedback was given about the test's result, participants were asked the following question: ‘In comparison with your previous test, how would you rate your performance on today's test using the present scale?’ Patients rated their performance on a 7-point Likert scale. The scale ranged from −3 to +3 and included the following ratings: −3 (large deterioration), −2 (moderate deterioration), −1 (slight deterioration), 0 (no change), 1 (slight improvement), 2 (moderate improvement) and 3 (large improvement).15 22
Statistical analyses
The absolute (delta) and relative (%) difference between consecutive ESWT performances (expressed in both seconds and metres) was computed for each participant. The degree of association between the objective measures of change in ESWT performance and the participant's ratings of that change was assessed with Pearson correlations. The MID was then estimated using both distribution- and anchor-based approaches. For the distribution-based method, the MID was calculated as half the SD of the changes in ESWT endurance time and distance.23 For the anchor-based approach, we investigated the statistical relationship between the patient's rating of change and improvements in ESWT performance using models. First, Pearson correlations between participant rating and changes in ESWT endurance time or distance needed to be ≥0.5 to obtain valid regression equations.24 If this condition was fulfilled, participant rating of change (independent variable) from each intervention group was plotted in a linear regression model against the absolute or relative (% baseline value) change in ESWT performance. The resulting slopes, which represented the change needed in walking time or distance for participants' ratings to move one unit on the Likert perception scale, were considered as estimates of the MID. For the bronchodilation data subset, since each participant could contribute more than one comparison point, mixed regression models were adjusted to the data in order to account for potential dependency between observations. In the bronchodilation studies, three outlying observations were identified using the Cook distance measure and the DFFITS statistics; they were removed from the regression analysis. The analyses were done using the Mixed Procedure of SAS (SAS 9.2, SAS Institute, Cary, North Carolina, USA).
Results
Characteristics of the two study subgroups are presented in table 1. Overall, the sample consisted of participants with mild to severe airflow obstruction and varying exercise capacities, as indicated by the wide range of forced expiratory volume in 1 s (FEV1) values and performance on the ISWT and the ESWT. Patients in the pulmonary rehabilitation group were older, included a higher proportion of women and displayed significantly lower FEV1 (litres), forced vital capacity (FVC; litres), ISWT distance (m) and ESWT distance (m), time (s) and speed (m/s) compared with patients in the pharmacotherapy group.
Baseline characteristics of the study group (n=201)
Tables 2 and 3 show changes in ESWT time and distance following the two study interventions and according to perceived change (−3 to +3). After pulmonary rehabilitation, patient ratings of change correlated significantly with the changes in ESWT time expressed in seconds (r=0.37, p<0.0001) and percentage change (r=0.16, p=0.06), and with the changes in ESWT distance expressed in metres (r=0.35, p<0.0001) and in percentage change (r=0.17, p<0.05). Correlations between subjective and objective changes were stronger with pharmacotherapy than with pulmonary rehabilitation. In the bronchodilation studies, patient ratings of change correlated significantly with the change in ESWT time expressed in seconds (r=0.55, p<0.0001) and percentage change (r=0.59, p<0.0001), and with the changes in ESWT distance expressed in metres (r=0.53, p<0.0001) and in percentage change (r=0.59, p<0.0001).
Improvements in ESWT time according to perception of change
Improvements in ESWT distance according to perception of change
Using the distribution-based approach (half a SD), pulmonary rehabilitation data suggested an MID of 186 s or 203 m, which corresponded to a 136% change in ESWT performance. Our pharmacotherapy data resulted in an MID of 70 s or 115 m, which corresponded to a 15% change in ESWT performance.
With the rehabilitation data, obtaining a valid estimation of the MID using the anchor-based method was not possible because of the weak correlations between the anchors and the measured change in ESWT performance. In fact, using the perception of change as an anchor for the rehabilitation data resulted in estimations of the MID with wide CIs that included zero.
Results of the anchor-based approach to determine the MID for the pharmacotherapy data show that a 65 s (95% CI 45 to 85) change in the endurance time corresponding to an 85 m (95% CI 60 to 115) change in walking distance was associated with minimally important change. When expressed in relative values, the estimated MID was 13% (95% CI 9% to 17%) for the ESWT endurance time and 15% (95% CI 10% to 19%) for the ESWT distance. The anchor-based models resulted in coefficients of determination (R2 statistics) values of 0.261 and 0.267 for the absolute and relative value model, respectively.
Discussion
The aim of this study was to determine the MID of change for the ESWT and to verify whether this threshold would be similar for two different types of therapeutic interventions. We found that a change of 65 s (95% CI 45 to 85) or 85 m (95% CI 60 to 115), representing 13–15% of the baseline value, was considered meaningful in the pharmacotherapy trials. However we could not confirm this MID estimate with the rehabilitation intervention. These results suggest that the MID determined for one type of intervention may not necessarily apply to another.
Correlation between ESWT performance and the anchor (perceived change scale) was sufficient to use regression analysis for the determination of an MID value in the bronchodilation studies.25 The distribution- and anchor-based methods also provided very similar estimates of MID in the bronchodilation studies. The strong agreement between the two methods of estimating the MID further confirms the validity and the robustness of our findings.
We could not confidently determine an MID estimate in the rehabilitation studies. We see several explanations for why the anchor-based method failed to yield a convincing estimate of the MID in the rehabilitation group compared with the pharmacotherapy group. First, recall time in the medication studies was only a few days, but 7 weeks separated the two ESWT performances in the rehabilitation group. Recall accuracy decreases as time passes,26 even more so in older individuals.27 The longer recall time combined with older age in the rehabilitation group might have introduced a recollection error28 that affected the correlation between subjective and objective change.
Secondly, the one-item question used to evaluate patient rating of change essentially relies on the fact that a patient's internal perception of his health does not change over the course of the intervention,29 so that the frame of reference of the study question remains the same. Because pulmonary rehabilitation has a bigger potential to affect a patient's perception of his health than pharmacotherapy,30 the post-treatment state in the former group was probably more altered when compared with the medication group. In addition, the longer the time span, the more likely it is that other health conditions (eg, exacerbations) may influence a patient's present perception29 31 and alter the validity of the memory recall.
Thirdly, the differences in study design between interventions may also have had a significant impact on the MID calculated using the perception of global change. The medication arm used a double-blind, placebo-controlled, crossover study design: patients could therefore expect improvements, status quo or deterioration in their ESWT performance. This design might have had an impact on the expectations of the patients and might have required them to be more critical in their evaluation of change. In contrast, patients undergoing pulmonary rehabilitation were not subjected to the possibility of sham training. Hence, they more probably expected and desired improvements in ESWT performance.
Finally, the level of commitment and personal investment in the treatment is likely to influence the magnitude of improvement considered worth obtaining with a given treatment. The time and effort involved with pulmonary rehabilitation is greater compared with pharmacotherapy; it is likely that patients might judge their performance differently because of greater investment in resources.
Irrespective of the mechanisms involved, the finding that the MID may vary according to the nature of the intervention and baseline performance is important. In clinical practice, the MID that was found for the 6-MWT during pulmonary rehabilitation32 was subsequently applied to pharmacotherapy trials33 and to diseases other than COPD34: the present findings certainly question this practice. Investigators should be careful when applying the MID value obtained in a specific clinical context with that in a different clinical situation. We would suggest that further studies be done to try and define an MID estimate for pulmonary rehabilitation using anchors with sufficient correlation with the improvements in ESWT performance.
Another important issue is whether a unique absolute MID value applies throughout the range of baseline ESWT performance or whether the MID estimates should be expressed as a fraction of the baseline values. In the field of COPD research, MID values have usually been reported as fixed values, expressed in the unit of the instrument.32 35 36 The underlying assumption is that the MID value is identical across the range of observed scores. In the field of psychophysics, extensive research has been done on this matter, albeit on less complex constructs than exercise performance, and it may provide important insight. Studies have shown that the just noticeable difference between two stimuli is dependent on the intensity of the original stimulus.37–39
To address this possibility, we tested whether a model in which the changes in ESWT walking performance were expressed as a percentage of baseline values would be superior to the absolute value approach. We did not find any important differences in the accuracy of the MID estimation between the absolute and relative model; this would seem to indicate that a unique absolute MID values might apply across the range of ESWT performance.
The concept of MID seeks a patient-centred perspective in the interpretation of results. However, strictly speaking, we have defined in the present study a threshold of change in ESWT that could be perceived either positively or negatively by the patients. As others,32 we refer to this threshold as the MID. This relates more to the threshold of perception than a true significance of clinical change.
Despite the fact that they both involve walking, the ESWT and the 6-MWT have very different designs and properties. As such, the choice of the test that should be used in a given clinical or research situation should be based on the specific question being asked. The discriminative properties of the 6-MWT have been established in patients with COPD40–42 and in other pulmonary diseases.43 The 6-MWT is also easy to administer, although its methodology should be well standardised to optimise the validity of the results.44 The estimation of the MID value for the 6-MWT has been the subject of several investigations.32 45–48 Although the 6-MWT is responsive to rehabilitation,49 its responsiveness to pharmacological interventions may not be optimal. For example, bronchodilation provides small and inconsistent improvement in the distance covered during the 6-MWT.50–55 In pulmonary hypertension, pharmacological interventions also only provide modest improvement in the 6 min walking distance.56 The clinical experience with the ESWT is not as extensive as with the 6-MWT. To our knowledge, our study is the first to provide an estimation of its MID. Whether the endurance time during the ESWT is predictive of important clinical outcomes such as mortality has not been reported. Despite these limitations, one appealing feature of the ESWT is its responsiveness to treatment, which has been shown to be superior to that of the 6-MWT, with both bronchodilation19 and rehabilitation.12 These differences in discriminative and evaluative properties between both walking tests is in part related to their design, the main point being that the 6-MWT is self-paced and the ESWT externally paced. Our current view is that the 6-MWT will perform well to quantify exercise tolerance or to provide prognostic information. In turn, the bulk of evidence indicates that the ESWT is more responsive to interventions, such as bronchodilatation and pulmonary rehabilitation, than the 6-MWT and incremental exercise protocols.10 12 Our study further supports the use of the ESWT in clinical trials evaluating the effectiveness of bronchodilatation treatment by providing an estimation of the MID, which will facilitate calculation of sample size and the interpretation of results.
Methodological considerations and potential limitations
A larger sample size for both the pulmonary rehabilitation and the bronchodilatation groups would have provided additional statistical power to estimate the MID. Despite the small difference in the degree of similar airflow obstruction between the two groups, the rehabilitation group had a lower exercise tolerance at baseline. In the bronchodilatation studies, the MID estimations were not influenced by the baseline exercise capacity, as shown by the similar predictive power of the absolute and the relative change value regression models. Also, there was overlap in exercise performance in the two groups. Based on this discussion, we submit that the failure to estimate an MID value from the rehabilitation data has more to do with the nature of the intervention that with a lower exercise tolerance found in this group at baseline. In a previous study, we succeeded in finding an MID value for the constant workrate cycling test in a pulmonary rehabilitation context. In this study, the anchor that was used was the St. George's Respiratory Questionnaire57 that is composed of four domains. It would be of interest to try to evaluate whether a multidimensional anchor would allow estimation of the MID value for the ESWT with pulmonary rehabilitation.
Among the various distribution-based approaches to estimate MID, we selected the half SD approach for its simplicity and also because it performs reasonably well at identifying the MID when compared with other, more complicated methods.23 58 It is however important to acknowledge that there are inconsistencies across the different distribution-based approaches58 and that they should be viewed as providing preliminary MID estimates until validated anchor-based approach estimates become available. To increase the confidence of our anchor-based MID estimate, the data were also analysed using additional regression models; using the baseline endurance time values as a covariate in the linear regression model did not significantly alter the MID estimate. We noted that the residuals of the linear regression model were not normally distributed; however, no transformations could satisfy the postulates of normality and the Box–Cox method recommended that we did not transform the data. To address this potential limitation, the data were transformed to satisfy normality according to a method proposed by Huber.59 The MID estimate provided by this model (56 s (CI 43 to 70)) fell within the 95% CI of the MID value from the linear regression model, providing reassurance about the validity of our findings.
Conclusion
The present study suggests that the nature of the intervention may affect the patients' perception of change after treatment. Our data set suggests that a change in ESWT performance between 45 and 85 s (or 60–115 m) for a small and large improvement, respectively, is likely to be perceived by patients, following a bronchodilatation intervention. We were unable, however, to estimate with confidence an MID value for the pulmonary rehabilitation data set, therefore suggesting that an MID determined for a given intervention might not necessarily be valid for another intervention. This observation warrants caution in how the MID concept is applied in interpreting clinical trials.
Acknowledgments
The authors acknowledge the help of Marthe Bélanger, Marie-Josée Breton, Brigitte Jean and Josée Picard in accomplishing this study. They also thank Gaétan Daigle for his statistical assistance and Eric Nadreau for his help with the exercise tests.
References
Footnotes
VP and LL contributed equally to the work.
Funding VP is a research scholar of the Fonds de la Recherche en Santé du Québec. FM holds a GSK/CIHR Research Chair on COPD at Université Laval. This work has been supported in part by CIHR grant MOP-84091.
Competing interests None.
Ethics approval This study was conducted with the approval of the Pulmonary Rehabilitation Research Group, Institute for Lung Health, Department of Respiratory Medicine and Thoracic Surgery, Glenfield Hospital, Leicester, UK; Sherwood Forest Hospitals, Ashfield Community Hospital, Kirby-in-Ashfield, Nottinghamshire, UK; and the Institut Universitaire de Cardiologie et de Pneumologie de Québec, Canada.
Provenance and peer review Not commissioned; externally peer reviewed.