Interrater reliability of the Wolf Motor Function Test-Functional Ability Scale: why it matters - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015 Jun;29(5):436-43.
doi: 10.1177/1545968314553030. Epub 2014 Oct 16.

Interrater reliability of the Wolf Motor Function Test-Functional Ability Scale: why it matters

Susan V Duff  1 , Jiaxiu He  2 , Monica A Nelsen  3 , Christianne J Lane  3 , Veronica T Rowe  4 , Steve L Wolf  5 , Alexander W Dromerick  6 , Carolee J Winstein  7
Affiliations

Interrater reliability of the Wolf Motor Function Test-Functional Ability Scale: why it matters

Susan V Duff et al. Neurorehabil Neural Repair. 2015 Jun.

Abstract

Background: One important objective for clinical trialists in rehabilitation is determining efficacy of interventions to enhance motor behavior. In part, limitation in the precision of measurement presents a challenge. The few valid, low-cost observational tools available to assess motor behavior cannot escape the variability inherent in test administration and scoring. This is especially true when there are multiple evaluators and raters, as in the case of multisite randomized controlled trials (RCTs). One way to enhance reliability and reduce variability is to implement rigorous quality control (QC) procedures.

Objective: This article describes a systematic QC process used to refine the administration and scoring procedures for the Wolf Motor Function Test (WMFT)-Functional Ability Scale (FAS).

Methods: The QC process, a systematic focus-group collaboration, was developed and used for a phase III RCT, which enlisted multiple evaluators and an experienced WMFT-FAS rater panel.

Results: After 3 staged refinements to the administration and scoring instructions, we achieved a sufficiently high interrater reliability (weighted κ = 0.8).

Conclusions and implications: A systematic focus-group process was shown to be an effective method to improve reliability of observational assessment tools for motor behavior in neurorehabilitation. A reduction in noise-related variability in performance assessments will increase power and potentially lower the number needed to treat. Improved precision of measurement can lead to more cost-effective and efficient clinical trials. Finally, we suggest that improved precision in measures of motor behavior may provide more insight into recovery mechanisms than a single measure of movement time alone.

Keywords: assessment; impairment; motor control; observational; quality; stroke.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Flow diagram. The flow of data from initial administration of the WMFT by the BE to uploading of scores to the FTP site by the Rater Panel member is shown. WMFT = Wolf Motor Function Test, BE = Blinded Evaluator, FTP = File Transfer Protocol.
Figure 2
Figure 2
Timeline for systematic quality control process. Top – Inter-rater reliability (quadratic weighted kappa) for the WMFT-FAS Rater Panel overall for each round of analysis for each of the three rater pairs. The number of digital files (n) included in each round is listed. Bottom – Timeline of events (with dates) for the ICARE trial including revisions to the WMFT-FAS administration and scoring instructions (ASI) and other notable events before, during and after the 5 rounds of analysis. QCh = quality check conducted to ensure essential elements were met for each digital file.
Figure 3
Figure 3
Inter-rater reliability by task at Round 5. The quadratic weighted kappa values between members of the WMFT-FAS Rater Panel are shown for each task. The fifteen timed tasks are: 1) Forearm to table; 2) Forearm to box; 3) Extend elbow; 4) Extend elbow with weight; 5) Hand to table; 6) Hand to box; 8) Reach and retrieve; 9) Lift can; 10) Lift pencil; 11) Lift paper clip; 12) Stack checkers; 13) Flip cards; 15) Turning key in lock; 16) Fold towel; and 17) Lift basket. [Note that the two force tasks are not listed, but the full WMFT task numbering has been retained].
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Kitago T, Liang J, Huang VS, et al. Improvement after constraint-induced movement therapy: recovery of normal motor control or task-specific compensation? Neurorehabil Neural Repair. 2013;27:99–109. - PubMed
    1. van Kordelaar J, van Wegan EE, Kwakkel G. The impact of time on quality of motor control of the paretic upper limb after stroke. Arch Phys Med Rehabil. 2014;95:338–44. - PubMed
    1. Alberts JL, Butler AJ, Wolf SL. The effects of constraint-induced therapy on precision grip: a preliminary study. Neurorehabil Neural Repair. 2004;18:250–8. - PMC - PubMed
    1. Dobkin BH, Dorsch A. The promise of mHealth: daily activity monitoring and outcome assessments by wearable sensors. Neurorehabil Neural Repair. 2011;25:788–98. - PMC - PubMed
    1. Wade E, Chen C, Winstein CJ. Spectral analysis of wrist motion in individuals poststroke: the development of a performance measure with promise for unsupervised settings. Neurorehabil Neural Repair. 2013;28:169–78. - PMC - PubMed

Publication types