Hobart_edit.qxp 30/7/08 12:42 pm Page 15
Rating Scales for Clinical Studies in Neurology—Challenges and Opportunities
therefore, CTT has been called Weak-True Score Theory,
29,30
a tautology,
31
Figure 3: The Rivermead Mobility Index Also Represented
and a theory that has no theory.
28
As a ‘Ruler’
Solution 1—New Psychometric Methods
A
The solution to the first problem is to use new psychometric methods when
Please tick ‘No’ or ‘Yes’ for each question
No Yes
constructing and evaluating rating scales and when analyzing rating scale
1. Turning over in bed
data. These methods, known as Item Response Theory (IRT)
30,32–34
and Rasch
Do you turn over from your back to your side without help? boxopen ✔
measurement,
11,35–37
constitute the ‘something (that) must be done to turn
2. Lying to sitting
counts into measurements’ we mentioned above. Essentially, new
From lying in bed, do you get up to sit on the edge of bed on your own? boxopen ✔
psychometric methods are mathematical models that articulate the
3. Sitting balance
conditions (measurement theories) under which equal interval Do you sit on the edge of the bed without holding on for 10 seconds? boxopen ✔
measurements can be estimated from rating scale data. Thus, when rating 4. Sitting to standing
scale data satisfy (fit) the conditions required by these mathematical models,
Do you stand up (from any chair) in less than 15 seconds (using hands,
the estimates derived from the models are considered robust because the
and with an aid if necessary)? boxopen ✔
measurement theory is supported by the data. When data do not fit the
5. Standing unsupported
chosen model, two directions of inquiry are possible. Essentially, albeit simply
Observe standing for 10 seconds without any aid ✔ boxopen
6. Transfer
stated, when the data do not fit the chosen model, the IRT approach is to
Do you manage to move from bed to chair and back without any help? ✔ boxopen
7. Walking inside, and with an aid if needed
Do you walk 10 metres with an aid if necessary, but with no standby help? boxopen ✔
There is no doubt that both Item
8. Stairs
Response Theory and Rasch
Do you manage a flight of stairs without help? ✔ boxopen
9. Walking outside (even ground)
measurement offer substantial
Do you walk around outside on pavements without help? boxopen ✔
10. Walking inside with no aid
advantages over Classical Test Theory
Do you walk 10 meters inside with no calliper, splint, or aid, and no
for neurology research.
standby help? boxopen ✔
11. Picking off the floor
If you drop something on the floor, do you manage to walk 5 meters,
pick it up, and then walk back? boxopen ✔
find a mathematical model that best fits the observed item response data; in
12. Walking outside (uneven ground)
contrast, the Rasch measurement approach is to find data that better fit one
Do you walk over uneven ground (grass, gravel, dirt, snow, ice, etc.)
model (the Rasch model). Thus, it follows that proponents of IRT use a family
without help? ✔ boxopen
of item-response models, while proponents of Rasch measurement use only
13. Bathing
one model (Rasch model). Do you get in/out of bath or shower unsupervised and wash yourself? ✔ boxopen
14. Up and down four steps
There is no doubt that both IRT and Rasch measurement offer substantial
Do you manage to go up and down four steps with no rail, but using
advantages over CTT for neurology research. Other advantages, beyond the
an aid if necessary? ✔ boxopen
scope of this article, include item banking, scale equating, computerized
15. Running
scale administration, and the handling of missing data. As such, clinicians
Do you run 10 meters without limping in four seconds
(fast walk is acceptable)? ✔ boxopen
should be actively looking to apply these methods in the future. However,
Score (total number of ‘Yes’ responses) = 8
which approach is better, and does it matter which approach is used?
The answer to both questions depends on which central philosophy is
B
Person
followed, as this divides proponents of IRT and Rasch measurement. As IRT
prioritizes the observed data, it sees the Rasch perspective of using only one
12345678 9 10 1112131415
model as too restrictive and the ‘selection’ of data to meet that model as
Less Mobility More
threatening to content validity.
38,39
As Rasch measurement prioritizes the
mathematical model, it sees the process of modeling data as precluding the
A shows the Rivermead Mobility Index (RMI), a 15-item scale for measuring mobility. Each item, a
mobility-related task, has two response categories: ‘No’ = I am unable to do this task (scores 0) or
ability to achieve core requirements of measurement, too accepting of poor
‘Yes’ = I am able to do this task (scores 1). The RMI is clinician-scored by interview and
quality data, and threatening to construct validity. Not surprisingly, it has
observation. Item scores are summed to give a total score that ranges from 0 (all ‘No’ responses)
to 15 (all ‘Yes’ responses). This number (here = 8) is taken to be a ‘measure’ of the variable
been suggested that IRT and Rasch measurement have irreconcilable
quantified by the set of items (mobility). B illustrates the RMI mobility variable as a ‘ruler’ of a
differences, and the two groups have come into conflict regarding which
count up to 15 points, which represents the number of questions that can be ticked ‘Yes.’ For
convenience, the items are ordered sequentially at roughly equal intervals. The mark represents
approach is preferable.
40
the location of a person who scored 8 on the mobility variable.
Problem 2—Exactly What Do Scales Measure? constructs that are clinically meaningful and interpretable. Unfortunately,
Pivotal clinical trials obviously require rating scales that measure the current methods of establishing rating scale validity rarely enable these
health constructs they purport to measure (i.e. are valid) and health goals to be confirmed. To appreciate this opinion, some scale basics must
US NEUROLOGY 15
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76 |
Page 77 |
Page 78 |
Page 79 |
Page 80 |
Page 81 |
Page 82 |
Page 83 |
Page 84