Appendix E: How to Calculate Reliable Change Indexes (RCIs)
RCI starts by considering SEM for the particular progress-monitoring instrument used in each case (the Piers-Harris-2 in this instance example found in your Chapter 14). Thus, you must begin by going to a test manual and locating the SEM value for the score you will use (SEMs are usually found in a reliability-related chapter, often in a table of statistical values). But because you will consider two scores for each student (i.e., pre and post-treatment), RCI calculations cannot simply adopt unrefined SEM values. Instead you will use a statistic entitled “standard error of difference.” The standard error of difference accounts for the use of two test scores, each of which themselves is imperfectly reliable. It is the standard error of difference that will allow you to determine if a threshold-level of change has been attained relative to a real-world confidence level, for example, .90 or .95. Concerning the standard error of difference statistic, for those who prefer a formula, one follows:
Standard Error of the Difference: Sdiff = 2 √( SEM)2
Consulting the Piers-Harris-2 manual, one can see that the Total Score SEM is 3.07 (Piers & Herzberg, 2002). With this value in hand, step-by-step computations are shown in Table 15.2.
Table A.3.1 Steps in Calculating Reliable Change Index |
|||
| Steps related to standard error of the difference | Step 1 | Get SEM from the test manual† | 3.07 |
| Step 2 | Square SEM | 3.072 = 9.42 | |
| Step 3 | Multiply Step 2 by 2.0 | 9.42 X 2.0 = 18.84 | |
| Step 4 | Calculate square root of Step 3 | 4.34 | |
| Step 5 | Multiply value of Step 4 by 1.96 (for .05 confidence level) | 4.34 X 1.96 = 8.51 | |
| Additional steps for determining RC | Step 6 | Subtract pretest from post-test score | 35 – 24 = 11 |
| Step 7 | If a pre-post score difference > than 8.51, then a reliable change would be detected | Reliable change was detected | |
|
†Piers and Herzberg (2002)
|
|||
You now must decide a threshold level of confidence. If you desire 95% confidence, then standard error of difference is multiplied by 1.96. A last step is to subtract the student’s post-test score from pre-test score and compare any resulting difference with the number you just obtained from your RCI calculation. Again, if you don’t want to do calculations by hand, free online tools are available:
https://psychoutcomes.org/OutcomesMeasurement/ReliableChangeIndex.