Whenever a program is implemented there are always ways for things to go wrong. Treatment fidelity is a term used to describe how programs are not implemented as intended in the grant proposal. Below is a list of common ways that treatment fidelity can become a problem
- Adherence to implementation
- Implementation incompetence
- Variations in treatment
- Program drift
We will look at each of these below
Adherence to Implementation
Implementation adherence is whether the provider of the program follows the intended procedures. For example, if we have a reading lab program to boost students’ reading comprehension. The procedures may be as follows.
- Fifth-grade students are to use the reading lab on Monday, Wednesday, and Friday for 30 minutes each. (Dosage)
- The students must be engaged actively in using the reading software
If the provider wanders from these procedures it can quickly become an implementation issue. This is common. A teacher may take their kids on a field trip, there could be holidays, the teacher might do 1 hour one day and skip another day, etc. In other words, providers agree to a program but essentially do what they want when necessary. Every time these modifications happen it impacts the quality of the results as other factors are introduced into the study that were not originally planned for.
Implementation Competence
Implementation competence is defined as the provider’s ability to follow directions. If the procedures are too complicated the provider may not be able to follow them for the benefit of the students in the program.
An example would be if a provider is not comfortable with using computers and the reading software they may not be able to help students who are having technical issues. If too many students are unable to use the computers because the provider or teacher cannot help them this could lead to implementation competence concerns.
Difference in Treatment
The difference in treatment means that the treatment that the participants in the program receive should not be the same as participants who are not in the program. The treatments must be different so that comparisons can be made.
Sometimes when a new program is implemented providers will want all students to experience it. In our reading lab example, the procedures might call for allowing only half of the fifth-graders below grade level in reading comprehension. However, a teacher might decide to have all students participate in the reading lab because of the obvious benefits. If this happens, there is no way to compare the results of those who participate and those who do not.
Such well-meaning actions may benefit the students but damage the scientific process. It is always critical that there are differences in treatment so that it can be determined if the treatment makes a difference.
Program Drift
Program drift is the gradual weakening of the implementation of a program. People naturally lose discipline over time and this can apply to obeying the procedures of a program. For example, a provider might vigilantly follow the procedures of the reading lab in the beginning but may slowly allow more or less time for the students.
Program drift is hard to notice. One way to prevent it is to constantly re-train providers so that they are reminded about how to implement the program. Retraining is beneficial when providers want to implement the program correctly.
Conclusion
Treatment fidelity is critical to determine the quality and influence of a program. Evaluators need to be familiar with these common threats to fidelity so that they can provide the needed support to help providers.