What are process evaluations? Why are process evaluations important?

Question

What will be an ideal response?

Answer 1

What actually happens in a program? Once a program has been started, evaluators are often called on to document the extent to which implementation has taken place, whether the program is reaching the target individuals or groups, whether the program is actually operating as expected, and what resources are being expended in the conduct of the program. This is often called process evaluation or program monitoring. Rossi and Freeman (1989) define program monitoring as the systematic attempt by evaluation researchers to examine program coverage and delivery. Assessing program coverage consists of estimating the extent to which a program is reaching its intended target population; evaluating program delivery consists of measuring the degree of congruence between the plan for providing services and treatments and the ways they are actually provided.
Process evaluation (program monitoring) is evaluation research that investigates the process of service delivery.
Process evaluations are extremely important, primarily because there is no way to reliably determine whether the intended outcomes have occurred without being certain the program is working according to plan. For example, imagine you are responsible for determining whether an anti-bullying curriculum implemented in a school has been successful in decreasing the amount of bullying behavior by the students. You conduct a survey of the students both before and after the curriculum began and determine that rates of bullying have not significantly changed in the school since the curriculum started. After you write your report, however, you find out that, instead of being given in a five-day series of one-hour sessions as intended, the curriculum was actually crammed into a two-hour format delivered on a Friday afternoon. A process evaluation would have revealed this implementation problem. If a program has not been implemented as intended, there is obviously no need to ask whether it had the intended outcomes.
A process evaluation can take many forms. Because most government and private organizations inherently monitor their activities through such things as application forms, receipts, and stock inventories, it should be relatively easy to obtain quantitative data for monitoring the delivery of services. This information can be summarized to describe things such as the clients served and the services provided. In addition to this quantitative information, a process evaluation will also likely benefit from qualitative methodologies such as unstructured interviews with people using the service or program. Interviews can also be conducted with staff to illuminate what they perceive to be obstacles to their delivery of services.
Process evaluation can employ a wide range of indicators. Program coverage can be monitored through program records, participant surveys, community surveys, or number of utilizers versus dropouts and ineligibles. Service delivery can be monitored through service records completed by program staff, a management information system maintained by program administrators, or reports by program recipients (Rossi & Freeman, 1989).
Qualitative methods are often a key component of process evaluation studies because they can be used to understand internal program dynamics, even those that were not anticipated (Patton, 2002; Posavac & Carey, 1997). Qualitative researchers may develop detailed descriptions of how program participants engage with each other, how the program experience varies for different people, and how the program changes and evolves over time.