Program process monitoring

Apart from measuring the needs, inputs and outcomes of a program, evaluations also monitor the process of a program.^[1] According to ^[2](p. 171), program process monitoring "is the systematic and continual documentation of [the] key aspects of program performance that assesses whether the program is performing as intended or according to some appropriate standard." Therefore, program process monitoring involves a systematic and continuous way of monitoring certain aspects of a program's process which would indicate how well the program is performing, if at all. These activities help to facilitate the effective management of the program because continuous assessment allows for regular feedback about the program's performance.^[2] to

Management Information Systems (MIS) are tools often used by evaluators to monitor the performance of the program. An MIS is a compilation of data regarding the services that are routinely provided by the program, the staff that have provided the services, the treatments that have been provided and the costs involved in the program. Many evaluators use MISs as their major source of data when monitoring the process of a program because MISs contain all the relevant information about the program that is needed for an assessment.^[2]

The specific activities that form part of the monitoring process are aimed at generating information that is appropriate from the perspective of the consumer group, who the monitoring is for.^[2] Three key consumer groups are discussed below:

Due to outside funding, program managers need to provide regular information to funders and program sponsors about the implementation of the program, problems that have been encountered and how these were handled. Program managers are accountable for and how the program is run, which leads to either the success or failure of the program.^[2]

From a management perspective, process monitoring is not only aimed at finding out how the program is going, but it is also aimed at putting in place corrective measures to ensure that the program performs as it should. In this case, process monitoring can take place during the pilot testing of the program in order to find ways of dealing with unexpected problems. Management-oriented monitoring can also take place in already-developed programs in order to obtain information about the performance of the program and whether or not the target population is being reached by the planned intervention.^[2]

In order to evaluate the outcomes of a program, the evaluator first needs to monitor the process in order to assess the implementation of the intervention. The reason for this is that many program failures are due to failures in the implementation of the program. Therefore, in order to determine whether or not the planned outcomes have been reached, the evaluator needs to assess how the intervention was implemented.^[2]

Program process evaluation is an assessment of how the program is performing in terms of service utilisation and program organisation. Service utilisation examines program coverage in terms of intended services for the intended target population whereas program organisation looks at whether services provided are the ones actually laid down in the program.

According to ^[3](p. 136), an objective of many program projects is to ensure that project services or benefits reach a certain target population - defined in terms of its geographic, economic, or demographic characteristics. What is critical in program process monitoring is determining whether intended targets actually receive program services intended for them in the plan. This is of paramount importance especially in programs that are voluntary, that involve behaviour change, taking of instructions or learning of new procedures.^[2]

Coverage refers to the extent to which a program reaches its intended target population whereas bias is the extent to which subgroups of a target population are reached unequally by a program ^[2](p. 200). The aim of all programs is total coverage but very few social programs ever achieve total coverage, making bias an issue.

Causes of bias are, among others,:

• self-selection: participation of some groups more frequently than others;
• program actions: acting favourably to some subgroups while rejecting others; and
• creaming: selecting most success-prone targets.^[2]

Some programs can experience overcoverage, whereby the program captures numbers far exceeding the intended targets (e.g. feeding schemes). Depending on the nature of the program, overcoverage can be costly and problematic (e.g. social grants). However, most programs fail to achieve high target participation, either because of bias in the way targets are recruited or retained or because potential clients are unaware of the program, are unable to use it, or reject it.^[2](p. 185).

Overcoverage and undercoverage are basic concerns in program implementation and should be measured and monitored to ascertain the extent of their effect on service utilisation. Efficient service utilisation requires that the program serve as many people as possible who are in need and very few who are not in need. A drawback to this exercise is that people may distort information so as to be eligible for the project .^[3][2] identified three sources of information that can be used to assess the extent to which a program is serving the appropriate target population:

• program records;
• surveys of program participants; and
• community surveys.

In assessing bias, questions like: Do all eligible individuals participate in the program? Are there any dropouts? What is the dropout rate? What are the causes of non-participation or dropout?, could be asked. Information obtained from answers to these questions is valuable in judging the effectiveness, worth and possible modification of the program to suit the needs of the target population.^[2] The same methods of data collection mentioned under 'measuring and monitoring coverage' could be used to assess bias.