Pay for Performance Programme Impact Evaluation in Pwani Region 2012

The health facility is the primary sampling unit. Facilities were sampled from those that were eligible to participate in the P4P scheme (they offered reproductive and child health services and had submitted a one year backlog of HMIS data, enabling performance targets to be measured). All eligible hospitals (n = 6) and health centres (n = 16) from the intervention districts were included in the sample along with all eligible non-public dispensaries (n = 11). An equivalent number of facilities in control areas were sampled by level of care. Public dispensaries were sampled at random with probability proportional to the number of public dispensaries in a given district (n = 42). In control areas, hospitals and health centres were sampled to match as closely as possible with selected intervention facilities in terms of annual outpatient care visits and staffing levels. A total of 75 health facilities were sampled from intervention districts, and 75 were sampled from control districts (Figure 2). In Pwani region, 46% of all facilities in the region were included in the sample.

The aim of the sampling procedure for the selection of health facilities was to seek district representation, while for the health worker survey it was to obtain the views, attitudes, and perceptions of at least one health worker per facility. No sample size calculation was therefore carried out. In dispensaries, one health worker will be interviewed. If more than one health worker is on duty, preference will be given to someone other than the incharge to avoid overburdening them with questions (as they will be interviewed for the facility survey). In health centres and hospitals, two health workers will be interviewed. The health workers will be selected at random from those who are on duty at the facility on the day the interviewers are present.

For the exit and household surveys, the sample size calculation was based on the formula by Hayes and Bennett, 1999, adjusted for the cluster design of the study at the facility level [23]. We estimated the size needed to detect a 17% reduction in waiting time from 114 minutes (SD 66) [24] to 95 minutes, with a k value of 0.25, 80% power and a significance level at of 5% (two tailed test). We did not increase the sample size to account for non-response because response rates of 100% were observed in previous studies in Tanzania [25,26]. The estimated sample size was 10 exit interviews per facility, equivalent to a total of 750 interviews in intervention and control areas respectively. A balance in the number of interviews between antenatal, postnatal clients and non-targeted services will be sought.

Exit interview patients will be approached by interviewers upon entry to the health facility and asked a series of screening questions to check their eligibility. Eligible patients will then be asked for their informed consent to participate in the study. This process will be repeated until the required number of eligible consenting respondents has been attained. Participants will then be monitored by the interviewers from their time of arrival at the facility until their time of departure, and the waiting and consultation times will be measured using a stopwatch. The cadre of the provider seen by the woman/child will also be recorded by the interviewer. The survey tool will be administered to patients upon completion of their consultation in a quiet location within the facility, at distance from providers and other patients.

For the household survey, we estimated that the required sample size to detect an 11 percentage point increase in institutional deliveries (from 50 to 61%), with k value of 0.25, 90% power, and a significance level at of 5% (two tailed test), and a 90% response rate, was 20 households per cluster, equivalent to 1,500 women per study arm. The following process was followed to identify eligible households. First, villages were sampled from the facility catchment area; for all dispensaries, the village where the facility is located will be selected by the research team; for health centres and hospitals, two villages will be selected at random from all villages lying within the ward where the facility is located. Second, all hamlets (comprising approximately 100 households) within this village/these villages, and located within the catchment area of the facility will be dentified; a random sample of four of these hamlets will then be selected. In the case of dispensaries, all four hamlets will reside within the selected village. In the case of health centres and hospitals, two hamlets will be sampled from each village. Third, five households will be sampled from each of the selected hamlets, amounting to a total of 20 households within each facility's catchment area; households will be selected at random from the selected hamlets using a modified Expanded Programme of immunisation (EPI) type sampling scheme that ensures an equal chance of any household being selected.

At the centre of the hamlet the supervisor throws a pen to determine the direction, and counts 10 houses in the direction indicated by the pen. A number is picked at random by writing down ten numbers and picking one at random. The house with the corresponding number is the starting point for data collection. The supervisor introduces the study to the household head, or a representative of the household and asks him/her if there are any eligible women living in the household: a woman aged 16 to 49 who had a baby between Oct 2010 and Oct 2011. If there is an eligible woman, they leave a copy of the consent form with them and ask if it would be convenient to return for interview the next day, at an agreed time. The pen is then thrown again and the next household in the direction of the pen is selected for interview. The supervisor continues going household to household in this way until five eligible households consenting to being interviewed are identified. If there is a junction in the path, the supervisor throws a pen again to determine the direction.