Parental Nudges Project Impact Evaluation 2020-2021

All Rounds

Baseline:
The sample is drawn from two previously completed studies. First, an impact evaluation of the Communications for Development (C4D) study (2012-2016), launched by the Ghana Health Service with funding from UNICEF in 12 districts of the three poorest regions of Ghana. The sample included mothers with a child aged 0-5 years recruited in 2012. The C4D program relied on voice messages directly delivered to female respondents through their cell phones. The C4D sample has high rates of mobile phone ownership (83%). Second, even in a very negative scenario in which 52% of households from the C4D sample have changed phone numbers, we relied on a subsample of the Graduating the Ultra Poor (GUP) study from the same regions to obtain our desired sample size.

The samples from the C4D and GUP projects formed the sampling frame, from which we determined the eligibility of households and then sampled from those eligible for the PNP. An initial desktop screening was conducted to screen out households without phone numbers. Households with phone numbers were then contacted through a subject enrollment call to determine their eligibility, seek their consent to participate in the study and their consent to receive the text messages.

The 4500 eligible households in the two datasets were screened to determine their eligibility for the study, with the goal of selecting 2,500 households / primary caregivers (and 2 children per household) into either one of the experimental groups.

First, eligible 4500 households were identified in the dataset and randomization was conducted before households were recruited.

Second, each of the 2500 eligible households was called to confirm eligibility and enroll them in the study. The eligibility criteria were as follows:

1. A household with at least one member owning a working mobile phone.
2. A household with an adult aged 18 years and above. This age criterion satisfies the age requirement as provided in the 2014 The Unsolicited Electronic Communications Code of Conduct by the National Communications Authority of Ghana. It also forms the basis for recruiting the primary caregiver, defined as the person who is primarily responsible for the care and education of the school-aged children in the household and could best talk about their experiences in school and at home.
3. A household that is willing to participate in the study and/or to receive text messages.
4. A household with more than one member including at least 1 school-aged child.

Finally, 2 school-aged children were randomly sampled from each eligible household (5,000 total). Additionally, using a reserve list of replacement households, we replaced 128 households that had dropped out of the study after the Subject Enrollment and Caregiver Survey.

Midline and endline:
The sampling frame for the PNP consisted of two previously completed IPA studies. First, an impact evaluation of the Communications for Development (C4D) study (2012-2016), launched by the Ghana Health Service with funding from UNICEF in 12 districts of the three poorest regions of Ghana. The sample included mothers with a child aged 0-5 years recruited in 2012. The C4D program relied on voice messages directly delivered to female respondents through their cell phones. The C4D sample has high rates of mobile phone ownership (83%). Second, even in a very negative scenario in which 52% of households from the C4D sample have changed phone numbers, we relied on a subsample of the Graduating the Ultra Poor (GUP) study from the same regions to obtain our desired sample size.

A target sample of 2,500 primary caregivers/households was set. We assumed an average of 2 eligible school-going children (aged 5 to 17 years) per household and thus, planned to recruit approximately 5,000 school-going children (2 per household) from the 2,500 households. Our goal was to randomly sample one = younger school-aged child (5-9 years) and one older school-aged child (10-17 years). Taking into account an estimated 10% non-response rate and pilot sample, we targeted a sample of 2,750 households or primary caregivers. We followed the following steps in order to select the primary caregivers and children for the PNP. The steps involved screening study participants, assigning them to the intervention and comparison groups, and enrolling in the study.

Step 1. Desktop screening and random assignment of mobile phone numbers or households. We conducted desktop screening using the existing samples from the C4D and GUP to determine whether at least one household member within each household had a mobile phone number. Households without at least one household member with a mobile phone number were screened out. The desk-top screening resulted in 4,500 households with mobile phone numbers. Next, we randomly assigned the 4,500 households to the intervention and comparison groups. The random assignment resulted in an almost equal distribution of households across the intervention and comparison groups.

Step 2. Screening and enrollment of eligible households or primary caregivers. We implemented a phone-based subject enrollment call to (a) determine the eligibility of the households or primary caregivers for the study, (b) seek their consent and that of their children to be enlisted into the study, (c) inform them about the project, the intervention, and implementation modalities, (d) seek their consent to participate in the intervention, and (e) enroll 2,668 eligible primary caregivers into the study. The additional 168 households were selected to (a) serve as reserve households for replacement (n = 128) and (b) be used for the intervention piloting (n = 40).

We enrolled households or primary caregivers based on the following eligibility criteria:
a. A household with an adult aged 18 years and above. This age criterion satisfies the age requirement as provided in 2014 The Unsolicited Electronic Communications Code of Conduct by the National Communications Authority of Ghana. It also forms the basis for recruiting the primary caregiver, defined as the person who is primarily responsible for the care and education of the school-aged children in the household and could best talk about their experiences in school and at home.
b. A household that is willing to participate in the study and/or to receive text messages.
c. A household with more than one member including at least 1 school-aged child. A child roster was created for school-going children aged 5 to 17 years within the household.

Step 3. Selection of school-going children. Once the primary caregiver had been identified, the SurveyCTO in-built randomization protocol was triggered to randomly select at most 2 school-going children [and 2 replacements] from each household. School-going children who meet the eligibility criteria, namely, (a) aged 6-17 years, (b) attending school, and (c) meeting the household residency requirement were randomly selected for the study.

The SurveyCTO in-build randomization code was designed to:
a. Generate two age groups or strata of eligible school-aged children based on their age: (a) Stratum A comprised school-going children aged 6 to 9 years, (b) Stratum B consisted of school-going children aged 10 to 17 years. If there was no child aged 5-17, the selection was based on children aged 5, if available.
b. Randomly sample 3 eligible school-going children from each stratum and rank them from 1 to 3. The child with rank 1 in either stratum became the selected child. Those with ranks 2 and 3 in either stratum became the replacement children. If Stratum A (Stratum B) has no eligible child, the children in Stratum B (Stratum A) with ranks 1 and 2 became the selected children. Rank 3 was the replacement child. If either stratum had less than 3 eligible children, the child assigned rank 2 became the only replacement child for that stratum. If either stratum had only 1 child each, there was no replacement child.

We identified and sampled 4,685 school-aged children out of the 5,000 targeted for the study. This was largely due to some households not having two children in the target-age. On average, we reported 1 eligible school-going child per household.