The World Bank Listening to LAC (L2L) Pilot 2011
Peru, 2011 - 2012
Get MicrodataIdentification
PER_2011_L2L_v01_M
The World Bank Listening to LAC (L2L) Pilot 2011
| Name | Country code |
|---|---|
| Peru | PER |
Other Household Survey [hh/oth]
The rapid and massive dissemination of mobile phones in the developing world is creating new opportunities for the discipline of survey research. The World Bank is interested in leveraging mobile phone technology as a means of direct communication with poor households in the developing world in order to gather rapid feedback on the impact of economic crises and other events on the economy of such households.
The World Bank commissioned Gallup to conduct the Listening to LAC (L2L) pilot program, a research project aimed at testing the feasibility of mobile phone technology as a way of data collection for conducting quick turnaround, self-administered, longitudinal surveys among households in Peru and Honduras.
The project used face-to-face interviews as its benchmark, and included Short Message Service (SMS), Interactive Voice Response (IVR) and Computer Assisted Telephone Interviews (CATI) as test methods of data collection.
The pilot was designed in a way that allowed testing the response rates and the quality of data, while also providing information on the cost of collecting data using mobile phones. Researchers also evaluated if providing incentives affected panel attrition rates.
The random stratified multistage sampling technique was used to select a nationally representative sample of 1,500 households. During the initial face-to-face interviews, researchers gathered information on the socio-economic characteristics of households and recruited participants for follow-up research. Questions wording was the same in all modes of data collection.
In Peru, households were randomly assigned to a communication mode (SMS, IVR, CATI), which stayed constant for all rounds (waves) of the survey.
Sample survey data [ssd]
- Households
Version
v01
Scope
The scope of the study includes rigorous testing using experimental design of:
- reliability testing
- criterion validity testing
- attrition rate
- cost of data collection
- incentives to respondents
Coverage
Includes the entire national territory, with the exception of neighborhoods where access of interviewers is extremely difficult, due to lack of transportation infrastructure or for situations that threaten the physical integrity of the interviewers and supervisors (i.e. extremely high crime rate, warfare, etc.)
Producers and sponsors
| Name |
|---|
| World Bank |
| Name | Role |
|---|---|
| Gallup | Survey implementation |
| Name |
|---|
| Spanish Trust Fund for Latin America and the Caribbean (trust fund managed by the World Bank) |
| Rapid Social Response (trust fund managed by the World Bank) |
| Environmentally and Socially Sustainable Development (trust fund managed by the World Bank) |
| The Brightstar Corporation |
Sampling
The Peru panel was built on a nationally representative sample of 1,500 households. The sample was based on the sampling frame for the National Household Survey (ENAHO) conducted by the Peruvian National Statistics Office (INEI) every three months.
In Peru, the sample selection was guided by the following criteria: (i) the sample should be representative nationally, and in urban and rural areas, and (ii) households close to poverty line should be oversampled because policy decisions in time of crises need to be especially mindful of the poor and vulnerable. For the purposes of this project, "close to poverty line" was defined as 40 percent of consumption distribution that symmetrically band the national poverty line: 20 percent above and 20 percent below. In 27 percent of Peruvian households monthly per capita consumption was below the moderate poverty line in 2010 (ENAHO).Those households whose monthly per capita consumption falls between 7 and 47 percent of the national distribution were oversampled.
The L2L sample frame comprises all the panel conglomerados from the fourth trimester of ENAHO 2010, or 281 conglomerados.
Detailed information about the sampling procedure is available in "Listening to LAC: Using Mobile Phones for High Frequency Data Collection, Final Report" (p. 65-69) and "The World Bank Listening to LAC (L2L) Pilot Project Sample Design for Peru."
A number of restive communities in Peru did not allow Gallup's interviewers to enter the area. Where possible, these were replaced following INEI's standard methodology. When confronted with a problem in a particular location, INEI moves to the next "Centro Poblado" in the same "Conglomerado."
In Peru, 67 percent of recruited households failed to answer the first round of follow-up surveys. Attrition slightly increased with each wave of the survey (between 1 and 3 percentage points per wave), reaching 75 percent in wave 6.
As part of the survey administration process Gallup implemented a number of mechanisms to maximize the response rate and panelist retention. The following strategies were applied to respondents who did not replay first time:
- The surveys were left open for responses for up to 2 weeks after the original transmission of the survey (from original call in the case of IVR and CATI).
- First reminder was sent within 72 hours of first attempt (SMS and IVR).
- Second reminder was sent within 144 hours of first attempt (SMS and IVR).
- Call backs were made within 72 and 144 hours of first attempt (CATI); or
- Up to 2 call backs were made per appointment with respondent (CATI).
Also, in order to minimize non-response, three types of incentives were given. First, households that did not own a mobile phone were provided one for free. Approximately 200 phones were donated in Peru. Second, all communications between the interviewers and the households were free to the respondents. Finally, households were randomly assigned to one of three incentive levels: one-third of households received US$1 in free airtime for each questionnaire they answered, one-third received US$5 in free airtime, and one-third received no financial incentive (the control group).
The sample design overrepresented conglomerates where household income was close to the poverty line (20% above or 20% below it) versus all other conglomerates. The oversample needed to be corrected by accounting for the sample selection probability by creating a base weight. The National Household Survey (ENAHO) data on the size of the conglomerates that made up the 2 L2L strata was used to create targets for the size of the L2L strata.
Using the ENAHO data, Gallup created targets for the distribution of the age of the head of household, and the highest level of education of the head of household. The L2L data were trimmed so as to balance bias (how close the demographics of the weighted data align to the targets) and efficiency (the size of the design effect from weighting). Then the data were renormalized so that the sum of the weights equals the number of observations.
Survey instrument
The following survey instruments were used in the project:
- Initial face-to-face questionnaire
In Peru, the starting point was the ENAHO (National Household Survey) questionnaire. Step-wise regressions were done to select the set of questions that best predicted consumption. For the purposes of robustness, the regressions were also done with questions that best predicted income, which yielded the same results.
The survey gathered information on households' demographics, household infrastructure, employment, remittances, income, accidents, food security, self-perceptions on poverty, Internet access and cellphones use.
- Monthly questionnaires (SMS, IVR, CATI)
The questionnaires were worded exactly the same way, regardless of the mode, which meant short questions, since SMS is limited to 160 characters. A maximum of 10 questions had to be chosen for the monthly questionnaire. In addition, two questions sought to ensure the validity of the responses by testing if the respondent was a member of the household. Most questions were time-variant and each questionnaire was repeated to observe if answers changed over time. All questions related to variables that strongly affect household welfare and are likely to change in times of crisis.
A maximum of 10 questions was chosen for the monthly questionnaire. In addition, two questions sought to ensure the validity of the responses by testing if the respondent was a member of the household. To accomplish this, the first two questions in each monthly questionnaire asked the respondent for their gender and year of birth, and the answers were compared to the household roster obtained during the face-to-face interview.
- Final face-to-face questionnaire
Gallup conducted face-to-face closing surveys among 700 panelists. The researchers asked about issues the respondets had with mobile phones and coverage during the test. Panelists were also asked what would motivate them to keep on participating in a project like this in the future.
Data collection
| Start | End |
|---|---|
| 2011-10 | 2012-06 |
| Name |
|---|
| Gallup |
On average, 20 interviewers and 5 supervisors were trained in each country.
Gallup conducted a total of 12 training sessions with all its operational partners prior to launching the project. The training included:
- Survey methodology workshop aimed at identifying and reflecting on the most common sources of survey errors typically encountered in projects like the L2L
- Review of the L2L project’s objectives and methodology
- Panel recruitment and management workshop, including incentive administration and other attrition containment strategies
- Detailed review of (and hands on training on) the L2L face-to-face questionnaire and recruitment protocol
- Live SMS survey self-administration session
- Live IVR survey self-administration session
- Pilot face-to-face survey administration followed by a simulated recruitment for the panel among a group of actual households.
In Peru, households were randomly assigned to a communication mode (SMS, IVR, CATI), which stayed constant for all six waves of the survey.
Through a competitive bidding process, Gallup won the contract to implement L2L on the ground. Working with affiliates in Peru and Honduras, Gallup successfully completed the initial face-to-face interviews, monthly or bi-monthly surveys using mobile phones, and a final face-to-face interview of a sub-group of the initial sample. Gallup completed the face-to-face interviews as well as the IVR and CATI interviews in-house, and subcontracted the SMS interviews to an SMS aggregator.
Despite extensive planning by the World Bank and Gallup teams, no SMS messages went out during the week of April 23 due to an error by the SMS aggregator, which simply forgot to program the system to send out messages that week. However, in this case, the messages went out the next week and the response rates did not suffer as a result.
Data Access
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example:
World Bank. Peru the World Bank Listening to LAC (L2L) Pilot 2011. Ref. PER_2011_L2L_v01_M. Dataset downloaded from [url] on [date].
Disclaimer and copyrights
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Contacts
| Name | Affiliation | |
|---|---|---|
| Amparo Ballivian | World Bank | aballivian@worldbank.org |
| Joao Pedro Azevedo | World Bank | jazevedo@worldbank.org |
Metadata production
DDI_PER_2011_L2L_v01_M_WB
| Name | Affiliation | Role |
|---|---|---|
| Development Data Group | World Bank | Survey documentation |
2014-06-19
Metadata version
v01 (June 2014)