ETH_2004_WSS_v01_M
Water and Sanitation Household Survey at Project Level 2004 - Oromia Region
Name | Country code |
---|---|
Ethiopia | ETH |
Other Household Survey [hh/oth]
The Water and Sanitation Sector of the World Bank is facing increasing demands for access and performance data. This is driven in part by a corporate focus on the Millennium Development Goals and the World Bank’s contribution to their achievement, and in part by a general need to better monitor the impact of sector interventions. The aim of this report is to propose a range of dedicated project level WSS indicators at household level and a survey methodology on how to obtain these indicators.
The goal of the household survey presented in this report is to obtain a better understanding of households' access to water, sanitation, and hygiene across a project area, and the safety and quality of the services provided from a household
perspective. The methodology has been developed and tested simultaneously on urban households in Ethiopia and rural households in Peru.
The survey instrument has been developed taking the work of the Joint Monitoring Programme (JMP) as a starting point. Hence the questions on water and sanitation in the JMP-questionnaire form part of survey instrument presented in this report. However, whereas the JMP approach aims at few comparable indicators which can be measured as part of a general national household survey, the aim of the methodology described here is to obtain more detailed information on water, sanitation and hygiene in a geographically limited project area.
For the sample selection, a multiple stage cluster sampling methodology was proposed. When this procedure is followed carefully, a statistically representative sample is obtained which is crucial for the applicability of the results. The approach
enables data collection with a balance between the precision of the estimates and practical implementation costs.
A subset of 73 towns in the Oromia region was selected as the target of the survey. These towns were selected for projects to improve the water and sanitation services and are generally characterised by a low level of services. A two-stage cluster sampling procedure was applied to select the sample. From a total of 73 towns, 24 towns were selected randomly with a probability proportionate to size (PPS). In the second stage of the sample selection, enumeration maps were used in order to obtain a sample of households that is spread across each town. A total of 1,013 households were interviewed.
The practical steps of survey administration included preparatory activities, testing, and data collection. Here we have placed emphasis on the training of the enumerators to ensure that they understand the task they are undertaking, and on quality control to ensure that the interviews are conducted in a professional manner which yields more accurate results.
A set of 28 indicators is proposed from which the most locally relevant may be selected as the basis on which the monitoring should be carried out. The results of the survey are reported by presenting the 28 selected indicators, their mean values, and the confidence interval for the mean value. The survey indicates that 46% (±7%) of the households have access to improved water services and 38% (±5%) of the households have access to an improved sanitation facility, according to the definitions adopted in this report. More than four out of every five households have soap, while few households (5%, ±3%) are observed to have adopted appropriate hand washing practices, (as gauged by the absence of one or more of the necessary preconditions for hand washing: soap, water and sink, bucket or equivalent.) At the same time 30% (±9%) of the respondents claim to wash their hands at critical times. This means that there is a discrepancy in the results depending on the method used to obtain the data; this may be due to biases in both indicators; the enumerators
may not be probing sufficiently to see the hand washing practice of household members, and the respondents may respond with “expected” or “desirable” behaviour rather than actual behaviour when asked to state when they wash their hands. It is proposed to use both observation and questions, although most emphasis should be put on the results from the questions when interpreting the results.
Ten percent of the sample was back-checked after the end of the main data collection phase by having a separate group of enumerators visiting the interviewed households. The back-checking results include information about the two central indicators; access to improved drinking water and access to improved sanitation; and household size. The back-checking reveals a discrepancy between the results from the main data collection phase and the back-checking. This underlines that the backchecking should be used actively during the data collection in order to ensure early correction of quality flaws.
Sample survey data [ssd]
Households
Dataset and documentation provided by COWI to Pete Kolsky.
The 2004 Water Supply, Sanitation and Hygiene Household Survey covered the following topics:
The survey was carried out among households in urban areas in the Oromia region. There are approximately 550 towns in this region, and 73 towns had been chosen as candidates for possible water and sanitation projects. A number of selection criteria provided the background for this selection and, generally, the selected towns have a low level of water and sanitation services while there is some potential for cost recovery. The towns vary in size between 2,000 and 45,000 inhabitants. Thus the population from which the sample was taken is the 73 towns and not the entire population of towns in Oromia. A sample of households from 24 towns was selected to represent this universe of 73 towns.
Name |
---|
COWI |
Name |
---|
World Bank |
Through discussion with the regional planning office in Oromia and the World Bank it was decided to carry out the survey among households in urban areas in the Oromia region. There are approximately 550 towns in this region, and 73 towns had been chosen as candidates for possible water and sanitation projects. A number of selection criteria provided the background for this selection and, generally, the selected towns have a low level of water and sanitation services while there is some potential for cost recovery. The towns vary in size between 2,000 and 45,000 inhabitants.
A subset of 73 towns in the Oromia region was selected as the target of the survey. Households were selected in 23 of these 73 towns. A total of 1,013 households were interviewed. The population from which the sample was taken is thus the 73 towns and not the entire population of towns in Oromia.
The two-stage cluster sampling procedure is proposed for household surveys on water and sanitation. The term cluster refers to a natural grouping within the population, such as a neighbourhood, town, district or other community, from which a sub-sample may be selected.
This procedure is based on the selection of a certain number of these clusters, i.e. primary sampling units (PSUs) with a probability proportional to size. 73 towns were selected in the first stage as PSUs. The selection of towns with a probability proportional to size (PPS) is carried out by creating a cumulative list of town populations and selecting a systematic sample from a random start. Within each of these PSUs a fixed number of households or basic sampling units (BSUs) is selected. Each BSU in the population had an equal probability of being in the sample.
Such a sampling procedure is said to be self-weighting and leads to simplified formulas for analysis.
A sample frame was defined for PSUs as well as BSUs.
First stage - selection of towns (PSUs)
Information about the population in the 73 towns in 2004 was applied as the basis for a sample selection based on probability proportionate to size (PPS). A total of 25 clusters were selected resulting in 24 towns, because two clusters were chosen from one large town (Waliso). The number of primary sampling units was decided on the basis of the available resources and the marginal gain in precision of increasing the number of PSUs compared to the additional costs of increasing the number of clusters. With the assumption of a rate of homogeneity of 0.10 for the measured variable, it was decided that the survey should consist of 25 towns with 40 interviewed households in each town. For a variable with an expected proportion - or frequency - of 40% in the population, this means a standard error of 4.3% and a 95% confidence interval of 33% - 47%. If this interval is too wide to be acceptable vis-à-vis the object of the survey, the number of towns should be increased in order to reduce the standard deviation of the estimator.
The 25 primary sampling units were drawn with a probability proportionate to the size of the towns from the population of 73 towns. In practice, the PPS procedure is carried out by creating a cumulative list of town populations and then selecting a systematic sample from a random start. In order to obtain a self-weighted sample, a constant number of households is selected within each of the 25 PSUs. Using this method, one town can be selected twice or more and this is the reason for distinguishing between towns and clusters. In the town that was chosen twice, Waliso, 80 households were interviewed. The following 24 towns were represented in the sample:
ABOMSA, AMBO, BEKE, BOKOJI, BUBE, GEDO, GIMBI, GOBA, GUDER, HAGERE MARIAM, HIRNA, MEGADO, MEKI, MELKA, RAFU, MENNA, METTU, MOJO, MOYALE, NEGELE BORENA, NEJO, SUPHEE, WALENCHITI, WALISO, YABELO
Second stage - selection of households (BSUs)
For the selection of households in each town, enumeration maps from the last population census were used. This was to ensure that the households selected were spread all over the town. Enumeration maps are particularly helpful when selecting a sample for a water and sanitation survey (and other infrastructure services) because households that are geographically close are likely to have the same level of service. Hence, the application of enumeration maps gives a sample, which is implicitly stratified and this lowers the variance of the estimates.
Enumeration maps were obtained for the 20 medium-sized towns and, hence, not for the smallest four towns, as enumeration maps for very small towns are large and unwieldy. For example, in a town with 30,000 inhabitants there are around 30 enumeration maps. A random selection procedure was used to select 6 maps out of the 30.
The enumeration maps vary in size according to the population density and will typically be larger in the outer areas than in the more densely populated centres of the towns. The selection of the 5-7 households to interview within each enumeration map area is carried out using a method of random selection. Rather than instructing the enumerators to use a specific rule to obtain a randomly selected sample, the concept of randomness was carefully explained along with different ways of making a random selection (flipping a coin, spinning a bottle, taking every third household). Whether the enumerator interviews households spread out on the enumeration map is easily checked, as interviewed households are marked on each map by the enumerator.
In small towns with less than 6,000 inhabitants it was decided that the size of the enumeration maps become too large in order to work in practice, as the enumerators would have difficulties operating maps that measures 1x1 meter. As an alternative strategy of selecting households, the enumerators identified four points in the outer area of these towns where each enumerator begins his or her rounds of interview. From these starting points each enumerator made his or her way towards the centre of the town while randomly selecting households. As above, the random selection of households could be done in several ways. Each enumerator was free to choose his or her strategy for random selection, choosing between flipping a coin, spinning a bottle etc. The approach taken here was based on the idea to give the enumerators an understanding of the importance and meaning of random selection rather than giving an explicit rule.
The sample is self-weighted
The questionnaire proposed here for household surveys on water supply, sanitation and hygiene at project level takes its starting point in this core set of water, sanitation and hygiene questions as reflected in the UNICEF/WHO Joint Monitoring Program (JMP)'s "Guide for Water Supply, Sanitation and Hygiene Related Survey Questions" and "Questions for Household Survey Module - Water Supply, Sanitation and Hand Washing", June 2004.
The JMP recommended questionnaire was adapted to obtain more detailed information on water, sanitation and hygiene in a geographically limited project area. In response to the specific requirements of the instruments, lessons learnt from more detailed and specific questionnaires have been taken into consideration, in particular those related to hygiene behaviour. Discussions with the World Bank both in Washington and in Addis Ababa, and the following documents, were the major sources of inspiration:
• The London School of Hygiene and Tropical Medicine's WASH survey method developed under the Water, Sanitation and Hygiene campaign of the Water Supply and Sanitation Collaborative Council (WSSCC), in particular Kristof Bostoen's water, sanitation and hygiene study in LAO PDR, funded by the World Bank.
• The Environmental Health Project's (USAID) report on Assessing Hygiene Improvement - Guidelines for Household and Community Levels.
• Boot, M.T. & Sandy Cairncross: Actions Speak - The study of hygiene behaviour in water and sanitation projects.
The questionnaire was tested by conducting qualitative interviews. In addition, a back-check questionnaire was developed for the quality control.
A set of picture cards were used to support the enumerators in understanding and asking questions on the different water supply and sanitation facilities. The show cards were based on those used in the World Health Surveys (WHO), supplemented by materials obtained from the Loughborough University Image Library and other sources. The quality of the illustrations can, to some extent, be improved in order to more clearly present the characteristics of a given technology and to take into consideration the variations in the design of the different technology types across countries and continents. Such improved illustrations are currently being procured by the WHO.
SUBJECTS INCLUDED IN THE QUESTIONNAIRE
In accordance with the JMP questionnaire, reasonable access to water supply is assessed through questions on the following topics:
• main source of drinking water;
• main source and alternative source for other purposes;
• time to collect water;
• quantity of water used;
• individuals who collect water.
The term reasonable access has been adapted from the JMP in recognition of the need to further qualify access because access is a relative term that only exists as more or less/to a higher or lower degree, depending on the socioeconomic
and cultural context. The availability of an improved technology is not identical with accessibility, which depends on other factors such as the quality of the service, the location of the service, the reliability of the service etc. It has therefore been necessary to include generally acceptable standards or reasonable standards such as minimum quantity per person per day, minimum time spent on fetching water etc. Reasonable access measures the degree to which these standards are fulfilled and gives a more complete picture of access to water supply.
The main source type of drinking water provides an approximation for reasonable access understood as the availability of safe water of at least 20 litres per person per day from a source that can be reached within 15 minutes from the respondents' dwelling (approximately equivalent to one kilometre).
It is assumed by the JMP and others, for the purposes of making the surveys manageable, that certain types of technologies give an adequate supply of water, and these are accordingly defined as the improved water supply technologies.
Improved water sources are protected water sources, non-improved water sources are unprotected water sources. A protected water source is constructed in a manner that prevents water from being contaminated, particularly from surface
runoff (from rain, snow melt, or irrigation water). The following technologies are considered as improved:
• piped water into dwelling;
• piped water to yard/plot;
• public tap/standpipe;
• tubewell/borehole;
• protected dug well;
• protected spring;
• rainwater collection in closed containers.
Non-improved water supply facilities are:
• unprotected dug well;
• unprotected spring;
• rainwater collection in open containers;
• small-scale vendor;
• tanker-truck;
• surface water.
According to the JMP definitions, bottled water is considered non-improved because of concerns about the supplied quantity, not because of concerns about the quality. Here bottled water is treated differently. If the main source of drinking water is bottled water, the alternative source is used to define if the water source is improved. For instance in this report, if the alternative source is piped water into dwelling, the water supply is considered to be improved.
Water treatment is also included in the survey to discern whether households are treating the water before drinking.
As households frequently use alternative sources for other purposes than drinking, additional information is collected on the main and alternative water source used for bathing, washing, cleaning, etc. Questions on water transport time are included as an important determinant of access to water supply. Finally, a question on who fetches the water is included in order to obtain information about the burden from this work on women and children.
Compared to the JMP questionnaire, the proposed questionnaire has been supplemented with questions on the following topics to obtain a more complete picture:
• Handling of water during storage and for consumption: This is included as safe water from the source could be contaminated during storage and at the time of consumption if not handled properly;
• Seasonality is included because continuous access to improved water only applies if the household has access all year round.
• Reliability of water supply: Questions are included on the occurrence and duration of breakdowns. If the duration of breakdowns is excessive, i.e. more than one month in total during the last half year access is limited.
• Daily availability: Questions are included on the operation time of water supply facilities.
• Attitudes towards water services quality: This is to assess "reasonable access" in terms of the degree of the households' satisfaction with their water services such as water quality, distance, and stability of the services, etc.
• Expenditures on water, water treatment and water storage Experience gained from the trial of the WASH survey for water, sanitation and hygiene practices referred to in section 2.1 suggests that enumerators have difficulties in understanding and explaining the different water supply and sanitation technologies. Therefore visual aids (picture cards with drawings of the different technologies) are used for training purposes and to support the enumerators during field work. The visual aids are presented in Appendix 41 of the report.
Questions are combined with direct observations, as observations often achieve more accurate results, especially in the case of delicate issues such as cleanliness, hygiene, etc. "Water handling" is also assessed through observation, i.e. the condition and design of the storage container and water drawing devices.
In line with the JMP core questionnaire, "access to an adequate sanitation facility" is assessed through two questions:
• available sanitation technology;
• shared/non-shared facilities.
Knowing the type of sanitation facility used by a household provides information on the degree to which the facility meets basic sanitation requirements, i.e. if it hygienically separates human excreta from human contact. Meeting basic sanitation requirements is possible through the following facilities:
• flush to piped sewer system;
• flush to septic tank;
• flush/pour flush to pit;
• composting toilet;
• VIP/pit latrine with slab.
Facilities that do not meet basic sanitation requirements include:
• flush/pour flush to elsewhere;
• pit latrine without slab/open pit;
• bucket toilet;
• hanging toilet;
• open air/bush/field etc.
Whether a sanitation facility is shared with other households or is a public facility can impact its hygienic quality and in turn have an impact on its use. Shared facilities that are poorly maintained may be difficult for household members to use because of filth, odour and flies. Shared facilities - either private or publicare not considered adequate.
The JMP questions focus only on the type of available facilities. In the proposed questionnaire, additional indicators are included that measure the actual use of the facilities, cf. Appendix 1 of the report. Actual use is assessed by the presence of factors that are likely to encourage or discourage their use. The following are included:
• time to reach the facility;
• free access, signs of regular use;
• hygienic condition of facility.
If it takes a long time to reach the facility - if the facility is some distance away from the place of residence - this may discourage its use, e.g. by children or during the night. Whether a facility is actually used or not can be assessed by looking at certain conditions such as free passage (path is cleared, entrance not blocked, etc) and other signs of use. The hygienic condition of a facility is an indication of its use - unhygienic conditions are more likely to discourage its use. At the same time, the condition of the facility is an indication of hygiene practices.
Picture cards are used to assist the enumerators to identify/validate the type of sanitation available to a household. Also, questions are combined with observations.
The time it takes to reach the facility, the accessibility (indication of actual use) of the facility and its hygiene condition are assessed through observation.
The costs of building the sanitation facility and the costs of maintenance and repair are also assessed, because costs are important aspects.
Grey water consists of domestic wastewater exclusive of toilet waste. The rationale for including grey water is that from the point of view of "water hygiene" wastewater disposal is an integral part of the "water handling cycle" comprising drawing of water, storage, handling for consumption, reuse, and wastewater disposal.
Water used for cleaning clothes and nappies can be heavily contaminated with the same disease-causing organisms that hygiene and sanitation are intended to control. Stagnant water resulting from insufficiently drained grey water could also become breeding grounds for mosquitoes. Grey water therefore needs to be disposed of properly. The following observations are included to assess grey water disposal:
• mode of discharge of the household's wastewater;
• hygiene condition of the point of discharge.
Three questions are used to assess hygiene practices. They are concerned with:
• access to hand washing supplies;
• use of hand washing supplies at critical times;
• disposal of child faeces.
Hand washing is one of the most important preventive measures for reducing the prevalence of diarrhoeal disease. Whether a household has access to hand washing supplies such as soap can provide an indication of whether appropriate hand washing practices within the household are possible or likely. Along with appropriate hand washing supplies, hand washing at critical times is an important element of appropriate hand washing practices.
Knowing how children's faeces are disposed of is a critical aspect of hygiene improvement. Particular focus is placed on children's faeces because their faeces are more likely to be a source of contamination in the household environment than the faeces of other household members - and because in many societies they are considered harmless and therefore are not disposed of properly.
The JMP questions focus on the availability of soap alone, but in the proposed questionnaire soap alone is not considered a sufficient indication of appropriate hand washing practices, as soap may be used for other purposes. In many places/countries, no distinction is made between soap for hand washing and soap for other uses such as washing dishes, washing clothes, cleaning, etc.
It appears more useful to look at the entire "hand washing system". This is also supported by the WASH and EHP surveys, which find that the observed (concurrent) presence of items needed for hand washing, i.e. soap, water and basin, works best as an indicator for "good hygiene practice". For this reason, an observation is included of people's hand washing practices (they are asked to demonstrate how they usually wash their hands) to determine if the three prerequisites are present.
A fourth indicator of hand washing - the presence of a clean towel or cloth2 - is not used, as assessing cleanliness through the enumerators' judgement is very subjective while at the same time towels or cloths that are not clean constitute a health hazard.
In addition to the core questions on water, sanitation, drainage and hygiene a number of standard questions on household characteristics are included to establish the households' socio-economic situation. Although this is not the main objective of the present study, this may allow for identification of the relationship between poverty/well-being levels and water, sanitation and hygiene conditions.
Start | End | Cycle |
---|---|---|
2004-09-13 | 2004-09-14 | Test questionnaire, qualitative interviews |
2004-09-15 | 2004-09-15 | Pilot interviews |
2004-09-18 | 2004-09-30 | Main phase interviews |
2004-10-04 | 2004-10-13 | Data entry |
2004-10-20 | 2004-10-25 | Back-checking interviews |
The enumerators were supervised from a distance during the main phase. Each team leader was instructed to call the coordinator every day to discuss progresses and to report any difficulties encountered.
The survey methodology was tested thoroughly before the main data collection. This was done both by conducting qualitative interviews with households in order to test the relevance and wording of the questions, and by conducting pilot interviews where the instrument and the sample selection procedure were tested.
The implementation followed the time schedule below:
• Test of questionnaire, qualitative interview: 13-14 September 2004
• Pilot interviews: 15 September 2004
• Main phase interviews: 18-31 September 2004
• Coding answers into database 4-13 October 2004
• Back-checking interviews 20-25 October 2004
The time schedule for the implementation, test of questionnaire, and pilot interviews was tight. The coding of the answers was expected to take one week, but took instead one and a half weeks. The back-checking was initiated after the data file was received by the consultant.
Qualitative work
Prior to the testing of the questionnaire and the household selection procedure by the enumerators in the field, a qualitative assessment and test was conducted by the consultant's team with the aim to:
• obtain a first-hand impression of the present levels of services and hygiene conditions;
• validate the questionnaire in terms of relevance, clarity, and the cultural sensitivity of the questions;
• check the consistency of the questionnaire.
A first hand-impression of the existing services, hygiene conditions and other relevant background information was obtained from the local authorities (Wereda, Kebele, Water Office) during courtesy visits. This background information was useful in assessing the relevance and effectiveness of the questionnaire. A uniform check list was prepared specifying relevant water supply, sanitation and hygiene topics.
The qualitative assessment of the questionnaire was conducted in three towns. The towns were selected in such a way that that small towns as well as medium towns were included. Care was taken to ensure that different technical options were represented. Due to time constraints, highland towns close to Addis Ababa were chosen. Therefore, technologies, socio-economic, and sociocultural conditions that may be prevalent in other parts of Oromia region, in particular lowland towns, were not represented. Preferably, the locations for the qualitative assessment tests should be selected carefully so as to capture the full range of diversity.
The assessment approach was a combination of household in-depth interviews and group discussions conducted in a number of households that were selected to cover a variety of water supply and sanitation technologies and different socio-economic characteristics. The English version of the questionnaire was used and the interpreter translated all questions into the appropriate language. Questions that were not easily answered were discussed in detail with the interviewee and other members of the family, neighbours who had dropped in etc, to identify the causes of misunderstandings (lack of clarity, imprecise terms, etc) and to find improved or alternative formulations.
The interpreter also provided feed-back on the usefulness and clarity of the questions, the observation parts of the questionnaire, and the enumerator instructions.
The questionnaire was subsequently revised and adjusted in accordance with the findings of the qualitative assessment and translated into the local language.
Training of enumerators
The training of the enumerators lasted 4 days. Twelve enumerators were employed for the interviews and they were all graduate students. They were divided into three teams and a team leader was appointed for each team. In addition, each team was provided with a car with driver. The training sessions were as follows:
Composition of data collection team
The implementation team consisted of:
• one coordinator/supervisor
• twelve enumerators, divided into three teams, each with a team leader. The number of interviews one enumerator can conduct per day vary depending on the geography of the area. In the towns in Oromia one interviewer was able to conduct 8-10 interviews per day. Between 100 and 200 interviews per enumerator is thus appropriate.
• four back-checkers
• one drivers with vehicle per team
The answers were typed in using an SPSS form pre-coded for the questionnaire. Such SPSS format was used in order to avoid misunderstandings and errors in the coding. Both the results from the main phase questionnaire and the back-check questionnaire were coded into the database using the SPSS form.
In the final report, the analysis of the data was carried out by estimating the means of the indicators measured and the precision of these variables in terms of standard errors and the associated 95% confidence intervals.
Parameters used to compute means and variance
In Ethiopia, a sample consisting of 25 towns are selected from a population of 73 towns. In addition, in each town a sample of 40 households are selected. The total population in the 73 towns is N, while the population in the individual town is Nk, k = 1,..,M. Hence, the following parameters:
• m = 25 towns;
• M = 73 towns;
• nk = 40 interviewed households per town;
• N = Total population of the 73 towns: 797,065;
• Nk = Individual town population.
Design effect and rate of homogeneity
The design effect and the rate of homogeneity have been calculated for each of the 28 indicators (see table below). Both vary depending on the question forming the basis of the calculation. For instance for an indicator where the difference across towns is limited, the design effect and the rate of homogeneity will both be low. This table has been included because it may be useful for future surveys. The design effect should be used for the questions which are most important regarding the purpose of the survey.
Design effect
The design effect is conventionally set at two when calculating the sample size. The design effect and rate of homogeneity for Ethiopia were however calculated on the basis of the data collected in this project. Based on these data, the design effect is estimated to be 3 for the question on access to improved water services and 4 for access to improved sanitation.
Note: other studies have found even higher design effects including Kristof Bostoen et al. where the design effect for access to water and sanitation after stratification is estimated to be around 7 and around 13 when no stratification is used (i.e urban and rural households were included in the same sample).
Such high design effects mean that a cluster sampling procedure in some cases needs significantly more observations than simple random sampling in order to achieve the same level of precision - or equivalently, for a constant sample size, that the number of clusters needs to be increased.
Rate of homogeneity
In the survey in Ethiopia, the rate of homogeneity was estimated to be 0.08 for access to improved water supply and 0.13 for access to improved sanitation facilities, indicating a lower variance between the towns than seen in other studies.
The rate of homogeneity is a measure of the variability between clusters as compared to the variation within clusters. In a two-stage cluster sample it is equivalent to the "intra-cluster correlation". In a more complex design such as a stratified multistage survey, it is composed of the components of variability from all stages of the design. Its value will be higher for variables whose values vary more between clusters than within them. For example, because families in the same area tend to have broadly similar water and sanitation standard, variables such as access to improved water supply and access to improved sanitation facilities will be more likely to produce the same response for two individuals in the same cluster than for individuals in separate clusters. These variables are often assumed to have a relatively high value of the rate of homogeneity, around 0.20.
It should be noted that the rate of homogeneity depends on the variable measured and should be reported for each main indicator. For instance, in the case of urban areas in Ethiopia, the rate of homogeneity is higher for water supply than for access to sanitation meaning that the difference in water supply between towns is less than the difference in sanitation facilities.
Number of clusters (PSUs)
The design effect, the rate of homogeneity and the number of clusters in a sample are closely connected and have significant influence on the precision of the sample, i.e. the standard errors of the measured variables in the sample.
In Ethiopia, the number of primary sampling units was decided on the basis of the available resources and the marginal gain in precision of increasing the number of clusters compared to the additional costs of increasing the number of clusters.
As the sample is to be decided before the estimates of design effect and rate of homogeneity is available, the rate of homogeneity in Bennet et al. of 0.20 was assumed. It was decided that the survey should consist of 25 towns (clusters) with 40 interviewed households in each town. This was decided as a trade of between transportation cost and marginal precision of the estimates. For a variable with an expected proportion - or frequency - of 40% in the population, this means a standard error of 4.6% and a 95% confidence interval between 31.0 - 49.0%. As it was seen in Table 3.2, the estimated rate of homogeneity in Ethiopia turned out to be lower than expected, with means that the precision of the indicators is ±7% rather than ±9%.
Number of observations
The number of observations needed to obtain a certain level of precision was estimated based on prior estimates of the rate of homogeneity obtained from other studies in similar areas (in order to be able to estimate the expected level of precision and hence the necessary sample size and number of cluster).
If a certain precision of the estimate households with access to improved water supply is needed, emphasis should be on the number of clusters rather than on the number of interviews. For instance if the results are needed with a precision of ±5%, this can be obtained with 1,000 interviews distributed on 60 clusters. Such precision cannot be obtained with 25 clusters even with 4,000 interviews.
Design effect and rate of homogeneity by indicator
No / Indicator / Design effect / Rate of homogeneity
I1 / Continuous access to improved water sources as main drinking water source / 4.25 / 0.09
I2 / Access to improved water supply as main drinking source at least part of year / 5.49 / 0.12
I3 / Time to collect water is 30 minutes or less / 4.88 / 0.12
I4 / Daily availability / 2.46 / 0.04
I5 / Continued supply of main drinking water source/ limited irreg. interr. during last six months / 4.41 / 0.09
I6 / Availability of 20 l/c/d / 3.08 / 0.11
I7 / Reasonable access / 5.29 / 0.15
I8 / Appropriate and hygienic drinking water storage facilities / 2.67 / 0.13
I9 / Satisfaction with the quality of their main drinking water services / 8.95 / 0.20
I10 / Water treatment within the last week / 0.53 / -0.01
I11 / Average initial costs of water services / 2.01 / 0.07
I12 / Average monthly costs of water services / 6.96 / 0.35
I13 / Average cost of storage facility / 1.21 / 0.03
I14 / Average costs of water treatment / 0.87 / -0.07
I15 / Access to improved sanitation facilities / 2.91 / 0.05
I16 / Access and actual use of improved sanitation facility / 4.03 / 0.08
I17 / Sanitation facility within less than 5 min. from household / 1.84 / 0.03
I18 / Improved sanitation facility in good hygienic condition / 1.55 / 0.02
I19 / Satisfaction with the quality of the sanitation facility / 4.84 / 0.12
I20 / Average cost for building facility / 1.82 / 0.05
I21 / Average annual costs for maintenance and repair / 1.74 / 0.12
I22 / Access to improved grey water disposal methods / 3.72 / 0.07
I23 / Availability of soap in the household / 3.92 / 0.08
I24 / Adoption of appropriate hand washing practices / 3.51 / 0.08
I25 / Use of soap at critical times / 8.48 / 0.23
I26 / Awareness of children's hand washing practices / 1.39 / 0.01
I27 / Properly disposal of children's faeces / 1.84 / 0.06
I28 / Clean surroundings around residence / 6.09 / 0.14
Note: Some of the estimated values of the rate of homogeneity are less the zero (I10, I14). This is due to low values of the indicator and sampling variation. The rate of returns less than zero should be interpreted as 0, i.e. that there is not more correlation between the households living in the same town than household living in different towns. Water treatment is an example of this; very few household treat the water and almost no households are able to give an estimate of the amount spent on the water treatment. On the other hand, the largest variation across towns is found for reasonable access (I7) which means that there is a correlation between household with reasonable access and the town that they live in.
A back-checking procedure was initiated to ensure the quality of results and to check that the questionnaires were genuine and not filled in by the enumerators at home.
A team of three back-checkers was employed for the work and they were trained by explaining the purpose of the research, the importance of their job as well as the questions in the back-check questionnaire. It was emphasised that the difficult part of the back-checking was to relocate the households interviewed in the main phase.
Back-check sample
A sample of 10% of the households was back-checked and these households were chosen so that interviews conducted by all twelve enumerators could be checked. This was ensured by selecting a limited number of towns and backchecking a large fraction of households interviews in these towns.The towns were selected by the consultant after the termination of the main interview phase: Gedo, Meki, Agere Marayam and Yabelo.
The back-check questionnaire was in Oromo and in order to ease the job of finding the households, which participated in the survey, the back-checkers received the relevant enumeration maps and addresses of the households (though the addresses contain very limited information). Ideally the back-checking is conducted during the main phase of the survey, because this may allow early correction of quality flaws. However, in Ethiopia this was not possible, because the data entry was done in Addis Ababa only after the return of the enumerators.
Back check results
The quality assurance is carried out by comparing the result of two identical questions. These are the key questions about main source of drinking water and toilet facility/toilet practice of the household. In addition, a question about the number of household was included in the back-check questionnaire. The back-checking procedure is in this project used for two purposes:
• to check if the interviews were actually carried out
• to check the reliability of the results technology
Results:
Check that the interviews were actually carried out
The results were not as good as expected but not sufficiently to conclude that the interviewers did not fill out questionnaires themselves without interviewing the households.
Check the reliability of the results
Main drinking water source
The back-checking of the main drinking water source indicates to what degree the result depends on the enumerator collecting the data. Comparing the answers for main drinking water source for the main phase and for the backcheck, shows that 57% indicates that they have an improved drinking water source in the main survey, whereas 66% indicates this in the back-checking. Less than half of the observations were in accordance. This may be due to uncertainty regarding the definition of the technology both among the enumerators and the back-checkers as well as the respondents. This may indicate that there is need for more training or better defined categories than used in the present survey.
Toilet facility/toilet practice
Similarly, the results of the back-checking are compared to the result from the main phase of the data collection. Here, whereas 40% of the households has improved toilet facilities according to the main phase interviews, only 17% of the household has these facilities according to the back-checking. For 37% of the households, the same answer is obtained in the main phase as in the back-checking. The results of the back-checking may indicate that more work needs to be done in describing the various technologies and in training the enumerators to distinguish between the technologies.
Main drinking water source
The back-checking of the main drinking water source indicates to what degree the result depends on the enumerator collecting the data. Comparing the answers for main drinking water source for the main phase and for the backcheck, shows that 57% indicates that they have an improved drinking water source in the main survey, whereas 66% indicates this in the back-checking.
Use of the dataset must be acknowledged using a citation which would include:
Example:
COWI. Ethiopia Water and Sanitation Household Survey at Project Level 2004 - Oromia Region. Dataset downloaded from [website/source] on [date].
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
DDI_WB_ETH_2004_WSS_v01_M
Name | Affiliation |
---|---|
Olivier Dupriez | World Bank |
2005
Version 1.1: (June 2011)