BACKGROUND:
THE NMIS METHODOLOGY
The NMIS uses a methodology known as Sentinel Community Surveillance (SCS). This is described in detail elsewhere6,10,11. It has the underlying aim of 'building the community voice into planning'. SCS can be described as a multi sectoral community-based information management system. There are a number of particular features of the SCS methodology.Data are collected from cluster sites, selected to be representative of a district, a region or a country.
Typically, cluster sites are communities of around 120 households, and all households in the site are included in data collection.
SCS is a repeated cyclical process, with each cycle including planning and instrument design, data collection, data analysis and interpretation, and communication of results.
Each cycle focuses on a particular area or problem, rather than trying to collect data on a wide range of problems.
Quantitative data from household questionnaires are combined with qualitative data from focus groups, key informant interviews and institutional reviews from the same communities (that is, the data are coterminous) to allow a better understanding of the quantitative data. This combined analysis is called mesoanalysis11.
Data analysis is not only in terms of indicators (for example, rate of childhood diarrhoea) but also in terms of risk (for example the risk of diarrhoea in a child with access to safe water compared with a child who does not have access to safe water).
Data analysis, and especially risk analysis, is intended to produce results in a form that can be useful for planning at household, community, district and national levels.
The same sites are revisited in subsequent cycles of data collection, allowing easy estimation of changes over time or as a result of intervention.
Each cycle of data collection and analysis requires a communication strategy to get the information to those who need it for planning.
Transfer of skills of data collection, analysis and communication over a number of cycles is an explicit aim of the methodology.
SCS is deliberately designed to concentrate data collection efforts: in
time (a series of cycles in the sentinel sites, at approximately 6 monthly intervals); in
space (representative communities are surveyed rather than collecting data from all
communities); and in subject matter (each cycle focuses on one area at a time, rather than
trying to collect all possible data on every occasion). SCS employs a type of cluster
survey methodology, but the clusters are larger than in many cluster surveys: typically
100-120 households per site, rather than the 10-50 used in most cluster surveys. And in
the SCS method, there is no sampling within each site; every household is included. This
gives greater statistical power in the data analysis and also allows the linkage of data
from the household questionnaires to other, mainly qualitative, data from the same sites.
This data relating to the whole site is combined with the household data in a
mesoanalysis11.
A key issue in the SCS methodology and in the NMIS is the selection of sites so as to be representative. In some countries, random sampling is not a possibility because no adequate sampling frame exists. In these situations, purposive selection is used, drawing on local knowledge of conditions to choose sites as representative as possible of the situation in a district, region or country. When possible, random sampling methods are used and this is the case in Nepal, where a reasonably good census sampling frame exists. In both cases, stratification is first used to ensure that certain types of sites are included in proportion to their occurrence in the population. For example, stratification can be by urban and rural sites, or by ecological zones. In the NMIS, the sample sites for the NMIS were drawn by the Central Bureau of Statistics (CBS), after stratification into development regions, ecological zones and urban/rural sites. The details of the sampling method and the selected sites are given in the report of the first NMIS cycle1 and the annexes to that report.
NMIS SAMPLE SITES
As mentioned above, these are the same sites as for the first five NMIS cycles, selected by a multistage random sampling method. As discussed in the report of the first NMIS cycle, the sites are representative of the country, of the five development regions, of the three ecological zones, of the 15 eco-development regions, and of urban and rural situations. The rural sites were selected primarily to give representation of the 15 eco-development regions but in 18 districts there are sufficient sites (four or more) to ensure reasonable district representativeness. In a further 19 districts, only 1-2 sites were selected so they cannot be relied upon to be representative of that district. Note that representation of the 15 eco-development regions is among the rural sites only; the urban sites are stratified separately and are not intended to be part of the representation of the different eco-development regions. This reflects the high proportion of the population living in rural communities (around 90%) and the difficulty of having a large enough urban samples to stratify separately among the 15 eco-development regions.
There are a total of 144 sites in the sample: 126 rural and 18 urban.
The location of the sites is shown on the map in Annex 1. Annex 1 also gives the names of
the districts in the NMIS sample, with the number of sites in each. It also includes a
list of all Districts in Nepal grouped into the 15 eco-development regions. This is
intended for officials from non-NMIS Districts who read the report to find which results
most nearly approximate to their situation (the results for the relevant eco-development
region).
THE POPULATION IN NMIS CYCLE 6
18,770 households were visited. Information was available from 18,580 (99%) households. 188 (1%) households had no one at home at the time of the visit and only two households refused to provide information.
108,899 people were included in the survey. The mean average family size is 7.4, and median 6.0. The smallest family had one member, and the largest had 90 people.
INSTRUMENTS
The instruments used in cycle 6 are reproduced in Annex 2. They include a household questionnaire, focus group guides on health and agriculture service, key informant interviews with VDC chairman, DDCs, DAO, LDO, DHO, community key informants, and health and agriculture facility institutional reviews. The instruments were designed with a Technical Group of the NMIS Steering Committee, including the Health Department, Agriculture Department, Ministry of Local Government, university and UNICEF experts in health and agriculture services, and subsequently approved by the Steering Committee.A focus group on health services was held in 141 of the 144 communities and on agriculture services in 139 of the communities.
A number of key informants were interviewed at different levels as follows:
Community key informants |
142 |
Village Development Committee Chairmen |
138 |
District Health Officers |
35 |
District Agricultural Officers |
39 |
District Development Committee Chairmen |
31 |
Local Development Officers |
34 |
An institutional review was carried out in 132 health facilities and 115 agricultural/livestock service centres.
Coding sheets and data entry formats were created for each instrument. Data entry was programmed using Epi Info (version 6)12. The household data were entered twice and validated using Epi Info. The questionnaires and other data collection instruments, after translation into Nepali, were piloted several times to ensure that they were appropriate to the households, health facility workers and focus groups concerned and that the coding and data entry arrangements were satisfactory.
Training and field work
Field staff were recruited in October 1997. They were recruited from and trained in five regional centres: Kathmandu, Birgunj, Nepalgunj, Pokhara and Sunsari. Twenty-six teams, each containing four or five members, were recruited. The thirty field supervisors were trained in Kathmandu. The field survey was carried out between October and November 1997.
When communities were revisited during cycle 6, the opportunity was taken to give them a summary of the results of cycle 5 and conduct focus groups to discuss the implications of the key findings and the ways in which important messages might best be disseminated.
Data coding and entry began during fieldwork, with messengers bringing back as much data as possible to Kathmandu from each of the field teams. Data entry began in December 1997 and data entry and cleaning of quantitative data was completed by end of January 1998. Data entry of qualitative data was completed by early April 1998.
Analysis
The analysis had several aims: to produce national indicators on coverage and perceptions of government health and agriculture services, and information about other government services including local government services; to examine variables that might be related to the use of health and agriculture services, and to look for contrasts so as to discover actionable factors that might help to improve the situation.
The analysis was performed using the Epi Info package (version 6)12. This public domain computer software package assists with questionnaire creation, data entry and data analysis. SPSS was used to handle data from questions with multiple responses.
The quantitative analysis was supplemented by qualitative data from focus groups, key informants and observation. The records of the focus groups were reviewed to get an overview of the ideas expressed. Each focus group was then coded according to the issues raised by the participants. These codes were then related to information from the household questionnaires from the same community. The focus group themes are shown in annex 3, with the frequency of each one.
Weighting of results to give national indicators.
As explained in the report of the first cycle of the NMIS1, the sample sizes of Districts were not proportional to the populations of the Districts and weights were calculated to take this into account when producing national indicators. These weights are used when giving national level indicators. The weighted and unweighted values for key indicators are shown in annex 4. The Epi Info programme CSAMPLE was used to calculate weighted values of key indicators. In practice, the weighted values are close to the unweighted values. Unless stated otherwise, values of indicators quoted in the Results section for the whole of Nepal are weighted. Values of indicators at subnational levels (such as eco development regions and in urban and rural sites separately) given in Annex 5 are not shown weighted. The weights give the correct balance of individual areas in national figures, for example taking into account the relative over sampling of mountain eco development regions and urban sites.