Reproductive Health Survey 2001

The estimates for a sample survey are affected by two types of errors: non-sampling error and sampling error. Non-sampling error is the result of mistakes made in carrying out data collection and data processing, including the failure to locate and interview the right household, errors in the way questions are asked or understood, and data entry errors. Although intensive quality control efforts were made during the implementation of the AZRHS01 to minimize this type of error, non-sampling errors are impossible to avoid altogether and difficult to evaluate statistically. Sampling error is a measure of the variability between an estimate and the true value of the population parameter intended to be estimated, which can be attributed to the fact that a sample rather than a complete enumeration was used to produce it. In other words, sampling error is the difference between the expected value for any variable measured in a survey and the value estimated by the survey. This sample is only one of the many probability samples that could have been selected from the female population aged 15-44 using the same sample design and projected sample size. Each of these samples would have yielded slightly different results from the actual sample selected.

Because the statistics presented here are based on a sample, they may differ by chance variations from the statistics that would result if all women 15-44 years of age in Azerbaijan would have been interviewed. Sampling error is usually measured in terms of the variance and standard error (square root of the variance) for a particular statistic (mean, proportion, or ratio). The standard error (SE) can be used to calculate confidence intervals (CI) of the estimates within which we can say with a given level of certainty that the true value of population parameter lies. For example, for any given statistic calculated from the survey sample, there is a 95 percent probability that the true value of that statistic will lie within a range of plus or minus two SE of the survey estimate.
The chances are about 68 out of 100 (about two out of three) that a sample estimate would fall within one standard error of a statistic based on a complete count of the population. The estimated sampling errors for 95% confidence intervals (1.96 x SE) for selected proportions and sample sizes are shown in Table A.1 of the Final Report. The estimates in Table A.1 can be used to estimate 95% confidence intervals for the estimated proportions shown for each sample size. The sampling error estimates include an average design effect of 1.6, needed because the AZRHS01 did not employ a simple random sample but included clusters of elements in the second stage of the sample selection.

The selection of clusters is generally characterized by some homogeneity that tends to increase the variance of the sample. Thus, the variance in the sample for the AZRHS01 is greater than a simple random sample would be due to the effect of clustering. The design effect represents the ratio of the two variance estimates: the variance of the complex design using clusters, divided by the variance of a simple random sample using the same sample size (Kish L, 1967). For more details regarding design effects for specific reproductive health variables, the reader is referred to the Le and Verma report, which studied demographic and health surveys in 48 countries (Le TN and Verma JK, 1997). The pattern of variation of design effects is shown to be consistent across countries and variables. Variation among surveys is high but less so among variables. Urban - rural and regional differentials in design effects are small, which can be attributed to the fact that similar sample designs and cluster sizes were used across domains within each country. At the country level, the overall design effect, averaged over all variables and countries, is about 1.5 (we used 1.6 in Table A. 1 to be slightly more conservative).

To obtain the 95% CI for proportions or sample sizes not shown in the table, one may interpolate. For example, for a sample size of 200 and a point estimate of 25% (midway between 0.20/0.80 and 0.30/0.70), the 95% CI would be plus or minus 7.5 percentage points; for a sample size of 300 (midway between 200 and 400) and an estimate of 20%, the 95% CI would be plus or minus 6.0 percentage points.

Differences between estimates discussed in this report were found to be statistically significant at the five percent level using a two-tailed normal deviate test (p=0.05). This means that in repeated samples of the same type and size, a difference as large as the one observed would occur in only 5% of samples if there were, in fact, no differences between the proportion in the population.

The relative standard error of a statistic (also called "coefficient of variation") is the ratio of the standard error (SE) for that statistic to the value of the statistic. It is usually expressed as a percent of the estimate. Estimates with a relative standard error of 30% or more are generally viewed as unreliable by themselves, but they may be combined with other estimates to make comparisons of greater precision. For example, an estimate of 20% based on a sample size of only 50 observations yields a SE of 7% (one half the 95% confidence interval shown in Table A.1). The relative standard error would be 35% (the ratio of the SE of 7% to the estimate of 20%), too large for the estimate to be reliable.