Data Editing
Data Checking and Data Entry
Data preparation commenced soon after the main data collection was completed. The NRCs had to organize the safe return of all materials to the Ministry of Education where the data collection instruments could be checked, entered into computers, and then "cleaned" to remove errors prior to data analysis. The data-checking involved the "hand editing" of data collection instruments by a team of trained staff. They were required to check that: (i) all questionnaires, tests, and forms had arrived back from the sample schools, (ii) the identification numbers on all instruments were complete and accurate, and (iii) certain logical linkages between questions made sense (for example, the two questions to school heads concerning "Do you have a school library?" and "How many books do you have in your school library?").
The next step was the entry of data into computers using the WINDEM software. A team of 5-10 staff normally undertook this work. In some cases the data were "double entered" in order to monitor accuracy.
The numbers of keystrokes required to enter one copy of each data collection instrument were as follows: pupil questionnaire: 150; pupil reading test: 85; pupil mathematics test: 65; teacher questionnaire: 587; teacher reading test: 51; teacher mathematics test: 43; school head questionnaire: 319; school form: 58; and pupil name form: 51.
This information can be re-expressed to give the total number of keystrokes for the whole body of data for one country by multiplying the above figures by the number of instruments in the final data collection. In the case of Namibia the total number of keystrokes was as follows: pupil questionnaire: 762,600; pupil reading test: 429,080; pupil mathematics test: 328,250; teacher questionnaire: 358,657; teacher reading test: 15,504; teacher mathematics test: 14,061; school head questionnaire: 86,130; school form: 39,150; and pupil name form: 259,284. That is, a total of 2,292,716 keystrokes were required to enter all of the data for Namibia.
An experienced keyboard operator can work at a rate of 25 keystrokes per minute (working from multi-paged questionnaires and stopping occasionally to clarify individual questionnaire entries with the supervisor). Assuming that this kind of work rate could be sustained for, say, around a maximum of six hours per day, then the whole data entry operation for Namibia was estimated to amount to around 255 person days of data entry work This implied an estimated five weeks of work for the 10 person data entry team that operated in Namibia.
The Seychelles data collection was much smaller than Namibia's - with an estimated total of only 68 person days of data entry required. However, this implied an estimated seven weeks of work because the Seychelles only had access to a two-person data entry team.
There was a great deal of variation in the delivery dates for the initial versions of the computer-stored SACMEQ II data files. This occurred because of different testing dates and also because of different amounts of time required to complete entry of data into computers
Data Cleaning
The NRCs received written instructions and follow-up support from IIEP staff in the basic steps of data cleaning using the WINDEM software. This permitted the NRCs to (i) identify major errors in the sequence of identification numbers, (ii) cross-check identification numbers across files (for example, to ensure that all pupils were linked with their own reading and mathematics teachers), (iii) ensure that all schools listed on the original sampling frame also had valid data collection instruments and vice-versa, (iv) check for "wild codes" that occurred when some variables had values that fell outside pre-specified reasonable limits, and (v) validate that variables used as linkage devices in later file merges were available and accurate.
A second phase of data preparation directed efforts towards the identification and correction of "wild codes" (which refer to data values that that fall outside credible limits), and "inconsistencies" (which refer to different responses to the same, or related, questions). There were also some errors in the identification codes for teachers that needed to be corrected before data could be merged.
During 2002 a supplementary training programme was prepared and delivered to all countries via the Internet. This training led each SACMEQ Research Team step-by-step through the required data cleaning procedures - with the NRCs supervising "hands-on" data cleaning activities and IIEP staff occasionally using advanced software systems to validate the quality of the work involved in each data-cleaning step.
This resulted in a "cyclical" process whereby data files were cleaned by the NRC and then emailed to the IIEP for checking and then emailed back to the NRC for further cleaning.
The number of cycles required to complete all of the data cleaning ranged from lows of 5 and 9 cycles in the Seychelles and Namibia, respectively, to highs of 27 and 31 cycles in Zanzibar and Uganda, respectively. The time required to complete the all of the data cleaning took from lows of 4 and 9 months in the Seychelles and Namibia, respectively, to highs of 23 and 24 months in Uganda and Mozambique, respectively.