South Africa participated in the Southern and Eastern Africa Consortium for Monitoring Educational Quality study (SACMEQ II). SACMEQ came up with a radical shift from the traditional way of reporting learner achievement in mean scores and used the Rasch model to organise and report leaner achievement in a hierarchy of competencies from the simplest to the complex. Around 80% of South African grade 6 learners in the study reached the lower half of eight levels of competence in mathematics on the SACMEQ continuum. The lowest levels of competency were observed among learners in rural schools. In the main these were also schools where the lowest levels of resources were reported and infrastructure was inadequate. Analysis of local textbooks of mathematics showed significant gaps between what texts presented and what the official curriculum requires. This had serious implications given that textbooks are often the only resource available particularly to the disadvantaged rural schools. The considered view in this paper is that, although textbooks may not be the sole cause of poor learner performance, the relevance of learner support materials is a matter that needs to be prioritized. Finally, this analysis demonstrated the value that SACMEQ, as a monitoring mechanism for quality education, can add to providing relevant information for policy decision-making.