Home / Central Data Catalog / WLD_2003_PISA_V01_M
central

Programme for International Student Assessment 2003

Australia, Austria, Belgium...and 36 more, 2003
Reference ID
WLD_2003_PISA_v01_M
Producer(s)
Organisation for Economic Co-operation and Development
Metadata
DDI/XML JSON
Created on
Sep 05, 2014
Last modified
Jun 14, 2022
Page views
53697
Downloads
14663
Download related resources
Questionnaires
Educational Career Questionnaire
Download [PDF, 259.54 KB]
Author(s) Australian Council for Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Service (ETS, USA) National Institute for Educational Policy Research (NIER, Japan) Westat (USA)
Language English
Download http://catalog.ihsn.org//catalog/4921/download/60981
Information Communication Technology Questionnaire
Download [PDF, 484.26 KB]
Author(s) Australian Council for Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Service (ETS, USA) National Institute for Educational Policy Research (NIER, Japan) Westat (USA)
Language English
Download http://catalog.ihsn.org//catalog/4921/download/60982
School Questionnaire
Download [PDF, 921.83 KB]
Author(s) Australian Council for Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Service (ETS, USA) National Institute for Educational Policy Research (NIER, Japan) Westat (USA)
Language English
Download http://catalog.ihsn.org//catalog/4921/download/60983
Student Questionnaire
Download [PDF, 1.03 MB]
Author(s) Australian Council for Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Service (ETS, USA) National Institute for Educational Policy Research (NIER, Japan) Westat (USA)
Language English
Download http://catalog.ihsn.org//catalog/4921/download/60984
Reports
Where immigrant students succeed: A comparative review of performance and engagement in PISA 2003
Download [PDF, 3.49 MB]
Author(s) Petra Stanat and Gayle Christensen at the Max Planck Institute for Human Development, Berlin, Germany.
Language English
Description This report analyses evidence from PISA 2003 on outcomes of schooling including how well immigrant students perform in key school subjects at the age of 15, as well as how they assess themselves as learners and what their general attitudes are towards school.
Table of contents Foreword.............................................................................................. 3
Executive Summary............................................................................... 7
READER’S GUIDE.....................................................................................13
Chapter 1
Countries’ immigration histories and populati ons................................................15
Introduction ...........................................................................................16
Immigration and integration.........................................................................17
Immigration histories and general approaches to immigration and integration ................18
Immigrant populations ................................................................................21
Research questions addressed in the report .....................................................24
Immigrant students in the PISA sample ..........................................................25
Chapter 2
Performance of immigrant students in PISA 2003................................................29
Introduction............................................................................................30
Immigrant student performance in the OECD and partner countries........................30
Performance of immigrant students and the language spoken at home....................46
Performance of immigrant students and gender...............................................49
Performance of immigrant students in the context of migration trends in the receiving country...49
Conclusions..........................................................................................54
Chapter 3
Bac kground charact eristics, mat hematics performance and learning
environments of immigrant students.............................................................57
Introduction..........................................................................................58
Immigrant families’ educational and socio-economic background .............................60
Relationships between performance differences and differences in educational and socio-economic
background among immigrant and non-immigrant student groups ...........................64
Disparities specifically related to students’ immigrant status..................................69
Differences between immigrant and native students within and between schools..........71
Summary and conclusions...........................................................................79
Chapter 4
Immigrant students’ app roac hes to learning...................................................83
Introduction1..........................................................................................84
Students’ interest and motivation in mathematics.................................................88
Students’ self-related beliefs ........................................................................97
Emotional dispositions in mathematics.........................................................103
Students’ attitudes towards and perceptions of schools.....................................104
Summary of differences between immigrant and non-immigrant students in learning
characteristics........................................................................................110
Conclusions...........................................................................................114
Chapter 5
Policies and practic es to help immigrant students attai n proficiency in the
languag e of instruction.............................................................................117
Introduction..........................................................................................118
PISA 2003 supplementary survey on national policies and practices to help immigrant students
attain proficiency in the language of instruction..................................................118
Policies and practices designed to help newly arrived immigrant adults attain proficiency in the
case countries’ official language(s) ..................................................................121
Assessment of language proficiency in pre-primary (ISCED 0) and primary (ISCED 1) education.... 128
Language support for immigrant students in pre-primary education (ISCED 0) .............................129
Language support for immigrant students in primary education (ISCED 1) and lower secondary
education (ISCED 2).........................................................................................131
Country descriptions of language support measures in primary (ISCED 1) and lower secondary
(ISCED 2) education.........................................................................................134
Supplementary classes to improve proficiency in immigrant students’ native languages.................145
Additional school resources...................................................................................153
Summary and conclusions....................................................................................153
References....................................................................................................157
Annex A
Annex A1: Technical bac kground..........................................................................165
Annex A2: Summary descriptions of the five levels of reading proficiency........173
Annex B
Annex B1: Data tab les for chapt ers 1, 2, 3 and 4....................................................175
Annex C
Annex C1: The development and implementati on of PISA – a collaborati ve effort......................219
Download http://catalog.ihsn.org//catalog/4921/download/60985
Learning Mathematics for Life: A Perspective from PISA
Download [PDF, 6.13 MB]
Author(s) This thematic report is the product of a collaborative effort between the countries involved in PISA, the members of the Mathematics Expert Group listed in Annex A6 who worked to develop the assessment items, the experts who guided the thematic report t
Language English
Description This report concentrates on in-depth analysis of PISA 2003 mathematics performance data at the level of individual tasks and test items.
Download http://catalog.ihsn.org//catalog/4921/download/60986
Cadre d’évaluation de PISA 2003
Download [PDF, 1.06 MB]
subtitle Connaissances et compétences en mathématiques, lecture, science et résolution de problèmes
Author(s) Cette publication a été préparée par la Direction de l’éducation de l’OCDE sous la responsabilité d’Andreas Schleicher et de Claudia Tamassia. Elle est publiée sous la responsabilité du Secrétaire général de l’OCDE.
Language French
Table of contents AVANT-PROPOS ......................................................................... 7
PRÉSENTATION DU CYCLE 2003 DU PROGRAMME PISA DE L’OCDE.......... 9
Vue d’ensemble ........................................................................ 9
Caractéristiques fondamentales du cycle PISA 2003........................... 10
Originalité du programme PISA .................................................. 15
Aperçu des aspects évalués dans chaque domaine .............................. 16
Mise en oeuvre du cycle PISA 2003 et présentation des résultats ........... 19
Nature et utilisation des questionnaires contextuels ........................... 20
Élaboration conjointe du programmeOCDE/PISA
et de ses cadres conceptuels d’évaluation ....................................... 22
Chapitre 1 : CULTURE MATHÉMATIQUE .......................................... 25
Défi nition du domaine .............................................................. 26
Fondements théoriques du cadre d’évaluation
de la culture mathématique OCDE/PISA ....................................... 28
Organisation du domaine ........................................................... 33
• Situations et contextes .......................................................... 35
• Contenu mathématique : les quatre « idées majeures » ................... 37
• Processus mathématiques ...................................................... 42
Évaluer la culture mathématique .................................................. 55
• Caractéristiques des épreuves ................................................. 55
• Structure de l’évaluation ....................................................... 59
• Présentation des résultats : développement
des échelles de compétence en mathématiques............................. 60
• Outils et instruments ........................................................... 62
Synthèse ............................................................................... 62
Exemples supplémentaires ......................................................... 63
• Mathématiques, unité 1 : PHARE ............................................ 64
• Mathématiques, unité 2 : TARIFS POSTAUX............................... 68
• Mathématiques, unité 3 : BATTEMENTS DE COEUR.................... 71
• Mathématiques, unité 4 : PAIEMENT À LA SUPERFICIE ............... 74
• Mathématiques, unité 5 : TAILLE DES ÉLÈVES ........................... 76
• Mathématiques, unité 6 : BALANÇOIRE ................................... 78
• Mathématiques, unité 7 : RÉSERVOIR D’EAU ............................ 80
• Mathématiques, unité 8 : TEMPS DE RÉACTION ........................ 82
• Mathématiques, unité 9 : ASSEMBLAGE DE BLOCS..................... 85
• Mathématiques, unité 10 : TAUX D’UN MÉDICAMENT
DANS LE SANG................................................................. 90
• Mathématiques, unité 11 : L’IMMEUBLE TORSADÉ..................... 94
• Mathématiques, unité 12 : CONCERT ROCK............................. 99
• Mathématiques, unité 13 : TAPIS ROULANTS ...........................101
Élaboration des idées majeures ...................................................103
• Quantité..........................................................................103
• Espace et formes................................................................106
• Variations et relations ..........................................................109
• Incertitude .......................................................................113
Chapitre 2 : COMPRÉHENSION DE L’ÉCRIT ......................................119
Défi nition du domaine .............................................................120
Format de texte .....................................................................121
• Textes continus..................................................................121
• Textes non continus ............................................................122
Caractéristiques des items .........................................................124
• Cinq processus (aspects).......................................................124
• Types d’items....................................................................130
• Codage ...........................................................................131
Situations .............................................................................131
Présentation des résultats..........................................................133
• Établissement des échelles de compréhension de l’écrit.................133
• Présentation des résultat ......................................................133
• Élaboration d’une « carte » des items .......................................135
• Niveaux de compétence en compréhension de l’écrit ...................136
Chapitre 3 : CULTURE SCIENTIFIQUE .............................................145
Défi nition du domaine .............................................................146
Organisation du domaine ..........................................................149
• Connaissances ou concepts scientifi ques....................................150
• Processus scientifi ques .........................................................151
• Situations ou contextes : les champs d’application........................154
Caractéristiques des épreuves et exemples .....................................156
• Science, unité 1: ARRÊTEZ CE MICROBE ! ................................157
• Science, unité 2 : PETER CAIRNEY..........................................159
• Science, unité 3 : MAÏS ........................................................163
Structure de l’évaluation...........................................................166
Échelles de compétence ...........................................................168
Autres questions.....................................................................170
Chapitre 4 : RÉSOLUTION DE PROBLÈMES .......................................171
Introduction..........................................................................172
Contexte .............................................................................172
Défi nition du domaine .............................................................174
Organisation du domaine ..........................................................176
• Types de problèmes ............................................................178
– Résolution de problèmes, unité 1 : DITES « NON » À LA DOULEUR...181
– Résolution de problèmes, unité 2 : GESTION DES VENTES DE CD...185
– Résolution de problèmes, unité 3 : POMPE DE VÉLO.................190
• Processus de résolution de problèmes ......................................191
• Synthèse des types de problèmes ............................................193
• Situations.........................................................................194
La place de la résolution de problèmes dans le cycle
d’évaluation PISA 2003 ............................................................194
• Compétences clés ..............................................................194
• Rôle de la résolution de problèmes dans la perspective de l’évolution
du marché du travail et des compétences requises ........................195
Caractéristiques de l’évaluation ..................................................199
• Accessibilité et équité..........................................................199
• Calculatrices.....................................................................199
Types d’items ........................................................................200
• Items à choix multiple .........................................................200
• Items fermés à réponse construite...........................................201
• Items ouverts à réponse construite ..........................................201
• Groupes ou unités d’items ....................................................202
• Guides de correction...........................................................202
• Codes à deux chiffres ..........................................................203
• Structure générale de l’évaluation ...........................................204
Analyses et présentation des résultats ...........................................204
Développements potentiels du cadre d’évaluation pour les cycles
d’évaluation OCDE/PISA à venir................................................205
• Résolution de problèmes dans le cadre d’un travail de groupe .........205
• Administration informatisée des tests d’évaluation .......................206
Exemples supplémentaires ........................................................206
• Résolution de problèmes, unité 4 : PILES ..................................207
• Résolution de problèmes, unité 5: ROUAGES .............................211
• Résolution de problèmes, unité 6: VENTE DE LIVRES ...................216
BIBLIOGRAPHIE ...................................................................... 221
ANNEXE ............................................................................... 225
Download http://catalog.ihsn.org//catalog/4921/download/60987
Problem Solving for Tomorrow’s World
Download [PDF, 1.66 MB]
subtitle First Measures of Cross-Curricular Competencies from PISA 2003
Author(s) The report is the product of a collaborative effort between the countries participating in PISA, the experts and institutions working within the framework of the PISA Consortium, and the OECD.
Language English
Description How can countries raise their performance in this increasingly important competency area and what can countries with lower performance levels learn from those where students do well? This report seeks to answer such questions. It complements Learning for Tomorrow’s World – First Results from PISA 2003, which focuses on knowledge and skills in mathematics, science and reading, and it goes beyond an examination of the relative standing of countries in students’ problemsolving performance by considering how problem-solving performance relates to learning outcomes in other areas and how it varies between the genders and between socio-economic groups. It also provides insights into some of the factors that are associated with the development of problem-solving skills and into how these factors interact and what the implications are for policy development. Most importantly, the report sheds light on countries that succeed in achieving high performance standards while at the same time providing an equitable distribution of learning opportunities. Results in these countries pose challenges for other countries by showing what it is possible to achieve.
Table of contents CHAPTER 1
PISA 2003 AND PROBLEM SOLVING ....................................................................... 11
Introduction .................................................................................................................... 12
Problem solving in PISA 2003 .................................................................................... 16
Organisation of this report .......................................................................................... 20
READERS’ GUIDE ........................................................................................................... 22
CHAPTER 2
PROBLEM SOLVING IN PISA 2003 – HOW IT WAS MEASURED
AND HOW STUDENTS PERFORMED ........................................................................ 25
Introduction .................................................................................................................... 26
Problem solving in PISA ............................................................................................. 26
Organisation of the assessment area .......................................................................... 27
Problems chosen for the PISA problem-solving assessment ................................ 28
The PISA problem-solving scale ............................................................................... 28
• Level 3: Reflective, communicative problem solvers ................................... 29
• Level 2: Reasoning, decision-making problem solvers ................................ 30
• Level 1: Basic problem solvers .......................................................................... 30
• Below Level 1: Weak or emergent problem solvers ..................................... 30
• Decision making – the Cinema Outing problem .......................................... 32
• System analysis and design – the Children’s Camp problem ..................... 34
• Trouble shooting – the Irrigation problem .................................................... 36
The percentage of students at each proficiency level of problem solving ........... 39
• Mean performance of countries ........................................................................ 41
The distribution of problem-solving capabilities within countries ................... 44
Implications for policy .................................................................................................. 46
CHAPTER 3
STUDENT PERFORMANCE IN PROBLEM SOLVING COMPARED WITH
PERFORMANCE IN MATHEMATICS, READING AND SCIENCE ....................... 49
Introduction .................................................................................................................... 50
Problem-solving framework and test development ............................................... 50
• Emphasis on problem-solving processes ......................................................... 50
• Low content requirements ................................................................................. 51
• The key skills tested in problem solving ......................................................... 51
• Correlations between performance in reading, mathematics,
science and problem solving .............................................................................. 54
Comparison between performances in mathematics and
problem solving at the country level ......................................................................... 55
Implications for policy .................................................................................................. 57
CHAPTER 4
STUDENT PERFORMANCE ON THE PROBLEM-SOLVING ITEMS .................... 59
Introduction .................................................................................................................... 60
Decision-making units .................................................................................................. 62
• Energy Needs ........................................................................................................ 62
• Cinema Outing ..................................................................................................... 67
• Holiday .................................................................................................................... 70
• Transit System ....................................................................................................... 73
System analysis and design units ................................................................................. 76
• Library System ...................................................................................................... 76
• Design by Numbers© ........................................................................................... 82
• Course Design ....................................................................................................... 88
• Children’s Camp ................................................................................................... 91
Trouble-shooting units .................................................................................................. 94
• Irrigation ................................................................................................................ 94
• Freezer .................................................................................................................... 98
Summary ........................................................................................................................101
CHAPTER 5
THE ROLE THAT GENDER AND STUDENT BACKGROUND
CHARACTERISTICS PLAY IN STUDENT PERFORMANCE IN PROBLEM
SOLVING .......................................................................................................................103
Introduction ..................................................................................................................104
Gender differences in problem solving ...................................................................104
Comparison with gender differences in other assessment areas ..........................107
Parental occupational status ......................................................................................110
Parental education .......................................................................................................112
Possessions related to “classical” culture .................................................................113
Family structure ...........................................................................................................115
Place of birth and language spoken at home ..........................................................116
Implications for policy ................................................................................................119
REFERENCES .................................................................................................................121
ANNEX A ...................................................................................................................123
Annex A1 Construction of indices and other derived measures
from the student context questionnaire ...........................................124
Annex A2 Detailed results from the factor analysis in Chapter 3 ..................126
Annex A3 The PISA target population and the PISA samples .........................128
Annex A4 Standard errors, significance tests and subgroup comparisons ......137
Annex A5 Quality assurance ...................................................................................138
Annex A6 Development of the PISA assessment instruments ........................139
Annex A7 Reliability of the marking of open-ended items .............................141
ANNEX B
Data tables for the chapters .......................................................................................143
ANNEX C
The development and implementation of PISA – a collaborative effort ..............157
Download http://catalog.ihsn.org//catalog/4921/download/60988
Learning for Tomorrow’s World: First Results from PISA 2003
Download [PDF, 5.43 MB]
Author(s) The report is the product of a collaborative effort between the countries participating in PISA, the experts and institutions working within the framework of the PISA Consortium, and the OECD.
Language English
Description This report presents first results from the PISA 2003 assessment, which focused on mathematics.
Table of contents CHAPTER 1
INTRODUCTION ........................................................................................................... 19
PISA – An overview ...................................................................................................... 20
What PISA measures and how .................................................................................... 23
• Literacy in PISA: what is measured .................................................................. 25
• The PISA instruments: how measurement takes place ................................ 25
• The PISA student population ............................................................................. 27
What is different about the PISA 2003 survey? ...................................................... 28
• It establishes a detailed understanding of student performance in
mathematics ........................................................................................................... 28
• It deepens exploration of cross-curricular competencies ........................... 29
• It introduces new background information about students
and schools ............................................................................................................. 29
• It allows for comparison of change over time ............................................... 29
Organisation of the report ........................................................................................... 30
READERS’ GUIDE ........................................................................................................... 33
CHAPTER 2
A PROFILE OF STUDENT PERFORMANCE IN MATHEMATICS ........................ 35
Introduction .................................................................................................................... 36
The PISA approach to assessing mathematics performance ................................. 37
• How mathematics is defined .............................................................................. 37
• How mathematics is measured .......................................................................... 38
• How the PISA tests were constructed ............................................................. 42
• How the PISA tests were designed, analysed and scaled ............................. 44
• How results are reported ................................................................................... 46
What students can do in four areas of mathematics .............................................. 51
• Student performance on the mathematics/space and shape scale ............ 51
• Student performance on the mathematics/change and
relationships scale ................................................................................................. 64
• Student performance on the mathematics/quantity scale .......................... 74
• Student performance on the mathematics/uncertainty scale .................... 85
Overall performance in mathematics ........................................................................ 89
• The relative strengths and weaknesses of countries in different areas
of mathematical content ..................................................................................... 89
• A summary picture of mathematics performance ........................................ 90
• Gender differences in mathematics ................................................................. 95
The socio-economic context of country performance ........................................ 99
Implications for policy ................................................................................................103
CHAPTER 3
STUDENT LEARNING: ATTITUDES, ENGAGEMENT AND STRATEGIES ..........109
Introduction ..................................................................................................................110
• Existing evidence on student approaches to learning and
how it frames PISA’s approach ........................................................................113
• Measuring whether students are likely to adopt effective
approaches to learning ......................................................................................114
Students’ engagement with learning in mathematics and school
more generally ..............................................................................................................116
• Interest in and enjoyment of mathematics ....................................................116
• Instrumental motivation ...................................................................................121
• Students’ perception of how well school has prepared them for life ........125
• Students’ sense of belonging at school ..........................................................127
Students’ beliefs about themselves ..........................................................................132
• Students’ self-concept in mathematics ..........................................................132
• Students’ confidence in overcoming difficulties in mathematics .............136
Students’ anxiety in mathematics .............................................................................138
Students’ learning strategies .....................................................................................141
• Controlling the learning process ....................................................................141
• Memorisation and elaboration strategies ......................................................145
How learner characteristics relate to each other
and influence performance ........................................................................................145
How learner characteristics vary across schools ...................................................150
A summary picture of gender differences in learner characteristics ...............151
Implications for policy ................................................................................................156
CHAPTER 4
HOW STUDENT PERFORMANCE VARIES BETWEEN SCHOOLS AND
THE ROLE THAT SOCIO-ECONOMIC BACKGROUND PLAYS IN THIS .........159
Introduction ..................................................................................................................160
Securing consistent standards for schools: a profile of betweenand
within-school differences in student performance .......................................160
The quality of learning outcomes and equity in the distribution
of learning opportunities ...........................................................................................164
Socio-economic difference, school difference and the role that education
policy can play in moderating the impact of socio-economic
disadvantage ..................................................................................................................186
Implications for policy ................................................................................................191
CHAPTER 5
THE LEARNING ENVIRONMENT AND THE
ORGANISATION OF SCHOOLING ..........................................................................207
Introduction ..................................................................................................................208
The learning environment and school climate ......................................................211
• Students’ perceptions of individual support from their teachers ............211
• Student-related factors affecting the school climate for mathematics ......214
• Teacher-related factors affecting the general school climate ...................219
• The combined effect of school climate factors ............................................225
School policies and practices .....................................................................................228
• School admittance policies ...............................................................................228
• Assessment policies and practices ...................................................................229
• Approaches to school management ................................................................233
• The combined effect of school policies and practices ................................238
Resources invested in education ...............................................................................240
• Student time invested in learning ...................................................................240
• Availability and quality of human resources .................................................245
• The quality of schools’ physical infrastructure and
educational resources ........................................................................................248
• Public and private stakeholders .......................................................................250
• The combined effect of school resources .....................................................254
What makes a difference for school performance ...............................................255
Institutional differentiation ........................................................................................261
Implications for policy ................................................................................................265
CHAPTER 6
A PROFILE OF STUDENT PERFORMANCE IN READING AND SCIENCE ........271
Introduction ..................................................................................................................272
How reading literacy is measured in PISA .............................................................272
Student performance in reading ...............................................................................273
• The mean performances of countries in reading ........................................ 280
• Differences in reading performance between PISA 2000
and PISA 2003 .....................................................................................................282
• Gender differences in reading literacy ..........................................................284
How science performance is measured in PISA ...................................................286
Student performance in science ...............................................................................293
• The mean performances of countries in science .........................................293
• Differences in science performance between PISA 2000
and PISA 2003 .....................................................................................................295
• Gender differences in science ..........................................................................296
Implications for policy ................................................................................................298
• Reading .................................................................................................................298
• Science ..................................................................................................................299
REFERENCES .................................................................................................................301
ANNEX A ...................................................................................................................305
Annex A1 Construction of indices and other derived measures
from the student and school context questionnaires .....................306
Annex A2 Issues relating to the reporting of mathematics performance ........317
Annex A3 The PISA target population, the PISA samples and
the definition of schools .......................................................................320
Annex A4 Standard errors, significance tests and subgroup
comparisons ............................................................................................329
Annex A5 Quality assurance ...................................................................................332
Annex A6 Development of the PISA assessment instruments ........................333
Annex A7 Reliability of the marking of open-ended items .............................337
Annex A8 Comparison of results from the PISA 2000 and PISA 2003
assessments ..............................................................................................338
ANNEX B ...................................................................................................................339
Annex B1 Data tables for the chapters .................................................................340
Annex B2 Performance differences between regions within countries ..........451
ANNEX C ...................................................................................................................473
The development and implementation of PISA – a collaborative effort ............474
Download http://catalog.ihsn.org//catalog/4921/download/60989
Mathematics Teaching and Learning Strategies in PISA
Download [PDF, 4.92 MB]
Author(s) This report is the product of a collaborative effort between the countries participating in PISA, the experts and institutions working within the framework of the PISA Consortium, the OECD, and Edudata Canada at the University of British Columbia.
Language English
Download http://catalog.ihsn.org//catalog/4921/download/60990
Résoudre des problèmes, un atout pour réussir
Download [PDF, 1.72 MB]
subtitle Premières évaluations des compétences transdisciplinaires issues de PISA 2003
Author(s) Ce rapport est le fruit des efforts concertés des pays participant à l’enquête PISA, des experts et des institutions qui oeuvrent au sein du consortium PISA et de l’OCDE.
Language French
Table of contents CHAPITRE 1
L’ÉVALUATION PISA DES COMPÉTENCES DES ÉLÈVES EN RÉSOLUTION
DE PROBLÈMES .............................................................................................................. 11
Introduction .................................................................................................................... 12
L’évaluation des compétences en résolution de problèmes
du cycle PISA 2003 ....................................................................................................... 17
Structure du présent rapport ...................................................................................... 21
GUIDE DU LECTEUR ..................................................................................................... 23
CHAPITRE 2
L’ÉVALUATION DES COMPÉTENCES DES ÉLÈVES EN RÉSOLUTION
DE PROBLÈMES DU CYCLE PISA 2003 – MÉTHODE ET RÉSULTATS ................ 25
Introduction .................................................................................................................... 26
L’évaluation des compétences en résolution de problèmes
dans l’enquête PISA ..................................................................................................... 26
Organisation du domaine d’évaluation ..................................................................... 27
Items retenus en vue de l’évaluation PISA des compétences
en résolution de problèmes ......................................................................................... 29
L’échelle PISA de résolution de problèmes ............................................................ 30
• Niveau 3 : compétences poussées en résolution de problèmes
et grande faculté de réflexion et de communication .................................... 30
• Niveau 2 : bonnes compétences en résolution de problèmes,
capacité de raisonnement et de prise de décision ......................................... 30
• Niveau 1 : compétences élémentaires en résolution de problèmes .......... 31
• Sous le niveau 1 : compétences insuffisantes
(ou en voie de développement) en résolution de problèmes ..................... 31
• Problème de prise de décision – Unité « Sortie au cinéma » .................... 33
• Problème de conception et d’analyse de systèmes –
Unité « Colonie de vacances » ........................................................................... 36
• Problème de traitement de dysfonctionnements – Unité « Irrigation » ....... 38
Proportions d’élèves à chaque niveau de l’échelle de compétence
en résolution de problèmes ......................................................................................... 41
• Performance moyenne des pays ........................................................................ 43
La répartition des compétences en résolution de problèmes
au sein des pays .............................................................................................................. 45
Implications en termes de politique éducative ....................................................... 48
CHAPITRE 3
COMPARAISON DES PERFORMANCES DES ÉLÈVES EN RÉSOLUTION
DE PROBLÈMES ET EN MATHÉMATIQUES, EN LECTURE ET EN SCIENCES ..... 51
Introduction .................................................................................................................... 52
Cadre d’évaluation des compétences en résolution de problèmes
et développement des items y afférents ................................................................... 52
• Priorité aux processus de résolution de problèmes ..................................... 52
• Faibles exigences au niveau des connaissances « scolaires » ....................... 53
• Principales compétences testées en résolution de problèmes ................... 53
• Corrélations entre les performances en mathématiques, en lecture,
en sciences et en résolution de problèmes ..................................................... 57
Comparaison entre les performances en résolution de problèmes
et en mathématiques à l’échelle nationale ............................................................... 58
Implications en termes de politique éducative ....................................................... 60
CHAPITRE 4
PERFORMANCE DES ÉLÈVES EN RÉSOLUTION DE PROBLÈMES DANS PISA ..... 63
Introduction .................................................................................................................... 64
Unités de prise de décision .......................................................................................... 66
• Besoins en énergie ................................................................................................ 66
• Sortie au cinéma ................................................................................................... 72
• Vacances .................................................................................................................. 75
• Correspondances .................................................................................................. 78
Unités de conception et d’analyse de systèmes ...................................................... 82
• Système de gestion d’une bibliothèque ........................................................... 82
• Design by Numbers© ........................................................................................... 89
• Programme de cours ........................................................................................... 95
• Colonie de vacances ............................................................................................. 98
Unités de traitement de dysfonctionnements .......................................................101
• Irrigation ..............................................................................................................101
• Congélateur .........................................................................................................105
Résumé ...........................................................................................................................108
CHAPITRE 5
IMPACT DES CARACTÉRISTIQUES CONTEXTUELLES ET DU SEXE DES ÉLÈVES
SUR LEUR PERFORMANCE EN RÉSOLUTION DE PROBLÈMES .......................109
Introduction ..................................................................................................................110
Écarts de performance entre les sexes en résolution de problèmes ................110
Comparaison des écarts de score entre les sexes en résolution
de problèmes et dans les autres domaines d’évaluation ........................................113
Statut professionnel des parents ...............................................................................116
Niveau de formation des parents .............................................................................118
Patrimoine culturel « classique » à la maison ........................................................120
Structure familiale .......................................................................................................121
Pays d’origine et langue parlée à la maison ...........................................................123
Implications en termes de politique éducative .....................................................126
BIBLIOGRAPHIE ...........................................................................................................129
ANNEXE A ...................................................................................................................131
Annexe A1 Construction des indices et autres mesures dérivées
à partir des questionnaires Élève et Établissement .......................132
Annexe A2 Résultats détaillés de l’analyse factorielle présentée
au chapitre 3 ..........................................................................................135
Annexe A3 La population cible et les échantillons de l’enquête PISA ...........137
Annexe A4 Erreurs types, tests de signification et comparaisons
de sous-groupes .....................................................................................146
Annexe A5 Assurance qualité ..................................................................................147
Annexe A6 Développement des instruments d’évaluation PISA ....................148
Annexe A7 La fidélité dans la correction des items à réponse ouverte ............150
ANNEXE B
Tableaux de données ...................................................................................................151
ANNEXE C
Développement et mise en oeuvre de PISA – une initiative concertée ................163
Download http://catalog.ihsn.org//catalog/4921/download/60991
The PISA 2003 Assessment Framework
Download [PDF, 1.51 MB]
subtitle Mathematics, Reading, Science and Problem Solving Knowledge and Skills
Author(s) Andreas Schleicher and Claudia Tamassia
Language English
Download http://catalog.ihsn.org//catalog/4921/download/60992
Are Students Ready for a Technology-Rich World?
Download [PDF, 1.82 MB]
subtitle What PISA Studies Tell Us
Author(s) The report is the product of a collaborative effort between the countries participating in PISA, the experts and institutions working within the framework of the PISA Consortium, and the OECD.
Language English
Description This report sheds light on how students are using computers and shows that they use them for a wide range of functions and not just to play games.
Table of contents FOREWORD ................................................................................. 3
CHAPTER 1
ICT IN PISA AND EDUCATIONAL POLICY ............................................... 7
Introduction ................................................................................. 8
Drivers for the integration of technology into schools .................................. 8
PISA 2003 and how information on ICT was collected .................................. 9
Structure of report ..........................................................................12
READERS’ GUIDE ............................................................................13
CHAPTER 2
STUDENTS’ ACCESS TO ICT ................................................................15
Key points .....................................................................................16
How universal is access to ICT? ..........................................................17
ICT and other educational resources at home ............................................25
ICT resources at school .......................................................................26
Conclusions and implications ..................................................................30
CHAPTER 3
STUDENTS’ USE OF AND ATTITUDES TOWARDS ICT ..................................33
Key points ....................................................................................34
Introduction ...................................................................................36
Frequency of use by location .................................................................36
Frequency of use by type of use ............................................................37
Attitudes towards ICT .......................................................................43
Students’ confidence in using ICT ..........................................................45
Conclusions and implications .................................................................50
Download http://catalog.ihsn.org//catalog/4921/download/60993
Item Compendia 2003
Download [PDF, 1.46 MB]
School Compendia 2003
Download [PDF, 7.73 MB]
Student Compendia 2003
Download [PDF, 6.15 MB]
Technical documents
Cognitive Data CodeBook
Download [PDF, 194.84 KB]
PISA 2003 Data Analysis Manual
Download [PDF, 2.51 MB]
subtitle SPSS® Users
Author(s) There are two versions of this manual – one for SPSS® users and one for SAS® users. The OECD recognises the creative work of Christian Monseur in preparing the text for both versions of the manual in collaboration with Sheila Krawchuk and Keith Rust, as
Language English
Description The PISA 2003 Data Analysis Manual is in four parts – the first two sections give a detailed theoretical background and instructions for analysing the data; the third section lists the program codes (syntaxes and the macros), which are needed to carry out the analyses; and the fourth section contains a detailed description of the database.
Table of contents Users’ Guide ..............................................................................................................9
CHAPTER 1
The OECD’s Programme for International Student Assessment ...................................................... 11
An overview of PISA ................................................................................................. 12
What makes PISA unique? ........................................................................................... 13
How the assessment takes place .................................................................................... 15
About this manual ...................................................................................................... 16
CHAPTER 2
Sample Weights ......................................................................................................... 19
Introduction ........................................................................................................... 20
Weights for simple random samples ................................................................................. 21
Sampling designs for education surveys ............................................................................ 22
Why do the PISA weights vary? .................................................................................... 27
Conclusions .............................................................................................................. 30
CHAPTER 3
Replicate Weights ..................................................................................................... 31
Introduction .......................................................................................................... 32
Sampling variance for simple random sampling ................................................................... 32
Sampling variance for two-stage sampling ....................................................................... 38
Replication methods for simple random samples ................................................................. 44
Resampling methods for two-stage samples ....................................................................... 46
The Jackknife for unstratifi ed two-stage sample designs .................................................... 47
The Jackknife for stratifi ed two-stage sample designs .......................................................... 48
The Balanced Repeated Replication method ........................................................................ 49
Other procedures for accounting for clustered samples ...................................................... 51
Conclusions .............................................................................................................. 51
CHAPTER 4
The Rasch Model ................................................................................................... 53
Introduction ........................................................................................................ 54
How can the information be summarised? ...................................................................... 54
The Rasch model for dichotomous items ........................................................................ 56
Other Item Response Theory models .......................................................................... 69
Conclusions ........................................................................................................ 69
CHAPTER 5
Plausible Values .................................................................................................... 71
Individual estimates versus population estimates ............................................................. 72
The meaning of plausible values ................................................................................ 72
Comparison of the effi ciency of Warm Likelihood Estimates, Expected A Posteriori
estimates and plausible values for the estimation of some population statistics ......................... 76
How to perform analyses with plausible values ............................................................... 78
Conclusions ..................................................................................................... 80
CHAPTER 6
Computation of Standard Errors ............................................................................... 81
Introduction ...................................................................................................... 82
The standard error on univariate statistics for numerical variables .................................... 82
The SPSS® macro for computing the standard error on a mean .......................................... 85
The standard error on percentages ............................................................................ 87
The standard error on regression coeffi cients ........................................................... 90
The standard error on correlation coeffi cients .............................................................. 92
Conclusions ....................................................................................................... 93
CHAPTER 7
Analyses with Plausible Values .................................................................................. 95
Introduction ........................................................................................................ 96
Univariate statistics on plausible values ......................................................................... 96
The standard error on percentages with plausible values ...................................................101
The standard error on regression coeffi cients with plausible values .......................................101
The standard error on correlation coeffi cients with plausible values .......................................104
Correlation between two sets of plausible values .............................................................104
A fatal error shortcut ...............................................................................................108
An unbiased shortcut .................................................................................................109
Conclusions .........................................................................................................110
CHAPTER 8
Use of Proficiency Levels ........................................................................................111
Introduction ......................................................................................................112
Generation of the profi ciency levels ...........................................................................112
Other analyses with profi ciency levels .......................................................................118
Conclusions ......................................................................................................121
CHAPTER 9
Analyses with School Level Variables .......................................................................123
Introduction ..................................................................................................124
Limits of the PISA school samples .......................................................................125
Merging the school and student data fi les .............................................................126
Analyses of the school variables ...........................................................................126
Conclusions ..................................................................................................128
CHAPTER 10
Standard Error on a Difference .........................................................................129
Introduction ................................................................................................130
The standard error of a difference without plausible values ..........................................132
The standard error of a difference with plausible values ..............................................137
Multiple comparisons ......................................................................................140
Conclusions .................................................................................................141
CHAPTER 11
OECD Average and OECD Total ...........................................................................143
Introduction .................................................................................................144
Recoding of the database for the estimation of the OECD total and OECD average .............144
Duplication of the data for avoiding three runs of the procedure .....................................147
Comparisons between OECD average or OECD total estimates and a country estimate .........147
Conclusions ...................................................................................................150
CHAPTER 12
Trends ..........................................................................................................153
Introduction ...................................................................................................154
The computation of the standard error for trend indicators on variables other
than performance ...........................................................................................155
The computation of the standard error for trend indicators on performance variables .........158
Conclusions .................................................................................................164
CHAPTER 13
Multilevel Analyses ........................................................................................165
Introduction ................................................................................................166
Simple linear regression ....................................................................................166
Simple linear versus multilevel regression analyses .....................................................170
Fixed effect versus random effect ....................................................................173
Some examples with SPSS® ..............................................................................174
Limitations of the multilevel model in the PISA context .................................................190
Conclusions ..................................................................................................192
CHAPTER 14
Other Statistical Issues ....................................................................................193
Introduction ................................................................................................194
Analyses by quarters ....................................................................................194
The concepts of relative risk and attributable risk .....................................................198
Instability of the relative and attributable risks .......................................................200
Computation of the relative risk and attributable risk ................................................201
Conclusions .................................................................................................201
CHAPTER 15
SPSS® Macros ..............................................................................................203
Introduction .................................................................................................204
Structure of the SPSS® macros ........................................................................204
Appendix 1: PISA 2003 International Database ...................................................237
Appendix 2: Student Questionnaire .............................................................251
Appendix 3: Educational Career Questionnaire .....................................................265
Appendix 4: Information Communication Technology (ICT) Questionnaire ..........................267
Appendix 5: School Questionnaire ..........................................................................271
Appendix 6: Student Questionnaire Data File Codebook ............................................285
Appendix 7: School Questionnaire Data File Codebook ............................................331
Appendix 8: Student Cognitive Test Data File Codebook .........................................345
Appendix 9: Student and School Questionnaire Indices .........................................375
Appendix 10: Scores Allocated to the Items .........................................................401
References ...........................................................................................407
Download http://catalog.ihsn.org//catalog/4921/download/60998
PISA 2003 Main Study: The National Project Manager’s Manual
Download [PDF, 1.65 MB]
Author(s) Australian Council For Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Services (ETS) National Institute for Educational Policy Research (NIER, Japan) Westat
Date 2003-01-01
Language English
Download http://catalog.ihsn.org//catalog/4921/download/60999
PISA 2003 Technical Report
Download [PDF, 4.81 MB]
Author(s) PISA is a collaborative effort by the participating countries, and guided by their governments on the basis of shared policy-driven interests. Representatives of each country form the PISA Governing Board which decides on the assessment and reporting of
Language English
Description This report contains a description of the theoretical underpinning of the complex techniques used to create the PISA 2003 database, which contains information on over a quarter of a million students from 41 countries.
Table of contents Foreword ............................................................................................................ 3
Chapter 1. The Programme for International Student Assessment: An overview ........................ 7
Reader’s Guide ...........................................................................................................13
Chapter 2. Test design and test development ..................................................................15
Chapter 3. The development of the PISA context questionnaires ..........................................33
Chapter 4. Sample design .........................................................................................45
Chapter 5. Translation and cultural appropriateness of the test and survey material ...................67
Chapter 6. Field operations .......................................................................................81
Chapter 7. Monitoring the quality of PISA ................................................................... 101
Chapter 8. Survey weighting and the calculation of sampling variance .................................. 107
Chapter 9. Scaling PISA cognitive data ........................................................................ 119
Chapter 10. Coding reliability studies .......................................................................... 135
Chapter 11. Data cleaning procedures .......................................................................... 157
Chapter 12. Sampling outcomes ................................................................................. 165
Chapter 13. Scaling outcomes ................................................................................... 185
Chapter 14. Outcomes of coder reliability studies ........................................................... 217
Chapter 15. Data adjudication ................................................................................... 235
Chapter 16. Proficiency scale construction .................................................................... 249
Chapter 17. Scaling procedures and construct validation of context questionnaire data ............... 271
Chapter 18. International database .............................................................................. 321
References ......................................................................................................... 329
Download http://catalog.ihsn.org//catalog/4921/download/61000
Project Manager's Manual
Download [PDF, 731.61 KB]
Author(s) Australian Council For Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Services (ETS) National Institute for Educational Policy Research (NIER, Japan) Westat
Date 2005-12-01
Language English
Table of contents LIST OF EXHIBITS ...................................................................................5
ACRONYMS USED IN PISA ........................................................................6
CHAPTER ONE: INTRODUCTION................................................................8
THE MAIN STUDY NATIONAL PROJECT MANAGER’S MANUAL.................................8
CHECKLIST OF NPM ACTIVITIES ..................................................................8
KEY DOCUMENTS FOR THE PISA 2006 MAIN STUDY..........................................12
COMMUNICATIONS AND THE PISA WEBSITE ..................................................13
CHAPTER TWO: PRELIMINARY MAIN STUDY PREPARATION ACTIVITIES......................19
REVIEW YOUR NATIONAL DATA COLLECTION PLAN .........................................19
NEGOTIATE YOUR MAIN STUDY SCHOOL SAMPLING PLAN ...................................21
OBTAIN SCHOOL CO-OPERATION.................................................................22
ATTEND THE NPM MEETING IN MILDURA ........................................................23
PREPARE THE MAIN STUDY TEST INSTRUMENTS AND SURVEY MATERIAL...................23
NOMINATE PISA QUALITY MONITORS (PQMS)....................................................23
CHAPTER THREE: PREPARATION OF TEST INSTRUMENTS, MANUALS AND
MATERIALS.............................................................................................25
MATERIALS TO BE REVISED..........................................................................26
NEGOTIATE A VERIFICATION SCHEDULE WITH THE CONSORTIUM............................27
PREPARE THE LINK ITEMS ..........................................................................29
PREPARE NATIONAL SCHOOL COORDINATOR (SC) AND TEST ADMINISTRATOR (TA) MANUALS............30
PREPARE A ‘UH’ VERSION OF THE TA OR SA MANUAL IF IMPLEMENTING THE UH BOOKLET ................34
PREPARE BOOKLET COVERS AND OTHER ‘COMMON PARTS’ – E.G. GENERAL DIRECTIONS, EFFORT
SURVEY................................................................................................34
PREPARE THE NEW SCIENCE UNITS ...............................................................35
PREPARE NATIONAL QUESTIONNAIRES .......................................................37
SUBMITTING MATERIALS.........................................................................40
PROVIDING COPIES OF YOUR FINAL NATIONAL VERSION(S) TO ACER.................41
CHAPTER FOUR: SELECTION OF THE STUDENT SAMPLE.....................................43
INTRODUCTION TO STUDENT SAMPLING......................................................43
SEND THE INSTRUCTIONS FOR PREPARING A LIST OF ELIGIBLE STUDENTS TO SCHOOLS ........................43
SELECT STUDENT SAMPLES .......................................................................46
CHAPTER FIVE: ACTIVITIES PRIOR TO THE TESTING PERIOD ..............................49
INSTALL MAIN STUDY VERSION OF KEYQUEST ...............................................49
HIRE AND TRAIN TEST ADMINISTRATORS...................................................49
MAKE ARRANGEMENTS FOR THE VISIT OF THE NATIONAL CENTRE QUALITY MONITOR (NCQM)..........50
CHAPTER SIX: ASSEMBLY OF TEST INSTRUMENTS AND MATERIALS .......................51
INTRODUCTION .....................................................................................51
TEST ASSEMBLY ......................................................................................51
QUESTIONNAIRE ASSEMBLY ...................................................................52
CHAPTER SEVEN: PRINTING, PACKAGING AND SHIPPPING.................................54
OVERVIEW...........................................................................................54
PRINTING PISA MATERIALS .........................................................................54
PACKAGING AND SHIPPING PISA MATERIALS ..................................................55
CHAPTER EIGHT: ACTIVITIES DURING AND FOLLOWING THE TESTING PERIOD .......57
COORDINATE THE ACTIVITIES OF TAS .........................................................57
MONITOR SCHOOL AND STUDENT RESPONSE RATES ..........................................57
REVIEW CODING AND DATA ENTRY OPERATIONS, RECRUIT STAFF .......................58
ORGANISE FOR APPROPRIATE STAFF TO ATTEND CODER TRAINING MEETING ............58
RECEIVE MATERIALS FOLLOWING THE ASSESSMENT ........................................58
SUBMIT SCHOOL TRACKING FORM (SAMPLING FORM 12) SHOWING THE PARTICIPATION STATUS OF
SCHOOLS ..........................................................................................59
OVERSEE CODING AND DATA ENTRY OPERATIONS.........................................59
AN OVERVIEW OF DATA MANAGEMENT .....................................................59
SUBMIT DATABASE TO ACER..................................................................60
HAVE A DATA MANAGER AVAILABLE TO RESPOND TO QUERIES DURING DATA CLEANING ......................61
ARCHIVE YOUR MAIN STUDY MATERIALS ..................................................61
COMPLETE AND SUBMIT THE MAIN STUDY REVIEW SURVEY...............................62
SUBMIT A SAMPLE OF YOUR BOOKLETS TO THE CONSORTIUM FOR THE INTERNATIONAL CODING
REVIEW (ICR)......................................................................................62
CHAPTER NINE: MAIN STUDY CODING PROCEDURES.....................................63
INTRODUCTION .................................................................................63
OVERVIEW OF CODING REQUIREMENTS...................................................63
RECRUIT CODERS...............................................................................63
PROCESSING BOOKLETS AND PREPARING FOR CODING...............................65
PREPARING FOR CODER TRAINING ...........................................................67
TRAINING......................................................................................68
CODING.....................................................................................69
MONITORING CODER CONSISTENCY DURING CODING...................................70
CODER QUERY SERVICE ....................................................................70
THE RECORDING OF CODES .............................................................70
SINGLE CODING DESIGN ..................................................................72
MULTIPLE CODING OF BOOKLETS SET ASIDE FOR RELIABILITY STUDY ............75
ALTERNATIVE SINGLE- AND MULTIPLE- CODING DESIGNS..............................78
PISA STANDARDS IN RELATION TO CODING ........................................78
APPENDIX ONE: SAMPLE CONFIDENTIALITY AGREEMENT FOR MARKERS ........81
APPENDIX TWO: MULTIPLE CODING RELIABILITY SHEETS............................83
APPENDIX THREE: ALLOCATION OF TEST ITEMS TO CLUSTERS AND BOOKLETS ......91
Download http://catalog.ihsn.org//catalog/4921/download/61001
PISA Quality Monitor Manual
Download [PDF, 824.17 KB]
Author(s) Australian Council For Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Services (ETS) National Institute for Educational Policy Research (NIER, Japan) Westat
Date 2002-12-01
Language English
Description This manual is to be used by PISA Quality Monitors for the main study data collection for the Programme for International Student Assessment (PISA) project in the year 2003.
Table of contents ACRONYMS........................................................................ 4
CONTACTS AT ACER:............................................................. 5
ABOUT PISA QUALITY MONITORS ........................................... 6
OVERVIEW ...................................................................... 6
About PISA ................................................................... 6
About PISA Quality Monitors .............................................. 7
ACTIONS TO BE TAKEN BY PISA QUALITY MONITORS ................ 9
PAYMENT OF PISA QUALITY MONITORS................................... 17
OTHER ASPECTS OF THE PISA QUALITY MONITOR ROLE................. 19
HYPOTHETICAL SITUATIONS FOR CONSIDERING WHEN A PISA
QUALITY MONITOR MIGHT INTERVENE IN A TESTING SESSION ..................21
HONORARIA AND CONFIDENTIALITY AGREEMENT .......................25
PISA QUALITY MONITOR DATA COLLECTION SHEET.....................27
1 PREPARATION FOR THE ASSESSMENT .................................. 31
2 TEST SESSION ACTIVITIES ............................................... 33
Introducing the study ........................................................34
Distributing the materials.....................................................35
Beginning the Test Session ..................................................37
The Cognitive Session .......................................................39
Preparing for the Student Questionnaire Session .........................46
Administering the student questionnaire..................................47
OTHER COMMENTS ......................................................... 51
3 GENERAL QUESTIONS CONCERNING THE ASSESSMENT ............... 53
4 INTERVIEW WITH THE SCHOOL CO-ORDINATOR..................... 57
PISA QUALITY MONITOR GENERAL OBSERVATIONS SHEET .............63
Download http://catalog.ihsn.org//catalog/4921/download/61002
School Coordinator's Manual
Download [PDF, 428.51 KB]
Author(s) Australian Council For Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Services (ETS) National Institute for Educational Policy Research (NIER, Japan) Westat
Date 2002-10-01
Language English
Download http://catalog.ihsn.org//catalog/4921/download/61003
Take the Test: Sample Questions from OECD’s PISA Assessments
Download [PDF, 2.89 MB]
Author(s) This report was compiled by Susanne Salz and Diana Toledo Figueroa, with editorial assistance from Elisabeth Villoutreix and Juliet Evans.
Language English
Description This report brings together all the publicly available questions in reading, mathematics and science and, together with the PISA assessment frameworks, gives a solid overview of the PISA test.
Table of contents FOREWORD ......................................................................................................3
CHAPTER 1 Introduction to the Programme for International Student Assessment (PISA).........11
The aims of PISA............................................................................................12
The PISA surveys..........................................................................................13
The PISA results................................................................................................13
The PISA questions ..........................................................................................13
CHAPTER 2 Reading sample tasks..........................................................................15
Reading unit 1: Lake Chad.................................................................................17
Reading unit 2 : Flu...........................................................................................19
Reading unit 3 : Graffiti.....................................................................................22
Reading unit 4 : Labour.....................................................................................24
Reading unit 5 : Plan International.........................................................................26
Reading unit 6 : Police.......................................................................................28
Reading unit 7 : Runners....................................................................................30
Reading unit 8 : Gift..........................................................................................32
Reading unit 9 : Amanda and the Duchess.................................................................37
Reading unit 10 : Personnel................................................................................41
Reading unit 11 : New Rules...............................................................................43
Reading unit 12 : Moreland...................................................................................45
Reading unit 13 : Warranty...................................................................................46
Reading unit 14 : A Just Judge..............................................................................49
Reading unit 15 : In Poor Taste.............................................................................53
Reading unit 16 : Bullying..................................................................................54
Reading unit 17 : Bees........................................................................................56
Answers .......................................................................................................58
Lake Chad scoring...........................................................................................58
Flu scoring....................................................................................................60
Graffiti scoring...............................................................................................63
Labour scoring................................................................................................67
Plan International scoring.....................................................................................69
Police scoring................................................................................................71
Runners scoring.............................................................................................72
Gift scoring..................................................................................................74
Amanda and the Duchess scoring..........................................................................79
Personnel scoring..............................................................................................84
New rules scoring............................................................................................85
Moreland scoring.............................................................................................86
Warranty scoring...........................................................................................87
Just Judge scoring...........................................................................................88
In Poor Taste scoring..........................................................................................92
Bullying scoring...............................................................................................93
Bees scoring...............................................................................................94
CHAPTER 3 Mathematics sample tasks...................................................................97
Mathematics unit 1: Farms................................................................................100
Mathematics unit 2 : Walking...........................................................................101
Mathematics unit 3 : Apples...........................................................................102
Mathematics unit 4 : Cubes.........................................................................104
Mathematics unit 5 : Continent Area..............................................................105
Mathematics unit 6 : Growing Up....................................................................106
Mathematics unit 7 : Speed of Racing Car...........................................................107
Mathematics unit 8 : Triangles........................................................................109
Mathematics unit 9 : Robberies.....................................................................110
Mathematics unit 10 : Carpenter..................................................................111
Mathematics unit 11 : Internet Relay Chat........................................................112
Mathematics unit 12 : Exchange Rate............................................................113
Mathematics unit 13 : Exports.....................................................................114
Mathematics unit 14 : Coloured Candies...........................................................115
Mathematics unit 15 : Science Tests..................................................................115
Mathematics unit 16 : Bookshelves..................................................................116
Mathematics unit 17 : Litter...........................................................................116
Mathematics unit 18 : Earthquake...................................................................117
Mathematics unit 19 : Choices.......................................................................117
Mathematics unit 20 : Test Scores...................................................................118
Mathematics unit 21 : Skateboard...................................................................119
Mathematics unit 22 : Staircase.....................................................................120
Mathematics unit 23 : Number Cubes................................................................121
Mathematics unit 24 : Support for the President.................................................122
Mathematics unit 25 : The Best Car.................................................................123
Mathematics unit 26 : Step Pattern..................................................................124
Mathematics unit 27 : Lichen.........................................................................125
Mathematics unit 28 : Coins........................................................................126
Mathematics unit 29 : Pizzas........................................................................127
Mathematics unit 30: Shapes........................................................................127
Mathematics unit 31 : Braking......................................................................128
Mathematics unit 32 : Patio.........................................................................129
Mathematics unit 33 : Drug Concentrations.......................................................130
Mathematics unit 34 : Building Blocks...............................................................132
Mathematics unit 35 : Reaction Time...............................................................134
Mathematics unit 36 : Water Tank...............................................................135
Mathematics unit 37 : Spring Fair..................................................................136
Mathematics unit 38 : Swing........................................................................137
Mathematics unit 39 : Student Heights............................................................138
Mathematics unit 40 : Payments by Area.........................................................139
Mathematics unit 41 : Shoes for Kids...............................................................140
Mathematics unit 42 : Table Tennis Tournament..................................................141
Mathematics unit 43 : Lighthouse...................................................................142
Mathematics unit 44 : Decreasing CO2 Levels....................................................144
Mathematics unit 45 : Twisted Building............................................................146
Mathematics unit 46 : Heartbeat....................................................................148
Mathematics unit 47 : Space Flight.................................................................149
Mathematics unit 48 : Rock Concert................................................................149
Mathematics unit 49 : Moving Walkways............................................................150
Mathematics unit 50 : Postal Charges.............................................................151
Answers ..............................................................................................153
Farms scoring........................................................................................153
Walking scoring.........................................................................................153
Apples scoring.........................................................................................155
Cubes scoring...........................................................................................157
Continent Area scoring.................................................................................158
Growing Up scoring....................................................................................159
Speed of Racing Car scoring...........................................................................161
Triangles scoring.......................................................................................162
Robberies scoring.......................................................................................162
Carpenter scoring.......................................................................................164
Internet Relay Chat scoring............................................................................164
Exchange Rate scoring....................................................................................165
Exports scoring.........................................................................................166
Coloured Candies scoring................................................................................166
Science Tests scoring....................................................................................167
Bookshelves scoring.......................................................................................167
Litter scoring...............................................................................................167
Earthquake scoring......................................................................................168
Choices scoring..........................................................................................168
Test Scores scoring.......................................................................................168
Skateboard scoring.....................................................................................169
Staircase scoring.......................................................................................169
Number Cubes scoring................................................................................170
Support for the President scoring...................................................................170
The Best Car scoring....................................................................................170
Step Pattern scoring......................................................................................171
Lichen scoring..............................................................................................171
Coins scoring...........................................................................................173
Pizzas scoring.......................................................................................173
Shapes scoring.......................................................................................174
Braking scoring .......................................................................................175
Patio scoring.......................................................................................176
Drug Concentrations scoring......................................................................177
Building Blocks scoring............................................................................178
Reaction Time scoring............................................................................179
Water Tank scoring..............................................................................179
Spring Fair scoring.................................................................................179
Swing scoring.....................................................................................180
Student Heights scoring.........................................................................180
Payments by Area scoring.....................................................................180
Shoes for Kids scoring..........................................................................181
Table Tennis Tournament scoring............................................................181
Lighthouse scoring............................................................................181
Decreasing co2 Levels scoring...............................................................182
Twisted Building scoring.......................................................................183
Heartbeat scoring...............................................................................185
Space Flight scoring.............................................................................185
Rock Concert scoring..........................................................................185
Moving Walkways scoring....................................................................186
Postal Charges scoring........................................................................186
CHAPTER 4 Science sample tasks.............................................................187
Science unit 1 : Semmelweis’ Diary ...........................................................191
Science unit 2 : Ozone.........................................................................193
Science unit 3 : Daylight .....................................................................195
ScienCe unit 4 : Cloning.......................................................................197
Science unit 5 : Greenhouse.................................................................199
Science unit 6 : Clothes......................................................................201
ScienCe unit 7 : The Grand Canyon.........................................................203
Science unit 8 : Sunscreens.....................................................................205
Science Unit 9 : Mary Montagu...............................................................207
Science unit 10 : Acid Rain.....................................................................209
Science unit 11 : Physical Exercise............................................................211
Science unit 12 : Genetically Modified Crops................................................212
Science unit 13 : Biodiversity ..............................................................214
Science unit 14 : Buses........................................................................216
Science unit 15 : Climate Change .........................................................217
Science unit 16 : Flies .......................................................................218
Science unit 17 : Calf Clones ...............................................................219
Science unit 18 : Corn ........................................................................220
Science Unit 19 : Fit for Drinking............................................................222
Science unit 20 : Tooth Decay..............................................................224
Science unit 21 : Hot Work..................................................................227
Science unit 22 : Mousepox...................................................................228
Science Unit 23 : Stickleback Behaviour...................................................230
Science unit 24 : Tobacco Smoking............................................................233
Science unit 25 : Starlight.....................................................................235
Science unit 26 : Ultrasound..................................................................236
Science unit 27 : Lip Gloss.....................................................................238
Science unit 28 : Evolution...................................................................240
Science unit 29 : Bread Dough...............................................................242
Science unit 30 : Transit of Venus...........................................................245
Science unit 31 : Health Risk?...............................................................247
Science unit 32 : Catalyctic Converter........................................................248
Science unit 33 : Major Surgery...............................................................250
Science unit 34 : Wind Farms.................................................................252
Answers ..........................................................................................254
Semmelweis’ Diary scoring..................................................................254
Ozone scoring..................................................................................256
Daylight scoring.................................................................................259
Cloning scoring..................................................................................262
Greenhouse scoring............................................................................262
Clothes scoring.................................................................................266
The Grand Canyon scoring..................................................................266
Sunscreens scoring.............................................................................267
Mary Montagu scoring...........................................................................268
Acid Rain scoring..................................................................................269
Physical Exercise scoring........................................................................271
GM Crops scoring..................................................................................272
Biodiversity scoring................................................................................272
Buses scoring.....................................................................................273
Climate Change scoring...................................................................273
Flies scoring.................................................................................274
Calf Clones scoring.......................................................................276
Corn scoring....................................................................................276
Fit for Drinking scoring.......................................................................277
Tooth Decay scoring.........................................................................279
Hot Work scoring...........................................................................279
Mousepox scoring..........................................................................280
Stickleback Behaviour scoring.............................................................280
Tobacco Smoking scoring..................................................................281
Starlight scoring.............................................................................282
Ultrasound scoring.........................................................................283
Lip Gloss scoring............................................................................284
Evolution scoring...........................................................................284
Bread Dough scoring......................................................................285
Transit of Venus scoring...................................................................286
Health Risk scoring.........................................................................287
Catalyctic Converter scoring...............................................................287
Major Surgery scoring........................................................................289
Wind Farms scoring........................................................................290
ANNEX A PISA scales.......................................................................293
ANNEX B Country results for PISA 2000, 2003 and 2006 questions...................295
ANNEX C List of questions with codes and sources.....................................307
Reading sample tasks.....................................................................308
Mathematics sample tasks................................................................310
Science sample tasks.....................................................................313
ANNEX D Guide to further reading.................................................317
Download http://catalog.ihsn.org//catalog/4921/download/61004
School Sampling Preparation Manual
Download [PDF, 595.43 KB]
Author(s) Australian Council For Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Services (ETS) National Institute for Educational Policy Research (NIER, Japan) Westat
Date 2002-06-01
Language English
Table of contents 1. INTRODUCTION .......................................................... 3
1.1 SAMPLING IN PISA................................................... 3
1.2 RELATIONSHIP OF FIELD TRIAL SAMPLING PROCEDURES
TO MAIN STUDY SAMPLING PROCEDURES ........................... 3
1.3 THE RESPONSIBILITIES OF NATIONAL PROJECT
MANAGERS AND THE CONSORTIUM IN SAMPLING SCHOOLS......... 4
1.4 A NOTE ABOUT VERSIONS............................................ 6
1.5 CHANGES FROM PISA 2000 .......................................... 6
2. OVERVIEW OF SAMPLE DESIGN ...................................... 7
2.1 SURVEY GOALS......................................................... 7
2.2 SAMPLING GOALS AND EXPECTATIONS............................ 7
2.3 SAMPLE DESIGN OVERVIEW.......................................... 7
3. STEP 1–TARGET POPULATION DEFINITION......................... 9
3.1 INTERNATIONAL DESIRED TARGET POPULATION ................ 9
3.2 NATIONAL DESIRED TARGET POPULATION ....................... 12
3.3 NATIONAL DEFINED TARGET POPULATION........................ 14
3.3.1 Exclusions............................................................. 14
3.3.2 School-Level Exclusions ............................................ 14
3.3.3 Within-School Exclusions ......................................... 15
4. STEP 2 – NATIONAL SAMPLING PLAN................................. 19
4.1 INTRODUCTION........................................................ 19
4.2 SAMPLE DESIGN FRAMEWORK......................................... 19
4.3 COUNTRY-SPECIFIC ISSUES IN SAMPLE DESIGN .................. 20
4.3.1 Coefficient of Intra-class Correlation............................... 20
4.3.2 Samples of Students in a Particular Grade.......................... 21
4.4 PARTICIPATION RATES .............................................. 21
4.4.1 Schools............................................................... 22
4.4.2 Students............................................................... 24
4.4.3 Reporting Participation Rates ..................................... 24
5. STEP 3 – THE SCHOOL SAMPLING FRAME.......................... 25
5.1 INTRODUCTION....................................................... 25
5.2 SCHOOL-LEVEL SAMPLING FRAME .................................. 25
5.3 AREA-LEVEL SAMPLING FRAME ................................... 26
5.3.1 Area-Level ........................................................ 26
5.3.2 School Level......................................................... 27
5.4 OTHER TYPES OF SAMPLING FRAMES .......................... 28
5.5 EXCLUDING SCHOOLS FROM THE SAMPLING FRAME ......... 29
5.6 STRATIFICATION .................................................. 29
5.6.1 Overview.......................................................... 29
5.6.2 Explicit Stratification............................................ 30
5.6.3 Implicit Stratification......................................... 31
5.6.4 Stratification Guidelines ......................................... 31
5.7 TREATMENT OF SMALL SCHOOLS................................. 31
5.7.1 Stratification of Small Schools............................... 32
5.7.2 Sample Sizes for the Small School Strata.................. 33
5.8 ASSIGNING A MEASURE OF SIZE (MOS) TO EACH SCHOOL ....... 36
5.9 SAMPLING FRAME STATISTICS ......................................36
6. STEP 4 – THE SAMPLE OF SCHOOLS.................................... 41
6.1 THE SCHOOL SAMPLE .................................................. 41
6.2 REPLACEMENT SCHOOLS................................................ 41
6.3 SCHOOL ID’S............................................................. 42
6.4 THE SCHOOL TRACKING FORM ....................................... 43
6.4.1 Sampled Schools ...................................................... 43
6.4.2 Replacement Schools .................................................. 44
CHECKLIST OF SCHOOL SAMPLING ACTIVITIES ......................... 46
APPENDIX A..................................................................... 47
APPENDIX B..................................................................... 73
APPENDIX C..................................................................... 87
C.1 OVERVIEW ................................................................. 87
C.2 SCHOOL SAMPLE ALLOCATION OVER EXPLICIT STRATA ........... 87
C.3 SORTING THE SAMPLING FRAME........................................ 88
C.4 DETERMINING WHICH SCHOOLS TO SAMPLE ......................... 90
C.4.1 Calculating the Sampling Interval ..................................... 90
C.4.2 Generating a Random Start ............................................ 91
C.4.3 Calculating the Selection Numbers................................... 91
C.5 IDENTIFYING THE SAMPLED SCHOOLS.................................. 93
C.6 IDENTIFYING REPLACEMENT SCHOOLS ................................. 94
C.7 ASSIGNING SCHOOL ID’S ............................................... 95
Download http://catalog.ihsn.org//catalog/4921/download/61005
School Questionnaire CodeBook
Download [PDF, 72.99 KB]
Student Questionnaire CodeBook
Download [PDF, 114.15 KB]
Test Administrator's Manual
Download [PDF, 942.24 KB]
Author(s) Australian Council For Educational Research (ACER) Netherlands National Institute for Educational Measurement (CITO group) Educational Testing Services (ETS) National Institute for Educational Policy Research (NIER, Japan) Westat
Date 2003-01-01
Language English
Table of contents 1. INTRODUCTION .................................................................................. 4
1.1 OECD PROGRAMME FOR INTERNATIONAL STUDENT ASSESSMENT (PISA) ............. 4
1.2 THE ROLE OF THE TEST ADMINISTRATOR .................................................. 4
1.3 CHECKLIST OF ACTIVITIES.................................................................... 6
2. PREPARING FOR THE ASSESSMENT .......................................................... 7
2.1 ATTENDING THE PISA TEST ADMINISTRATOR TRAINING ............................... 7
2.2 WORKING WITH THE SCHOOL CO-ORDINATOR ......................................... 7
2.3 RECEIVING AND SECURING ASSESSMENT MATERIALS .................................. 8
2.4 UPDATING THE STUDENT TRACKING FORM .............................................. 9
2.5 STUDY PROGRAMMES: COLUMN 7 OF THE STUDENT TRACKING FORM ............... 10
2.6 SPECIAL EDUCATION NEEDS AND INCLUSION: COLUMNS 8 AND 9 OF THE STUDENT TRACKING FORM .............. 10
2.7 ARRANGING FOR ASSISTANCE (OPTIONAL) ............................................... 12
2.8 CONFIRMING ASSESSMENT PLANS......................................................... 12
3. CONDUCTING THE ASSESSMENT SESSION .............................................. 13
3.1 ARRIVING ON ASSESSMENT DAY ......................................................... 13
3.2 PREPARING THE ASSESSMENT BOOKLETS ................................................ 13
3.3 DISTRIBUTING MATERIALS TO THE STUDENTS ......................................... 14
3.4 TIMING THE ASSESSMENT SESSIONS ..................................................... 14
3.5 READING THE SCRIPT ..................................................................... 15
3.6 SUPERVISING THE SESSION .............................................................. 15
3.7 ENDING THE SESSION ..................................................................... 15
4. CONCLUDING ASSESSMENT ACTIVITIES ................................................ 16
4.1 RECORDING PARTICIPATION STATUS ON THE STUDENT TRACKING FORM (STF).... 16
4.2 DETERMINING IF A FOLLOW-UP SESSION IS NEEDED................................... 16
4.3 COMPLETING THE SESSION REPORT FORM ............................................... 17
4.4 COLLECTING THE SCHOOL QUESTIONNAIRE ............................................17
4.5 PACKING AND SHIPPING THE ASSESSMENT MATERIALS ...................................................... 17
APPENDIX 1. TEST ADMINISTRATOR'S SCRIPT TO BE READ FOR THE SESSIONS........ 18
APPENDIX 2. OECD/PISA SESSION REPORT FORM ............................................. 34
APPENDIX 3. RETURN SHIPMENT FORM ........................................................ 37
APPENDIX 4. EXPLANATORY NOTES ON QUESTIONNAIRE ITEMS ........................... 38
APPENDIX 5. EXAMPLE STF COMPLETED WITH SEN AND INCLUSION STATUS .......... 46
APPENDIX 6. EXAMPLE STF COMPLETED WITH PARTICIPATION STATUS.................. 50
Download http://catalog.ihsn.org//catalog/4921/download/61008
Technical Standards for the Implementation of PISA 2003
Download [PDF, 293.91 KB]
Language English
Table of contents (NOTE BY THE SECRETARIAT) ....................................................................................................... 2
TECHNICAL STANDARDS FOR THE IMPLEMENTATION OF PISA 2003....................................... 4
Sampling.......................................................................................................................................... 4
Translation and translation verification .............................................................................................. 5
Selection of the translators ............................................................................................................. 6
Submission of questionnaire adaptations and modifications for approval........................................... 6
Submission of material for translation verification ........................................................................... 7
Test Administration .......................................................................................................................... 7
Selection of Test Administrators..................................................................................................... 8
Training of Test Administrators...................................................................................................... 8
Security of the material .................................................................................................................. 8
Testing session .............................................................................................................................. 8
Quality monitoring ........................................................................................................................... 9
Site visits and training of school quality monitors ............................................................................ 9
Visits by School Quality Monitors .................................................................................................10
PISA International Standard Indicators ..........................................................................................10
Coding ............................................................................................................................................10
Selection of the markers................................................................................................................10
Single marking .............................................................................................................................11
Multiple marking..........................................................................................................................11
PISA International Standard Indicators ..........................................................................................11
Data entry and data submission.........................................................................................................11
Material to be submitted ...............................................................................................................11
Data cleaning ................................................................................................................................12
PISA International Standard Indicators ..........................................................................................12
Download http://catalog.ihsn.org//catalog/4921/download/61009
Back to Catalog